Static Sites and API docs with Sphinx, Python, and Markdown Course

Sphinx.
Sphinx Sphinx.
Hi everybody.
Welcome to static websites with Sphinx and Markdown.
A collaboration between Jetbrains and Talk Python training.
We all know Sphinx for documentation.
We all know Markdown for writing content, But the Sphinx is usually meant writing docs in restructured text.
Not Markdown.
Not anymore.
The MyST parser, from executable bugs brings markdown to Sphinx.
In This course we'll show you how to add MyST to Sphinx, not just for documentation, but for a static website.
Sphinx has a lot of really unique power and MyST has found a way to express that in kind of a markdown friendly syntax.
I've always been a big fan of the Ideas in Sphinx.
And as we go through this course, I'll talk about the why as well as the what?
Let's get started by setting the scene.

This course on static websites with Sphinx and Markdown needs a scenario, so we'll simulate building the website for a fictional company named Schlock Chain.
The company has obviously a lot of marketing it wants to do then, otherwise, how are you going to spend the venture capitalist funding?
It also has some code with obviously patents that it wants to show.
The VC's want a website.
The Founders aren't really technical, so they want to use Markdown.
Thus we're going to use Sphinx with Markdown, thanks to MyST, and we're gonna show building out some content, customizing and adding a site design.

Let's start from, well, the beginning an empty directory into which we will slowly build a website first by making a new Python project.
I'll make a directory.
I'll name it after the fictional company's name.
CD into it.
We're going to use MacOS.
Linux.
WSL command shell syntax in this tutorial.
Since Python recommends using a virtual environment for isolation, we'll go ahead and make one.
I'll use the Python3 syntax and I like to store my virtual environments in the project.
I will then use the command to activate this terminal so that its path is first, which then lets me run the pip that is in this virtual environment What we always have to do is upgrade our pip.
As soon as we make a new one, take advantage of any security updates and stuff like that.
And with that in place, our new blank Python project is now ready for a Sphinx installation.

We have a blank Python project.
We're now ready for our Sphinx installation.
What we'll do is we'll during this project will work methodically by having a requirements.txt file that we put our website apps requirements into and then we'll use pip to install from there all from within our favorite editor.
For this course I'll be using the free and open source PyCharm community edition but your choice of editor shouldn't matter too much use what you're comfortable with.
Here I have my requirements.txt file open.
I've had one entry to it called Sphinx and my editor reminds me that my virtual environment does not have this installed.
So I'll sit in a terminal that has this virtual environment activated, which is what we did in the previous step so that I get the pip from the virtual environment.
And now when I run pip install, I'm going to point it at the requirements.txt file which contains Sphinx, which will then go and get Sphinx to install as well as all the dependencies that Sphinx has.
Once that's done.
I can kind of confirm that Sphinx was installed correctly by seeing if the path contains one of the Sphinx command line tools.
So I'll run 'which sphin quick start' and you see that my virtual environments bin directory thus indeed contain that console command.

We're all set up Sphinxes installed using a requirements.txt file, Time to make a site.
Sphinx Ships with a site generator command aka scaffold called sphinx-quickstart, which we saw at the end of the previous step.
It's now in the bin directory of our virtual environment.
Let's run and answer some questions and let it generate the beginning of our site.
I'm in the terminal that has the virtual environment activated so I can start typing 'sphinx quickstart' and it will auto complete.
I'll pass it no arguments and just hit enter and let it get to work.
It's going to ask me some questions.
I'll accept the defaults on most of these and this is where I want the build directories kind of the output to go and I'll give it a name, I'll give it the name of this fictional company Schlock chain.
I'll put myself as the author with my email address and then I'll start accepting the defaults on a number of these.
Sphinx can have multiple languages for output and it's now generated some extra files.
So if we go take a look in our project tree, it's generated a build directory, which is the output.
I generally in my tool, I mark this as excluded because I don't need the generated output to be indexed and it's going to have a static directory for things in the static website, like images and CSS files, templates that are used to help generate the site, which we'll see in a subsequent step.
Very importantly, it generated a configuration file for Sphinx, which we will look at in the next step, a sample first document some tools to help build the site, and we still have our requirements.txt file.

Sphinx is a static site generator.
SSG other Static Site Generators, Especially modern ones have nice authoring servers, you type some stuff in the build happens.
And what's really cool is the browser automatically reloads to show you the content you're authoring that helps during the tutorial.
Let's do something similar for Python.
Using the live reload package.
Live reload has a bundled web server and a way of watching files for changes and then running some commands to rebuild things as those changes occur.
It's not perfect.
It's not as good as the pretty sophisticated ones that you see in modern web front ends.
We'll go ahead and give it a try by installing it into our virtual environment.
Back to my editor.
I'm going to go to requirements.txt and add live reload Since it's going to be helpful, I'm gonna go ahead and let it install.
The requirement for me.
Let the tool do it now that I've got live reload in place.
What I wanna do is make a file a Python script that I can run whenever content changes and this Python script needs to have some information in it about the kinds of things to watch are just chosen to be very explicit on it So whenever my content changes, whenever my code changes or whenever my templates or static stuff change, I want to run that command that we just ran in the last section.
So if I run this, I'm gonna, again, I'm gonna run it using my tool and just as a reminder, this particular editor even live reload aren't critical for the learning in it.
And this is the tutorial step instead.
We're just using it for convenience.
So with this in place it tells me it's got a URL.
That I can go and watch my changes on.
And when I click on that it opens my browser showing me the build rendered output that I can use to click around and make changes.
Let's see that in action by going back to my editor and I'm gonna go change the content and for example I will remove this comment at the top when I save You'll see that the live reload is rerunning and it's rebuilding this 'index.rst' file and in my content I don't see any changes because of that because that wasn't visible.
I'll go and add some words save and then you see without me having to reload in the browser I got the updates from my content.

We just edited an '.rst' file and showed how the build can happen automatically and then the browser gets reload.
But that was an RST file.
This course is about Markdown in this step, let's add markdown to our Sphinx site.
We'll do so by using 'MyST' a parser of markdown which works with Sphinx and doc tells to make it kind of transparent to Sphinx.
Whether you're using restructured text or markdown, I'll go ahead and do an install by switching back to my editor and of course I need to go to requirements.txt and here I will add the name of the package and install it.
That isn't enough to get Sphinx to use it.
Just because the packages installed doesn't mean that Sphinx knows it's really they're going to go back to my configuration file.
I'm gonna search for extensions.
I'm going to add the MyST parser as an extension So with that in place our site can now speak both restructured text and markdown, which we will do in next step.

With markdown enabled in our Sphinx site.
Thanks to MyST.
Let's add our first markdown page in my editor, I'll create a new file for an about us page on the Schlock chain website.
This is going to say about us as the title and some text with this text in place.
We see that we get a little bit of a preview of markdown in our tool.
Now when we saved, we got a warning from Sphinx, it's a little bit mis formatted because of the live reload tool.
But the warning said that I've got this document called About Us and it's not included in any doctree.
Now, what does that mean?
In Sphinx Just because you leave a file in that directory doesn't mean it's actually in your site you have to put it into a table of contents somewhere.
We currently have one table of contents.
This is at the top of the site in 'index.rst'.
I will go to that and say about_us to add that document to the doctree.
Now, when it runs it's no longer mad.
And if I go back over to my preview, I see a link to an about us page which I can actually click on and see the rendering of that markdown file right along side the restructured text file.

We made a lot of progress so far in the setup we created a Python project installed some dependencies.
Built a sphinx site using the scaffold added Markdown support, have a live reload server.
Before we make any more progress, let's do a little bit of cleanup and move some of the auto generated craft out of the way.
And also switch our index file from restructured text to markdown.
Now, start in the sphinx configuration file conf.py.
It's got a lot of these comments with all the possible things that you could put into Sphinx configuration file commented out.
We will remove all that and instead, let's just have a small portion that includes the settings that we're actually using.
There.
That's a lot cleaner.
Now, we'll go over to the 'index.rst'.
What we really want is something that's a markdown.
So let's rename it to 'index.md'.
And then for the contents of this file that's converted over to markdown, let's have the title be something about schlock chain and then we're going to replace the toc tree.
That was a restructured text directive with the MyST.
Markdown variation of this.
We'll see a lot more about how directives look and markdown in subsequent sections.
But for now we have changed our site to being 100% markdown.

We have a Python project.
We have Sphinx.
We have a couple of markdown documents.
Let's take a little bit more of a look at what we can do with markdown in Sphinx.
Thanks to MyST.
First, we have to clarify what do we mean when we say markdown, what we really mean is common mark because the word markdown actually has a little bit of a troubled history.
The creator of the word wants to control the usage of it.
Some of the common mark project arose MyST, Which implements marked down for Sphinx more specifically for doc details, says that it implements a super set of common mark markdown.

Let's take a look at some markdown formatting as it surfaces in Sphinx via MyST.
We'll do some simple formatting in this step will do some headings and paragraphs and character styling and lists and things like that.
I have my editor on the screen with my markdown on one side.
The editors markdown preview on the other side and then I've got my live reload server on this side with the web page.
So what we have first is I can add some Headings.
For example, we've got about us.
I could change it to an H3 and as I save you can see the preview shows a smaller one.
This is formatted differently because it is the title of the page.
If I reverted that and said heading 3 and saved.
Once the library loads server ran, you'd see that I get a different heading style.
I can also do some inline styling.
I'll make a new paragraph just by having a blank line.
I say we are a bold company.
Yes, we are and we are innovative.
And these use of the asterix.
The first one is bold as you can see in the preview.
Second one is italics as you can see in the preview when I save in the live reload server runs, we see that the rendering gets the bold and the innovative as well.
Let's say I want to do.
So we just showed headings, we showed in line character styling, we showed paragraphs, let's say I wanted to do.
And inline for example, code I'll paste a little snippet and you see that the back ticks put the code tag and then a colon followed by a blank line And indentation puts a pre tag.
When I say, if you see the rendered output in HTML.
Markdown has indentation which you can use with a sequence of greater than and multiple greater than signs.
We can see in the preview that I got a first level of indentation and then a second level of indentation, which you also see when missed renders to the HTML.
What do markdown lists look like?
I have a bulleted list just by having a dash.
Could also be other kinds of symbols such as an asterix and then a numbered list It is just a series of numbers.
They don't actually have to be in ascending order.
I could put them all as one and it would work.
I see in my preview and the editor and also in my pre in my rendering thanks to MyST rendering into HTML.
That I get ordered lists and numbered lists from HTML.
How about links to URL's.
This is a place where markdown really excels relative to restructure text I'll paste a little snippet and this is going to have a link to the Python homepage.
You can see the URL.
There, the URL.
Is in parenthesis and the link text is in square brackets and when I look at the preview and my editor I see the link text and the fact that it's an actual link that I could click on when I save and Sphinx renders via MyST.
I see that I have a paragraph with link text.
I hover over it and that's the link.
URL.
I don't have to link just to URL's out on the web.
I can link to other resources on my in my Sphinx site.
For example, if I want to link to the homepage from this about us, I can do a path to it.
I could leave off the ./ and when I save it will re render thanks to MyST.
I can also see in my editors preview and as I mouse over here I see that's going to point to 'index.html'.
This starts to bring up some of the coolness of MyST.
Combined with Sphinx.
The Sphinx knows all the pages on the site and knows the title of all the pages in the site.
If you want this link text to just use the title of the site and update Whenever the title changes, you can just leave empty link text.
Now, my tool doesn't know about Sphinx.
So it just removed the link text but over in the rendered site with MyST and Markdown, it supplied link text from the title of the thing I like to.

Markdown really is a simple syntax, gets a lot of things, right.
One of the things that really gets right is images.
Let's take a look at how you can use markdown image syntax for your images in Sphinx.
I'm back in the About us document and I'd like to add a little section.
It says something cool about the usage of Python, Schlockchain and I want to include the Python logo and I want to do that just after this paragraph.
So I'll put a blank line in and I'll paste in some text.
And this looks a lot like the link syntax that we just saw from Markdown.
It's got the URL in parenthesis in this case it's the URL to the Python logo as a PNG file and it's got some text in square brackets.
In this case it will be the alt text for an image But we're telling Mark down this is an image by putting the exclamation point, the exclamation point in front.
I get a preview in my tool that shows the rendered image and when I save and Sphinx runs MyST, converts that into an image that is displayed in line using the URL to.
That logo.
Now, that's going to have visitors retrieve the image from the Python site every time maybe you want to have a local image.
Let's do that instead.
Let's go to the terminal and let's run 'wget' to go download the Python logo to something local and now what I can do instead of providing the full URL.
I can just say get Python-logo, which is here in my file system as Python-logo, right beside the markdown file that we're editing and when I save my tool previews the markdown correctly, but also it still works when rendered in Sphinx.
in markdown more specifically what it's going to do is take that file that's on disk here and it's going to copy it into the build output into the target of HTML that we're building towards.
And there's now a subdirectory called images with all the images They're part of our content.
And when I look in there I will see this file Python-logo, the 'AboutUs.html' file that was just built We'll have a URL that's relative to go up and down into images to point at that logo file just to ramp up and show one last part of this markdown syntax, If I made a mistake in the file name, we'll see that the all text is displayed in place of the image.
This is the simple image syntax using the existing markdown image syntax as we'll see in later sections, MyST has a much more interesting, much more useful, much more powerful syntax that taps into the underlying Sphinx directive for images.

Okay, big deal.
A static site generator.
They can use markdown syntax.
There's hundreds if not thousands of these ones probably being written right now.
We'll show in the coming sections the rich engine.
Thanks to Sphinx underneath this markdown syntax.
But even for these simple formatting cases, there's a lot more going on than you think first.
It's not just HTML Sure.
You see an italic, you see a bold, but it's actually inserting a intermediate semantic node, which can then work with other builders, for example, to do italics in pdf.
The Rich linking.
Sure, you have markdown syntax.
That looks simple, but with Sphinx plus MyST its linking to structure, its connecting two pieces of information.
We saw that a little bit with generating the link, text from the target and finally the headings.
Sure, it's big and bold, right?
But in Sphinx it's actually sub structure in a document which can be put to use in many ways.
For example, toc trees can link to headings within a document.
There's a big engine underneath Sphinx and MyST even for italics links and headings.
This engine can be used for rich organized websites.
We'll see more of this in the course.

We've done some simple markdown but we're aiming higher static websites with richer content and structure of the kinds of things that Sphinx can do in this step, we'll look at more of what's possible thanks to MyST and it's Markdown parser we'll show how to do more with images.
Thanks to some alternate syntaxes that MyST introduces.
We'll go a little further into the knobs that govern the funky kind of table of contents thingy.
That's key to Sphinx.
If you've seen doc's you've probably seen air quotes, admonitions, those cool call outs for warnings and notes etcetera.
MyST has some cool stuff here too.
Do you want to include downloadable files such as PDFs into your generated site?
We'll take a look at that and just to flex a little, we'll talk a little bit about Sphinx's browser side search widget, which has had for a long time to give search capabilities to static websites.

Images are an important part of authoring and MyST gives us some interesting ways to express ourselves via images via markdown.
We'll take a look at some of the syntax for images in markdown.
Thanks to MyST and introduce some new cool things and MyST along the way.
We previously showed the syntax in MyST for remote and local images.
The second syntax has all the cool Sphinx stuff that you would expect.
It processes the image, copies it to the build directory, makes a relative URL inserts in the text etcetera.
But Sphinx can do a lot more with images and there's lots of options if you take a look at the Sphinx docs.
The image directive is really just a thin wrapper around restructured text which lets you do a lot of processing on the image.
How can we tap into some of that from syntax?
That's an alternative to the markdown syntax back in the editor.
We just want to make sure that everything is working with our live reload server.
I'm gonna just type Hello and I'm gonna save it go over my browser which is pointed at this page getting rendered.
I do see Hello.
So I'm back to life.
I can delete this and I'm gonna convert this simple markdown syntax to a MyST syntax using a code fence to insert the image, thus letting me tap into the underlying Sphinx directive.
So I'll start with a triple back tick and instead of a language, I'm going to put curly braces, which means the next thing is the directive and directive can have an argument which is the file name.
And then you can pass options to this.
I'll paste them in to speed things up, but we can have Python logo just as like we have in the square brackets.
We can put a CSS class, a width and alignment to say how it should appear on the page.
And when I save and go back over and take a look, I see the original one and then I see the second one which is aligned and 200 pixels wide.
Now we can also do something even cooler.
HTML has a figure element which puts kind of the image and some text that goes with it into a box that you can control.
And this is also supported in Sphinx and restructured text.
So let's take a look at what the syntax for that looks like.
What we're gonna use this time is an optional MyST syntax.
And these are things that you have to turn on in the conf file.
We're gonna use colon fence for this.
So back in my IDE.
I'll go to the conf file and see that I've got this enabled as an extension of the colon fence extension which lets me now type in the figure using a colon fence instead of a code fence.
I just pasted this in and you can see that it's got triple colons instead of triple back ticks.
And it's got a directive with an argument and some options and then a body.
The body has just a regular HTML image tag for the image asset and then some markdown that goes with it.
Let's go through a couple of these points.
Well first we'll just go ahead and save it and let it render and I go over to my browser, look at the bottom of my page and see that I've got a richer image now that has some body that goes with it, but that's not all.
Let's take a look first at why we want this triple colon instead of triple back tick.
If I go and look at how markdown tooling, such as previews and editors will treat it.
The first one is just rendered with everything inside the code fence.
In a big block.
The second one has some of it that's rendered in a way that of course this preview isn't gonna know what to do with this markdown MyST specific stuff but the part that is marked down, it can go ahead and render it and show it to you.
So for example, I could put in a bulleted list and it would handle that correctly if I took this for example and tried to put it as the body of this code fence is just gonna go right in the middle of the big blob.
So that didn't work correctly either.
One last point that really taps into some power that we're going to see later, is this argument is no longer the image file name.
It's a role.
It's something that is linkable within any documentation in Sphinx and therefore in MyST.
So for example, if I went back to my index file and put in some text that linked to it, go back to the rendering of it, go to the top level page.
I see that I now have a link to that figure and it extracted the title of the figure from the text that was in the figure element.
I can click on it and jump directly to that figure.
Why is all that possible?
Because the code fence, the colon fence that we used allowed us to specify a linkable role.

With Sphinx.
It isn't enough to just leave a file in a directory.
You need to include it in what Sphinx calls a 'Table of Contents'.
So let's take a look at the toctree directive, then explain what's going on behind the scenes and then explain why you should care.
Here is the Sphinx documentation page for the toctree directive and it explains a little bit more about the purpose of it.
The purpose is to link files most of the time in a directory in a particular order, including and excluding what you want to go into the generated output.
And you can see some of the options that are included with it.
You can add things such as captions and names and globs and we're gonna go over a couple of these back in our 'index.md' file.
We already have a toctree director of that.
We converted to the Myst Markdown syntax in a previous step.
It is done as a code fence with the triple back tick.
And then since the first argument of the code fences in a curly bracket, that means it's a tapping directly into a sphinx directive.
The toctree directive, we're passing to configuration options to the directive.
We're going to set how deep to go in the table of contents.
And what this does is each entry in the table of contents has headings and sub headings and subheadings and all of those can be plucked out and inserted into the hierarchy of the table of contents.
So you could jump directly to a heading from the table of contents and then the second one we're using is a caption where we set the information for the caption, we put it in quotes because there's a colon that we wanted to have.
Then we have a blank space and then we have a listing of items that are supposed to go into this table of contents.
These are paths, they can have relative and absolute slashes and dot thoughts and things in front of them and order matters.
The entry that you put these is the entry is the order that they will appear in the rendered output.
It also has some kind of semantic meeting as well.
So briefly if we wanted to change the caption that was displayed, we could edit this if we didn't want to manually list everything in a very large directory if order didn't matter if we want a convenience over meaning and structure or something there is a glob that is supported so I could put as an option to turn on globing and then here just say asterisk or some kind of glob character and I could have multiple lines on this to do different kinds of things for this.
What if we didn't like the syntax that was the syntax for these options This restructured text, what we want to YAML, MyST supports, YAML to send configuration arguments to directives and it would look like this kinda like front matter in markdown documents.
It's a triple dash and then a pile of YAML and another triple dash and a blank line and then everything else is the body.
So in this case we've basically converted key value pairs from the restructured text syntax into key value pairs for YAML now is wrap up.
Why do we care about table of contents?
Why do all this extra work?
Well Sphinx is about structure more than how it looks because it renders to multiple kinds of outputs and it has a lot of cool facilities for linking and structure and all that kind of stuff and so it cares about structure within a document but also between documents and that's where the toctree comes in play.
You can have a toctree to a range of information hierarchy.
You can even have multiple toctrees in the same document.

We'll close out with two short segments.
Sometimes you want to include a static file in your sight something that can be downloaded by clicking a link.
Maybe it's a pdf.
Maybe it's some code, maybe it's an image and you don't want it rendered in the page.
For example, click here for the full sized image as it turns out.
Sphinx has a role that you can add to your links to turn the link into a downloadable link instead of something that would try to navigate to the URL for that link.
When it's processed, the downloadable file pointed to that by that link for example 'example.py' would be copied into a under downloads directory and then given some unique file name so that it's hashed and whenever it changed it would get lead to a new one and then that hash file name is inserted into the to the link text that's generated and put into the document.
Let's take a look at this in action.
Going to go back to my editor gonna paste in a little bit of text that allows us to download the Python logo instead of just displaying it in line.
And for that we put the location of the file in back quotes which is kind of a common markdown thing to do but we preface it with curly brackets and a directive.
And what that's saying is this thing inside the curly brackets is a Sphinx roll, which is in our case download now that I save, I can go back over, take a look at the rendered output, scroll down and I see that you can download the Python logo and then I get some link text which is the file name, some styling and when I mouse over it I can see down in the bottom corner the URL.
Is some really long set of digits followed by Python-logo.png.
So that when you download it it will actually save on your computer with that.
If I click on it's going to prompt me to save it with a file name and that's the file name by default.
What if I don't want that As the link text this follows a convention that you'll see a lot in MyST where you can put this in greater than less than and put the logo some text that goes in the place of the file name and when I go back over I see that that is now my link text.
This download functionality uses roles.
We talked about this a little bit earlier on the figure element and how we can link to it using the role and we'll talk a lot more about this later in the course because roles are a really unique facility for Sphinx.

Sphinx has long provided a bundled solution for server list search which kind of makes sense Sphinx is a static site generator and there's not a search engine around and with this people type in search terms.
So for example in our site I could go and type 'thought' hit enter and even though it's just serving from files on desk, I'm able to get search results and highlighting and stuff like that.
Well how does this work.
At build time Sphinx writes an inverted index to a search index.js file.
If I went to my build directory and went to HTML in this HTML directory, I see that there's this file called searchindex.js and if I double click on it it's a big pile of java script that is a data structure for this inverted index.
And when someone asked to find search results, this data file is downloaded to the browser processed in the browser by a Sphinx javascript library and the results are presented.
Now there are limits to this approach.
Very large sites will generate a very large index and take a long time to download it and to process it and for these case many Sphinx sites use server side or cloud based integration with read the docks.
For example you'll get access to elastic search based indexing.
Not only is it a lot better performance and the browser download, but needless to say the search results are far better than you would get with this simple server list search.

Websites have links and static site generators can help write them, but Sphinx goes a lot further and MyST can help represent all of this in markdown

So imagine we are in a Sphinx site and it's got two documents, first.rst and second.rst.
And you want to link to one from the other in Sphinx with restructured text and one of the documents like the first document, you might put some text and then this colon doc colon thing.
And then in the back ticks you would put a link to the file name of the second document.
Now when you render, you'll see that the hyperlink actually has the title of the second document and we didn't put it in the link text, which is something you'd have to do in markdown.
And so it kind of makes you wonder, hey, what's going on here?
There's no link text.
And that turns out to be kind of the power of sphinx.
It keeps kind of a database of all the documents in your sight and everything that is linkable targets and it keeps the path and the title of all of the documents and resources.
So that when you make a link, it can insert the title and update the title in the link text.
If you change the targets title, it's not just for documents in this example we did colon dot colon.
But if you did colon ref colon, you could point to a location that was a role.
target somewhere in a document for example, a section heading.
And then one other thing that Sphinx can do in addition to providing you the link text and deep linking into a document is it will warn you if you link to something that doesn't exist and that's really not something you can get from some of these other static site generators.
And it's really valuable.
It's something that is worth looking at if you're doing a static website and markdown.

Markdown has a simple syntax for linking.
It's one of the things that's attractive about markdown versus restructured text, people writing in Sphinx usually get frustrated because the linking syntax is a little bit clumsy and you can't really remember, you always have to look it up.
Let's see the simple markdown linking syntax in action and see how it taps into the underlying Sphinx linking on the index page.
We'd like to link to the about us page in a sentence and use simple normal markdown linking along the way.
So I'll start by putting some sentence text and then I want to do the link and I'll do the normal markdown approach linking and if I wanted to I could, you know, preview this and a tool and it would give me the kind of rendered output.
But I can also go over to my browser and see what it did with it And I see that I've got a normal hypertext link with about us as the link text, so it looks just like normal markdown.
But if I do a couple of things to show that it's got the power of Sphinx underneath.
For example, I'm going to turn on the the run live reload command tool that that we introduced earlier and I'm gonna put in an error.
I'm going to link to something that doesn't exist and when I save, I'll see that I've got a warning that we referenced a Target about us X.
That doesn't exist and that's something you don't get from most markdown based static site generators Another thing you don't get from them is the ability to get the link text from the title of the target from about us so that I don't have to do the extra work to type it.
And if it changes in the future, I don't have to go all through my site changing it.
So I'm going to remove the link text from the square brackets and then when I go and take a look, I see that it has rendered as about us.
I can prove that it's getting the link text from the title by going over to about us and changing this to say about us really.
And when I go back to my browser, I see that it's automatically updated the link text for that link and any other links that point to about us.
And one last thing that's really cool about it is this all works not just with HTML but with pdf rendering and main page rendering and any custom kinds of builders that you might write.
This system of using simple markdown text of square brackets and parenthesis and then tapping into the underline power of Sphinx.

We just saw how to link to a document but can we link to a spot within a document, like a heading or even just some arbitrary location Sphinx has a way to insert.
We'll call the markers in documents and these markers can mean many things, one of which they can provide linkable targets.
And so these markers, which Sphinx calls roles can be written as colon, then role name and then the back tick and then the content for that role.
And so for example, I could just say roll target somewhere and then I've created a new spot in the links, the Sphinx system of linking.
I've created a new target that other things can link to and even get things like the title text.
And we'll show in this example a way of linking to a section and having the section title be used as the link text.
With roles It makes it really easy to have stable intuitive deep links in your website, which frees your links from using file names at paths and have something a lot easier to remember.
A lot easier to fix if you reorganize your file names and directory names and things like that and they have semantic meaning rather than whatever you might have used for the file name.
So back in about us, I want to add a section, it talks about our investors.
It's going to put in some text, normal markdown stuff that's going to be an H2 because it has double pound hashtag and then some a paragraph that could have other paragraphs and the whole thing would be considered a single block kind of a section.
And I'd like that section to be linkable deep linka ble within the document from other places in my website.
And the the MyST syntax for doing this in markdown is pretty simple.
I want to just say investors.
I don't want to have to say the name of the file.
I don't want to have to put some special sequence in order to deep link down into it.
And so I'm gonna say investors and I have now added a role called investors.
And so anything that wants to link to it can link to the role.
It will scroll down directly to this and it's actually a structural part of the site.
It has meaning that can be put to work in various cases.
And so now if I go back to my homepage and I'd like to give information about our investors, I start typing a sentence and I can say ref to tap into the ref directive using a curly brackets and then back tick in the name of the role and I'll save that.
And we saw this syntax previously about having a role in in the middle of a sentence an in line kind of thing.
And the role is just I'm sorry the directive is just curly brackets directive name.
We used it for download links now when I go over to the browser, I go to the homepage, I see that it has linked to the page but look at the URL.
Down at the bottom.
It's going to say investors to deep link all the way down to that section heading and it's taking the as link text is taking the name of the heading that I provided.
Let's take a look at that again.
I'm gonna change the heading and it's automatically going to change the link text.
I'll change this to our investors And when I save I didn't have to go back over to all the things that link to it and provide new link text.
They're all updated because they're all pointing to that role.
If I go back to the browser, I see that I've gotten my updated link text.
So just as a review what Sphinx is doing here.
Thanks to MyST and Markdown syntax, it's finding the role definition for investors.
It will warn if that role doesn't exist if it does exist, it will get the link text from the heading just after the role.
It will generate a link to that document with that link text and it will put an internal anchor to scroll down to that section.
Just as one last point on this.
You can also inside here override the link text.
If you want to provide your own link tag.

As you can imagine there's a lot more linking power in Sphinx beyond.
just rolls.
First if you really liked that thing about linking to a heading but you find it too much work to preface all of your headings with that syntax for a link, MyST has a setting for auto generating header anchors from the heading words.
And this can make it very convenient to just get that for free everywhere that you use a heading.
But I consider it a little bit iffy because those identifiers don't prove to be all that stable.
Remember they're going to be published as part of your website and outside people might be linking to it as well.
And if you decide to change the wording of one of your headings, then all of your roles and all of your external links will be broken.
It's just not a very stable identify.
Sphinx is known for documentation and it's auto doc tool generates all kinds of linkable targets in your source code.
We're going to have a section devoted to documentation and auto doc but for now just understand that you point something like Sphinx at a big pile of code and it not only turns it into words and highlighting on the screen, it turns it into structure including roles.
Linkable targets that can be pointed to as stable, identifiers from within your narrative documentation.
We've seen a little bit about the toctree directive and it gets your files listed in a certain order but it also helps on linking because thanks to the order in the nesting, it will generate previous and next links that you can use in your templates as well as up to go up A level and top to go all the way to the top and finally linking to external URL's that aren't managed by Sphinx that's a part of your website.
But how are you going to find out if those links are broken, Sphinx ships with a tool that you can use and run make link check.
And it will go through your built documentation and try to find any places that point to a URL on the web, which no longer works

Sure, this course is about static websites, but it's also about Sphinx, and Sphinx is traditionally known for documentation.
But you know, lots of static websites heck.
Even a website network like Medium, have articles that cover codes.
So what does it look like to include code examples in your static website?
But with markdown?

Embedding a code example, into a document is the simplest starting point and is quite familiar to markdown folks.
In markdown, this is known as a 'fenced code block' or more appropriately a code offense.
And it's triple back ticks some code and some triple back ticks.
And in when using this in Sphinx it will tap into the underlying 'Pygments' system for detecting the language and having some syntax highlighting and there's a little bit more that goes with it.
When you're using markdown, you can have a word after the back text.
That is the name of the language.
I guess it defaults to Javascript in most Javascript systems and then sometimes in addition to that argument for the language of the code snippet the code fence.
Sometimes it has some truly funky ways to pass options for things, such as highlighting certain lines.
Now in Sphinx, a code fence is represented by something called a code block directive and this taps into the underlying restructured text system as well.
It's very similar.
It contains code.
One of the things that's a little bit irritating about it is everything has to be indented within that code block to make the block itself.
But something that is a lot nicer about it is a well defined standard way to pass arguments to the code block, for example, to show line numbers what line number to start on things like that instead of these funky ways that are unique to each system for passing extra information such as options.
Let's see some code fences in action meaning the markdown syntax to tap into the Sphinx code block directive.
I'll do a very simple one.
Hello world in Python and I'll do it with the triple back tick for the code fence and I will eliminate any language and let it auto detect.
And so for sphinx and pigments it will be Python and I'll write some Python code that returns an F string saying hello name, I'll save this and then go over to the browser and take a look at the bottom of the about us page.
I now have a syntax highlighted and look I misspelled return syntax highlighted chunk of Python code embedded right in the middle of my markdown file I could make this more explicit about what language it is and use the normal markdown code fence syntax for saying the language and when I save and go back you'll see that not only have I correctly spelled return but nothing has changed.
It's still thinks this is Python.
Now what if I wanted the equivalent?
But for javascript I'll do a code fence, I'll say javascript this time as the language and I will just do a one liner function Hello message and I'll go ahead and put it on multiple lines.
Now when I save and go over to my about us page I see that I have a syntax highlighted chunk of javascript.
We're going to look at passing options, which is kind of a MyST thing rather than a markdown thing.
And for that we're going to kind of turn on the fact that this is a code block and use something that we saw previously in our code blocks.
If it's the code block argument starts with a curly brace, then we are telling MyST that this is a directive.
A Sphinx directive that we're talking to.
And then the next thing after the directive after the closing of the curly brace is the argument to the directive.
So I'll save that.
Go back and you see nothing has changed.
We're all still good.
The previous thing was such a shorthand for that.
But with this in place I can now do some of the Sphinx and restructured text arguments to code block as a directive.
And so I'll say line numbers as an option and when I save it and go back to the browser I see that I now get line numbers added to my code block.
This is using the Sphinx option syntax with colon, colon if you don't like that and you prefer to have YAML, we saw this in a previous step of the course I can do triple tick and then I can supply YAML and then triple tick again.
And when I go back to view this in the browser, I see that it still works the marriage of markdown syntax for code fences to the power of the code block directive in Sphinx works really well because Sphinx has lots of powerful options for working with code in your documents, things that I use all the time and really make a difference when you're trying to tell a story with code.

We just showed embedding code examples into a markdown document.
I almost never want to do that because I'm in the middle of a markdown document and all of my Python coding tooling, syntax highlighting, all of that basically doesn't work that well anymore.
I want to write a Python file and then I want to include that Python file into the document and then that lets me do things like write tests for the Python file to make sure I haven't broken anything.
So instead of code block I generally use something from Sphinx called 'literal include'.
We will take a look at it in this step.
And while other static site generators, particularly in the javascript space kind of have this bolted on it just never really stays working correctly.
And I always have to fiddle with it and fork the the library or something like that.
Whereas with Sphinx, this was built in from the start and you can tell because it just works it works perfectly.
It's what Sphinx was built for and so it's matured has a lot of options.
Let's take a look at literal include but from markdown.
So we're talking about including code from a file.
And Sphinx has the literal include directive.
If you go to the page documenting it, you use the directive pointed at the file name.
You give it some additional options.
A lot of these options overlap a lot with the code block directive, which makes it really handy to get a lot of the power when you're including like a Python file or javascript file or something like that.
So let's take a look at using this.
I'm back in my project and I've got a Python file called my under demo.
It's here.
My_demo.
It's just a class with a constructor and a method.
It's got a little bit of doc stringing and it will take a look at that a little bit later and we would like to include this code snippet into our markdown document into about us.
But what we don't really want to do is just keep cutting and pasting it and every time it changes, we have to remember to go over and cut and paste it back into the markdown And we certainly don't want to just type the original version in the markdown.
Especially if it's part of a library.
We can't ship the markdown document so we're gonna look at literal include to put this into the markdown document.
The very simplest way to do this is triple back tick code fence.
And instead of code block, which we just saw, I'm going to say you guessed it literal include and then my_demo.py.
Very very simple.
And I'll put a the new line because we're gonna be doing a little bit more with this in a second when I save and then head back over to my browser and about us.
I see that all the code in that Python file is now included in the document and syntax highlighted exactly as if I cut and pasted it into a code block.
What if we wanted to pass some arguments to it?
It works about the way you would expect.
I could take the sphinx syntax and say emphasize lines 2-3 and save when I go back and take a look.
I see I get some highlighting on this.
This is something I use all the time to show what changed from one example to another when I'm building a file up in multiple parts of a page and just like we saw before I can also use this as the Yaml syntax, triple ticks and then remove that colon.
Everything else remains the same.
And when I save I'll go back to the browser and see that nothing has changed.
I've just switched over to a different format for providing the options to the literal include directive as wrap up.
I really really enjoy using literal include especially the options to get a fragment from a start pattern to an end pattern instead of having to put in line numbers which might change.
And then all of my examples are highlighted incorrectly or included incorrectly.
I can put patterns into my code to say pluck this part out or highlight this part or whatever it's something I value Sphinx a lot and having a natural syntax in markdown is wonderful.

We've looked at code snippets directly in a markdown document as well as including code from a single file.
But what about a code base?
Sphinx is basically built for that to make the documentation for all of Python itself.
In fact, how does that look?
But in a markdown site this brings up the centerpiece of the Sphinx ecosystem.
Auto dock with auto doc.
You point a sphinx document at your code and it generates kind of a structured highlighted interlinked collection of sections and even better the symbols in your code, whether it's Python or whatever become roles in Sphinx, which you can directly linked to from narrative documentation.
Let's set up auto doc for use in our MyST powered markdown website.
First in our conf.py file, which is where we set up Sphinx.
We need to turn on auto doc.
It's an extension that's bundled with Sphinx but not turned on by default.
So I'll say sphinx.ext.autodoc.
And with that extension enabled, I also need to make sure that the code for my project my my_demo.py file is available in Python path.
So I'll do this kind of trick where I import sys in list and add dot the current directory to the Python path, which will then let me point auto doc at a module.
So that's all it takes to get auto doc set up.
In fact, the Python part really isn't needed.
It's only if you're not working in kind of an editable installed Python package if you're just doing kind of a one off app or something like that.
So it's not that much work.
And once you get started, we can take in the next step a look at bringing documentation of modules and packages into your documentation.

To document a module or even a package.
Your starting point is actually in a markdown document.
This lets you add some narrative text as well as providing a glue point to attach the module into the documentation and thus into your site structure we saw previously, how to get auto doc configured for your Sphinx site.
Let's take a look at the kinds of directives we will use to actually glue code into our pages.
So the auto doc directive section tells us that there are things such as auto module, auto class, auto exception, etcetera.
And these let you point at some code that you want to be automatically documented and it goes on to show some of the restructured text syntax for how some of this might work.
But we're in markdown and it's a little bit tricky with MyST and the syntax for this, there isn't a direct rapper for getting auto doc expressed in markdown.
So what you wind up doing is kind of punting to a raw RST mode where you're just embedding restructured text into a directive in the middle of a markdown file.
Now you could do and in fact, this is what MyST does.
You could just leave file like 'Api.rst' in the middle of your markdown project, you could have a restructured text file and then plug that into the rest of your site.
But if you wanted to keep the mark down and just have the directive for the auto doc stuff.
This is the way you would do it with eval rst.
Let's go to the api page where we want to do the work for this.
I have a file api.md for markdown and I want to do a code fence that has a sphinx directive And in this case I guess it's more like a MyST directive.
'eval-rst'.
And then the body of this directive is going to be restructured text.
So I will write the restructured text version of auto class which is ..autoclass And then I'll say the name of the file.
This is where the Python path part was important.
The name of the module and then the name of the class that I want to automatically document and what I'd like to do is pass one argument, one configuration option to the documenting of this class members.
So now when I save I'll go back to the browser and I don't get anything to appear which is a good time for some debugging.
I'll head back over to my editor and I see that MyST warned me that I put sphinx.ext.autodoc in the wrong place in my conf file.
So I'll go fix that in its under MyST enable extensions when really it's supposed to be under the sphinx extensions.
So now when I save I'll go back to my API file and I'll put a blank line in just to force it to regenerate.
I don't see any errors.
So when I go back to my browser and look at the rendering, I see that it has included my class syntax highlighted, and it even has some extra stuff for the structure in it.

Perhaps you've gotten the point by now, Sphinx extracts a lot of structure in your site.
But what about other sites?
Wouldn't it be great if linking to structure in another site was almost as easy as linking to that structure in your own site And then if the path changed or if it disappeared, you'd get a warning.
And you could just use the target's current title as the link text enter, 'Intersphinx'.
One of those amazing Sphinx only things that kind of blows your mind.
It's actually really useful and you don't expect it when coming from other systems.

Inter Sphinx is a sphinx extension.
It's bundled with Sphinx but it isn't on by default.
Let's quickly show enabling it and setting it up.
I'm back in my sphinx .conf file and in the extensions to Sphinx I want to add an entry to turn on 'sphinx.ext.intersphinx'.
Now that the extension is loaded, I need to do a little bit of configuration for it.
I need to point it at the external websites that has Sphinx inventories available that I would like to use in my website.
So the first thing I want to add is the inter sphinx mapping which says when I'm doing links in my sight and I put the special prefix on it called Sphinx which we'll see in a bit.
Then that comes from the intersphinx inventory at this URL.
And you're basically telling Sphinx hey you gotta go download this special file at that place I have one other thing I have to do with MyST which is turn on this URL.
Schemes thing to allow MyST to work correctly with some of the URL Syntax.
And with this now set up we will start using it and see it how downloads the inevntory files.

We have a static website and we're riding content that has rich linking within that website Now that we've set up Inter Sphinx we can link to content on the Sphinx site as if it were local Let's see this in action first.
I want to demonstrate a full build of Sphinx from the command line so you can see the output that gets the inventory file from the Sphinx website.
So when I run the Sphinx build command a lot of the output that it dumps to the console and that includes loading inter sphinx inventory from the URL.
We specified up in the conf file so that's what it's doing behind the scenes is it checks to see if it has any of these objects.inv files that need to be downloaded, retrieves them and associates them with a key of Sphinx.
Now back in my about us document.
I might want to link to something that says we're using Sphinx and we're using Sphinx roles and how powerful they are.
So I could do it the traditional external link way with markdown where you put the link text and then you put a.
URL.
But that doesn't give me any of the sphinx goodness.
First you know that's a long.
URL.
But what if we typed it wrong?
Would we get any warnings?
No we would have to run some link checker.
What if the link what if the Sphinx stocks get reorganized so that it's not at that location anymore.
It's at some other location.
Well with roles.
You can hide the internal structure just by putting a semantic label on things.
And wouldn't it be great to apply that to external content and then finally it's no fun thinking about this link text which might change and get out of date with what the remote page says is the meaning of that link target.
Now the first of the three of these points can be solved with a separate link check but it's more work and won't help with some of this more semantic stuff.
So let's take a look at how we would do this with a sphinx roll.
Thanks to 'interphinx'.
So first thing I'll do is delete this really long URL .
I'll leave the link text in there.
But all I need to say now is that prefix that I had in the conf file, a colon and then the role that I'm pointing to.
So I will save that and I'll go back to my browser and I'll look at the bottom of my about us page and I'll see that I now have a link that takes me to this.
URL.
I didn't have to memorize some really long H to blah blah blah.
And in fact if I go back to my text I can do what you hope we could do which is leave out the link text and let the content on the other end hand over the title.
Save it go back to my document in the browser, see what it reloaded as.
And it gave me the link text of the thing that I was linking to.
What we also saw was that I wasn't linking to a remote document in its title.
I was linking a deep link into a section in the remote document and getting the link text from that section heading now as wrap up and as a reminder if you link to something that is broken, if I put this then my sphinx build would actually warn me that I have a broken link if I linked to this but the remote side changed it.
I would also get a warning saying that I link to something that doesn't exist.
This is something that really doesn't exist in other static site generators and is a very useful tool.