Doc generators » Python docs

Theme se­lec­tion

By de­fault, the dark m.css theme to­geth­er with doc­u­ment­a­tion-theme-spe­cif­ic ad­di­tions is used, which cor­res­ponds to the fol­low­ing con­fig­ur­a­tion:

STYLESHEETS = [
    'https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600',
    '../css/m-dark+documentation.compiled.css']
THEME_COLOR = '#22272e'
FAVICON = 'favicon-dark.png'

If you have a site already us­ing the m-dark.compiled.css file, there’s an­oth­er file called m-dark.documentation.compiled.css, which con­tains just the doc­u­ment­a­tion-theme-spe­cif­ic ad­di­tions so you can re­use the already cached m-dark.compiled.css file from your main site:

STYLESHEETS = [
    'https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600',
    '../css/m-dark.compiled.css',
    '../css/m-dark.documentation.compiled.css']
THEME_COLOR = '#22272e'
FAVICON = 'favicon-dark.png'

If you prefer the light m.css theme in­stead, use the fol­low­ing con­fig­ur­a­tion (and, sim­il­arly, you can use m-light.compiled.css to­geth­er with m-light.documentation.compiled-css in place of m-light+documentation.compiled.css:

STYLESHEETS = [
    'https://fonts.googleapis.com/css?family=Libre+Baskerville:400,400i,700,700i%7CSource+Code+Pro:400,400i,600',
    '../css/m-light+documentation.compiled.css']
THEME_COLOR = '#cb4b16'
FAVICON = 'favicon-light.png'

See the CSS files sec­tion be­low for more in­form­a­tion about cus­tom­iz­ing the CSS files.

Navbar links

The LINKS_NAVBAR1 and LINKS_NAVBAR2 op­tions define which links are shown on the top navbar, split in­to left and right column on small screen sizes. These op­tions take a list of (title, path, sub) tuples — title is the link title; path is either one of 'index', 'pages', 'modules' or 'classes' (link­ing to the main page or page / mod­ule / class in­dex path), a full URL (pas­ted as-is) or a path to a par­tic­u­lar page or mod­ule/class (in the form of ['module', 'sub', 'ClassName'] for module.sub.ClassName, which then gets format­ted ac­cord­ing to URL format­ting rules); and sub is an op­tion­al sub­menu, con­tain­ing (title, path) tuples, with path be­ing in­ter­preted the same way.

By de­fault the vari­ables are defined like fol­low­ing — there’s just three items in the left column, with no sub­menus and the right column is empty:

LINKS_NAVBAR1 = [
    ('Pages', 'pages', []),
    ('Modules', 'modules', []),
    ('Classes', 'classes', [])]
LINKS_NAVBAR2 = []

A menu item is high­lighted if a page with the same path is the cur­rent page. The path can be also a full URL — if it con­tains a scheme pre­fix (such as https://), then it’s taken as-is, without con­ver­sion.

Search op­tions

Sym­bol search is im­ple­men­ted us­ing JavaS­cript Typed Ar­rays and does not need any serv­er-side func­tion­al­ity to per­form well — the cli­ent auto­mat­ic­ally down­loads a tightly packed bin­ary con­tain­ing search data and per­forms search dir­ectly on it.

How­ever, due to re­stric­tions of Chro­mi­um-based browsers, it’s not pos­sible to down­load data us­ing XMLHttpRequest when served from a loc­al file-sys­tem. Be­cause of that, the search de­faults to pro­du­cing a Base85-en­coded rep­res­ent­a­tion of the search bin­ary and load­ing that asyn­chron­ously as a plain JavaS­cript file. This res­ults in the search data be­ing 25% lar­ger, but since this is for serving from a loc­al filesys­tem, it’s not con­sidered a prob­lem. If your docs are ac­cessed through a serv­er (or you don’t need Chrome sup­port), set the SEARCH_DOWNLOAD_BINARY op­tion to True. The search data are by de­fault fetched from the cur­rent dir­ect­ory on the web­serv­er, if you want to sup­ply a dif­fer­ent loc­a­tion, set it to a string and provide a cus­tom URL format­ter.

The site can provide search en­gine metadata us­ing the OpenSearch spe­cific­a­tion. On sup­por­ted browsers this means you can add the search field to search en­gines and search dir­ectly from the ad­dress bar. To en­able search en­gine metadata, point SEARCH_BASE_URL to base URL of your doc­u­ment­a­tion, for ex­ample:

SEARCH_BASE_URL = 'https://doc.magnum.graphics/magnum/'

In gen­er­al, even without the above set­ting, ap­pend­ing ?q={query}#search to the URL will dir­ectly open the search popup with res­ults for {query}.

If SEARCH_EXTERNAL_URL is spe­cified, full-text search us­ing an ex­tern­al search en­gine is offered if noth­ing is found for giv­en string or if the user has JavaS­cript dis­abled. It’s re­com­men­ded to re­strict the search to a par­tic­u­lar do­main or add ad­di­tion­al keywords to the search query to fil­ter out ir­rel­ev­ant res­ults. Ex­ample, us­ing Google search en­gine and re­strict­ing the search to a sub­do­main:

SEARCH_EXTERNAL_URL = 'https://google.com/search?q=site:doc.magnum.graphics+{query}'

The search bin­ary is im­pli­citly made with the tight­est pack­ing pos­sible for smal­lest down­load sizes. On large pro­jects with tens of thou­sands of sym­bols it may how­ever hap­pen that the data won’t fit and doc gen­er­a­tion fails with an ex­cep­tion such as the fol­low­ing, sug­gest­ing you to in­crease the packed type sizes:

Over­flowEr­ror: Trie res­ult ID too large to store in 16 bits, set SEARCH_RES­ULT_ID_­BYTES = 3 in your conf.py.

The rel­ev­ant con­fig­ur­a­tion is SEARCH_RESULT_ID_BYTES, SEARCH_FILE_OFFSET_BYTES and SEARCH_NAME_SIZE_BYTES. Simply up­date your conf.py with sug­ges­ted val­ues and re­start the gen­er­at­or. Due to the way the search data get pro­cessed dur­ing seri­al­iz­a­tion it’s un­for­tu­nately not feas­ible to es­tim­ate the pack­ing sizes be­fore­hand.

Cus­tom URL format­ters

The URL_FORMATTER op­tion al­lows you to con­trol how all fi­le­names and gen­er­ated URLs look like. It takes an entry type and a “path” as a list of strings (so for ex­ample my.module.Class is rep­res­en­ted as ['my', 'module', 'Class']), re­turn­ing a tuple of a fi­le­name and an URL. Those can be the same, but also dif­fer­ent (for ex­ample a file get­ting saved in­to my/module/Class/index.html but the ac­tu­al URL be­ing https://docs.my.module/Class/). The de­fault im­ple­ment­a­tion looks like this, pro­du­cing both fi­le­names and URLs in the form of my.module.Class.html:

def default_url_formatter(type: EntryType, path: List[str]) -> Tuple[str, str]:
    if type == EntryType.STATIC:
        url = os.path.basename(path[0])

        # Encode version information into the search driver
        if url == 'search.js':
            url = 'search-v{}.js'.format(searchdata_format_version)

        return url, url

    url = '.'.join(path) + '.html'
    assert '/' not in url
    return url, url

The type is an enum, if you don’t want to fiddle with im­ports, com­pare type.name against a string, which is one of 'PAGE', 'MODULE', 'CLASS', 'SPECIAL' or 'STATIC'. The 'SPECIAL' is for in­dex pages and in that case the path has al­ways just one item, one of 'pages', 'modules' or 'classes'. The 'STATIC' is for stat­ic data such as im­ages or CSS files and the path is ab­so­lute in­put fi­le­name in­clud­ing the ex­ten­sion and ex­cept for search data (which are gen­er­ated on-the-fly) it al­ways ex­ists. If the stat­ic path is an URL, the URL format­ter is not called.

The ID_FORMATTER handles format­ting of an­chors on a page. Again it takes an entry type (which in this case is al­ways one of 'ENUM', 'ENUM_VALUE', 'FUNCTION', 'PROPERTY', 'DATA' or, in case of py­bind11 code, 'OVERLOADED_FUNCTION'. The second para­met­er is again a path, be­ing al­ways just one item ex­cept for 'ENUM_VALUE' (in which case it’s enum name and value name to­geth­er) and for 'OVERLOADED_FUNCTION', in which case it con­tains also a llist of ar­gu­ment types. The de­fault im­ple­ment­a­tion simply re­turns the the path con­cat­en­ated with dashes:

def default_id_formatter(type: EntryType, path: List[str]) -> str:
    return '-'.join(path)

Doc­u­ment­a­tion-only over­rides

While the in­spec­tion and auto­de­tec­tion fol­lows the struc­ture of the ori­gin­al code, it might not be al­ways de­sir­able to show it ex­actly that way in the docs. For­tu­nately, thanks to Py­thon’s dy­nam­ic nature, a lot can be done by mon­key-patch­ing dur­ing doc gen­er­a­tion.

In case the auto­de­tec­tion in­cludes more than you want or, con­versely, you need to in­clude names that would oth­er­wise be ex­cluded (such as un­der­scored names), you can tem­por­ar­ily over­ride the __all__ at­trib­ute when gen­er­at­ing the docs. For ex­ample, the fol­low­ing will list just the pow() and log() func­tions from the math mod­ule, ig­nor­ing the rest:

import math
math.__all__ = ['pow', 'log']

INPUT_MODULES = [math]

In oth­er cases, es­pe­cially when nat­ive mod­ules are in­volved, the in­spec­ted name loc­a­tions might not be what you want. By put­ting the names in­to __all__ you tell the script it should map the in­spec­ted loc­a­tion to the one provided. Note you should also hide the ori­gin­al loc­a­tion from the script to avoid du­plic­ate defin­i­tions (un­less it’s un­der­scored, in which case it’ll get ig­nored auto­mat­ic­ally).

# module math

from _native_math import fast_sin as sin
from _native_math import fast_cos as cos
__all__ = ['sin', 'cos']

Ad­di­tion­ally, for map­ping types of ex­tern­al lib­rar­ies where the auto­de­tec­tion from __all__ can’t be per­formed, you can use the NAME_MAPPING op­tion:

NAME_MAPPING = {
    'fastmath._native.Vector3': 'fastmath.Vector3',
    'fastmath._native.Quaternion': 'fastmath.Quaternion',
    # or, equivalently, if the mapping is the same for all members:
    'fastmath._native': 'fastmath'
}

Doc­strings

By de­fault, the first para­graph of a mod­ule-level, class-level and func­tion-level doc­string is used as a doc sum­mary, copied as-is to the out­put without format­ting it in any way. What fol­lows is put (again without format­ting) para­graph-by-para­graph in­to de­tailed docs.

"""Module summary

First paragraph of module detailed docs."""

class Foo:
    """Class summary"""

    def bar(self):
        """Function summary"""

Us­ing just doc­strings, how­ever, comes with a few lim­it­a­tions:

• Class and mod­ule-level vari­ables can’t have a doc­string at­tached due to how Py­thon works
• Be­cause not every Py­thon API can be doc­u­mented us­ing doc­strings, the out­put con­tains everything, in­clud­ing un­doc­u­mented names
• In­stance vari­ables ad­ded in­side __init__() are not ex­trac­ted, as this would re­quire pars­ing Py­thon code dir­ectly (which is what Sphinx has to do to sup­port these). You can work around this by adding an­not­ated “de­clar­a­tions” to the class as shown above, how­ever no doc­strings can be spe­cified for those either.

To over­come the lim­it­a­tions, ex­tern­ally-sup­plied doc­u­ment­a­tion provides means to doc­u­ment names that can’t have a doc­string at­tached, and to­geth­er with the m.sphinx plu­gin ex­pand­ing format­ting cap­ab­il­it­ies bey­ond plain text.

Func­tion and vari­able an­nota­tions

The script uses inspect.signature() to query func­tion para­met­er / re­turn type an­nota­tions to­geth­er with de­fault val­ues and dis­plays them in the out­put. Sim­il­ar is for mod­ule and class vari­ables, ex­trac­ted from the __annotations__ prop­erty. If a vari­able type im­ple­ments __repr__(), a repr() of it is prin­ted as the value, oth­er­wise the value is omit­ted.

from typing import Tuple, List

def foo(a: str, be_nice: bool = True) -> Tuple[int, str]:
    pass

SETTINGS: List[Tuple[str, bool]] = []

For bet­ter read­ab­il­ity, if the func­tion sig­na­ture con­tains type an­nota­tions or a de­fault value, the ar­gu­ments are prin­ted each on one line. Oth­er­wise, to avoid wast­ing ver­tic­al space, the ar­gu­ments are lis­ted on a single line.

Sim­il­arly to how the built­in help() in Py­thon 3.7 star­ted an­not­at­ing bound­ar­ies between po­s­i­tion-only, po­s­i­tion-or-keyword and keyword-only ar­gu­ments with / and *, the same is done here — it’s es­pe­cially help­ful for nat­ive func­tions, where you can for ex­ample call math.sin(0.3) but not math.sin(x=0.3), be­cause the x ar­gu­ment is po­s­i­tion­al-only. Cur­rently, po­s­i­tion­al-only ar­gu­ments are pos­sible only with nat­ive func­tions, PEP570 adds them for pure Py­thon func­tions as well.

In some cases, es­pe­cially when doc­u­ment­ing nat­ive func­tions, the sig­na­ture can’t be ex­trac­ted and the func­tion sig­na­ture shows just an el­lip­sis (…) in­stead of the ac­tu­al ar­gu­ment list.

Class meth­ods, stat­ic meth­ods, dun­der meth­ods, prop­er­ties

Meth­ods dec­or­ated with @classmethod are put in­to a “Class meth­ods” sec­tion, @staticmethods in­to a “Stat­ic meth­ods” sec­tion. Double-un­der­scored meth­ods ex­pli­citly im­ple­men­ted in the class are put in­to a “Spe­cial meth­ods” sec­tion, oth­er­wise they’re ig­nored — by de­fault, Py­thon adds a large col­lec­tion of dun­der meth­ods to each class and the only way to know if the meth­od is user-provided or im­pli­cit is by check­ing the doc­string.

class MyClass:
    @classmethod
    def a_classmethod(cls):
        """A class method"""

    @staticmethod
    def a_staticmethod():
        """A static method"""

    def __init__(self, foo, bar):
        """A constructor"""

Prop­er­ties ad­ded to classes either us­ing the @property dec­or­at­or or cre­ated with the property() built­in are ad­ded to the “Prop­er­ties” sec­tion. Each prop­erty is an­not­ated with get set del if it has a get­ter, a set­ter and a deleter or with get and oth­er vari­ants if it has just some. The doc­string and type an­nota­tion is ex­trac­ted from the prop­erty get­ter.

from typing import Tuple

class MyClass:
    @property
    def a_read_write_property(self) -> Tuple[int, int]:
        """A read-write tuple property"""

    @a_read_write_property.setter
    def a_read_write_property(self, a):
        # Docstring and type annotation taken from the getter, no need to
        # have it repeated here too
        pass

Doc­u­ment­ing enum val­ues

Py­thon sup­plies an im­pli­cit doc­strings for enums de­rived from enum.Enum and enum val­ues im­pli­citly in­her­it the doc­string of the enum class. If either is de­tec­ted to be the case, doc­string of the enum or the value is ig­nored. While it’s pos­sible to doc­u­ment enum classes the usu­al way, there’s a non-ob­vi­ous way to doc­u­ment enum val­ues as well.

import enum

class MyEnum(enum.Enum):
    """My enum"""

    ZERO = 0
    TWO = 3
    CONSISTENCY = -73

MyEnum.ZERO.__doc__ = "Zero value"
MyEnum.TWO.__doc__ = "Three, but named TWO for compatibility"

The doc­u­ment­a­tion out­put for enums in­cludes enum value val­ues and the class it was de­rived from, so it’s pos­sible to know wheth­er it’s an enum or a flag.

In­clud­ing un­der­scored names in the out­put

By de­fault, names start­ing with an un­der­score (ex­cept for __dunder__ meth­ods) are treated as private and not lis­ted in the out­put. One way to ex­pose them is to list them in __all__, how­ever that works for mod­ule con­tent only. For ex­pos­ing gen­er­al un­der­scored names, you either need to provide a doc­string or ex­tern­al doc­u­ment­a­tion con­tent (and in case of plain data, ex­tern­al doc­u­ment­a­tion con­tent is the only op­tion).

Note that at the point where mod­ules and classes are crawled for mem­bers, doc­strings are not parsed yet — so e.g. a data doc­u­ment­a­tion via a :data: op­tion of the .. py:class:: m.sphinx dir­ect­ive won’t be vis­ible to the ini­tial crawl and thus the data will stay hid­den.

Some­times, how­ever, you’ll want the in­verse — keep­ing an un­der­scored name hid­den, even though it has a doc­string. Solu­tion is to re­move the doc­string while gen­er­at­ing the docs, dir­ectly in the conf.py file dur­ing mod­ule im­port:

import mymodule
mymodule._private_thing.__doc__ = None

INPUT_MODULES = [mymodule]

Func­tion sig­na­tures, prop­erty an­nota­tions

For reas­ons ex­plained in py­bind/py­bind11#990, py­bind11 is not able to provide func­tion sig­na­tures through in­tro­spec­tion and thus the script falls back to pars­ing ar­gu­ment names, type an­nota­tions and de­fault val­ues from the doc­string in­stead. By de­fault, un­less py::arg() is used, func­tion ar­gu­ments are po­s­i­tion­al-only (shown as arg0, arg1, …) and marked as such in the out­put.

Sim­il­arly, prop­erty types are ex­trac­ted from get­ter doc­strings.

Un­like Py­thon, py­bind11 has a built­in sup­port for over­loaded func­tions — de­pend­ing on types passed to a func­tion, it dis­patches to a par­tic­u­lar C++ over­load. The over­loads are ex­pan­ded in the out­put as well, mean­ing you can see one func­tion men­tioned more than once with dif­fer­ent sig­na­tures.

Be­cause stat­ic meth­ods in py­bind11 are not dec­or­ated with @staticmethod, they are de­tec­ted based on pres­ence of self as the first para­met­er — if it’s there, it’s an in­stance meth­od, oth­er­wise it’s a stat­ic meth­od.

Enums

Enums in py­bind11 are not de­rived from enum.Enum, but rather are plain classes. The only re­li­able way to de­tect a py­bind11 enum is by look­ing for a __members__ mem­ber, which is a dict provid­ing string names and their cor­res­pond­ing val­ues. With py­bind 2.2, it’s only pos­sible to doc­u­ment the enum class it­self, not the val­ues.

CSS files

By de­fault, com­piled CSS files are used to re­duce amount of HT­TP re­quests and band­width needed for view­ing the doc­u­ment­a­tion. How­ever, for easi­er cus­tom­iz­a­tion and de­bug­ging it’s bet­ter to use the un­pro­cessed stylesheets. The STYLESHEETS op­tion lists all files that go to the <link rel="stylesheet" /> in the res­ult­ing HTML markup, while EXTRA_FILES list the in­dir­ectly ref­er­enced files that need to be copied to the out­put as well. Be­low is an ex­ample con­fig­ur­a­tion cor­res­pond­ing to the dark theme:

STYLESHEETS = [
    'https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600',
    '../css/m-dark.css',
    '../css/m-documentation.css']
EXTRA_FILES = [
    '../css/m-grid.css',
    '../css/m-components.css',
    '../css/pygments-dark.css',
    '../css/pygments-console.css']
THEME_COLOR = '#22272e'

After mak­ing de­sired changes to the source files, it’s pos­sible to post­pro­cess them back to the com­piled ver­sion us­ing the postprocess.py util­ity as ex­plained in the CSS themes doc­u­ment­a­tion. In case of the dark theme, the m-dark+documentation.compiled.css and m-dark.documentation.compiled.css files are pro­duced like this:

cd css
./postprocess.py m-dark.css m-documentation.css -o m-dark+documentation.compiled.css
./postprocess.py m-dark.css m-documentation.css --no-import -o m-dark.documentation.compiled.css

Out­put tem­plates

Each out­put file is rendered with one of these tem­plates:

Each tem­plate gets passed all con­fig­ur­a­tion val­ues from the Con­fig­ur­a­tion table as-is, to­geth­er with a URL vari­able with URL of giv­en out­put file. In ad­di­tion to built­in Jin­ja2 fil­ters, the format_url fil­ter re­turns either a path format­ted ac­cord­ing to cus­tom URL format­ters, if the path is re­l­at­ive; or a full URL, if the ar­gu­ment is an ab­so­lute URL. It’s use­ful in cases like this:

{% for css in HTML_EXTRA_STYLESHEET %}
<link rel="stylesheet" href="{{ css|format_url|e }}" />
{% endfor %}

The ac­tu­al page con­tents are provided in a page ob­ject, which has the fol­low­ing prop­er­ties. All ex­posed data are meant to be passed dir­ectly to the HTML markup without any ad­di­tion­al es­cap­ing.

Each mod­ule page, rendered with module.html, has the fol­low­ing ad­di­tion­al prop­er­ties:

Each class page, rendered with class.html, has the fol­low­ing ad­di­tion­al prop­er­ties:

Ex­pli­cit doc­u­ment­a­tion pages rendered with class.html have ad­di­tion­al prop­er­ties taken from in­put metadata. If giv­en metadata is lis­ted in FORMATTED_METADATA, it’s rendered in­to HTML, oth­er­wise it’s ex­posed as plain text.

In­dex page tem­plates

The fol­low­ing in­dex pages are provided, show­ing a ex­pand­able tree of the con­tents:

Each tem­plate is passed all con­fig­ur­a­tion val­ues from the Con­fig­ur­a­tion table as-is, to­geth­er with an URL, as above. The nav­ig­a­tion tree is provided in an index ob­ject, which has the fol­low­ing prop­er­ties:

The form of each list entry is the same:

Mod­ule/class list is ordered in a way that all mod­ules are be­fore all classes.

2.

^ a b c i.type is ex­trac­ted out of func­tion an­nota­tion. If the types aren’t an­not­ated, the an­nota­tion is empty.

3.

^ a b c d e f g h page.has_*_details and i.has_details are True if there is de­tailed de­scrip­tion, func­tion para­met­er doc­u­ment­a­tion or doc­u­men­ted enum value list­ing that makes it worth to render the full de­scrip­tion block. If False, the mem­ber should be in­cluded only in the sum­mary list­ing on top of the page to avoid un­ne­ces­sary re­pe­ti­tion.

4.

^ a b page.filename and page.url is gen­er­ated by an URL format­ter, see Cus­tom URL format­ters for more in­form­a­tion

5.

^ a b c d e i.id is an ID used for link­ing to giv­en entry on a page. Gen­er­ated by an an­chor format­ter, see Cus­tom URL format­ters for more in­form­a­tion.