Sphinx eager ".. only::" directive and other selective rendition extensions

Project home page: https://github.com/pfalcon/sphinx_selective_exclude

The implementation of ".. only::" directive in Sphinx documentation

generation tool is known to violate principles of least user surprise

and user expectations in general. Instead of excluding content early

in the pipeline (pre-processor style), Sphinx defers exclusion until

output phase, and what's the worst, various stages processing ignore

"only" blocks and their exclusion status, so they may leak unexpected

information into ToC, indexes, etc.

There's multiple issues submitted upstream on this matter:

* https://github.com/sphinx-doc/sphinx/issues/2150

* https://github.com/sphinx-doc/sphinx/issues/1717

* https://github.com/sphinx-doc/sphinx/issues/1488

* etc.

They are largely ignored by Sphinx maintainers.

This projects tries to rectify situation on users' side. It actually

changes the way Sphinx processes "only" directive, but does this

without forking the project, and instead is made as a standard

Sphinx extension, which a user may add to their documentation config.

Unlike normal extensions, extensions provided in this package

monkey-patch Sphinx core to work in a way expected by users.

eager_only

The core extension provided by the package is called `eager_only` and

is based on the idea by Andrea Cassioli (see bugreports above) to

process "only" directive as soon as possible during parsing phase.

This approach has some drawbacks, like producing warnings like

"WARNING: document isn't included in any toctree" if "only" is used

to shape up a toctree, or the fact that changing a documentation

builder (html/latex/etc.) will almost certainly require complete

rebuild of documentation. But these are relatively minor issues

comparing to completely broken way "only" works in upstream Sphinx.

modindex_exclude

"only" directive allows for fine-grained conditional exclusion, but

sometimes you may want to exclude entire module(s) at once. Even if

you wrap an entire module description in "only" directive, like:

.. only: option1

.. module:: my_module

...

You will still have an HTML page generated, albeit empty. It may also

go into indexes, so will be discoverable by users, leading to less

than ideal experience. `modindex_exclude` extension is design to

resolve this issue, by making sure that any reference of a module

is excluded from Python module index ("modindex"), as well as

general cross-reference index ("genindex"). In the latter case,

any symbol belong to a module will be excluded. Unlike `eager_only`

extension which appear to have issued with "latexpdf" builder,

`modindex_exclude` is useful for PDF, and allows to get cleaner

index for PDF, just the same as for HTML.

search_auto_exclude

Even if you exclude soem documents from toctree:: using only::

directive, they will be indexed for full-text search, so user may

find them and get confused. This plugin follows very simple idea

that if you didn't include some documents in the toctree, then

you didn't want them to be accessible (e.g. for a particular

configuration), and so will make sure they aren't indexed either.

This extension depends on `eager_only` and won't work without it.

Note that Sphinx will issue warnings, as usual, for any documents

not included in a toctree. This is considered a feature, and gives

you a chance to check that document exclusions are indeed right

for a particular configuration you build (and not that you forgot

to add something to a toctree).

Summary

Based on the above, sphinx_selective_exclude offers extension to let

you:

* Make "only::" directive work in an expected, intuitive manner, using

`eager_only` extension.

* However, if you apply only:: to toctree::, excluded documents will

still be available via full-text search, so you need to use

`search_auto_exclude` for that to work as expected.

* Similar to search, indexes may also require special treatment, hence

there's the `modindex_exclude` extension.

Most likely, you will want to use all 3 extensions together - if you

really want build subsets of docimentation covering sufficiently different

configurations from a single doctree. However, if one of them is enough

to cover your usecase, that's OK to (and why they were separated into

3 extensions, to follow KISS and "least surprise" principles and to

not make people deal with things they aren't interested in). In this case,

however remember there're other extensions, if you later hit a usecase

when they're needed.

Usage

To use these extensions, add https://github.com/pfalcon/sphinx_selective_exclude

as a git submodule to your project, in documentation folder (where

Sphinx conf.py is located). Alternatively, commit sphinx_selective_exclude

directory instead of making it a submodule (you will need to pick up

any project updates manually then).

Add following lines to "extensions" settings in your conf.py (you

likely already have some standard Sphinx extensions enabled):

extensions = [

...

'sphinx_selective_exclude.eager_only',

'sphinx_selective_exclude.search_auto_exclude',

'sphinx_selective_exclude.modindex_exclude',

]

As discussed above, you may enable all extensions, or one by one.

Please note that to make sure these extensions work well and avoid producing

output docs with artifacts, it is IMPERATIVE to remove cached doctree if

you rebuild documentation with another builder (i.e. with different output

format). Also, to stay on safe side, it's recommended to remove old doctree

anyway before generating production-ready documentation for publishing. To

do that, run something like:

rm -rf _build/doctrees/

A typical artificat when not following these simple rules is that content

of some sections may be missing. If you face anything like that, just

remember what's written above and remove cached doctrees.

import sphinx

from docutils.parsers.rst import directives

class EagerOnly(sphinx.directives.other.Only):

def run(self, *args):

# Evaluate the condition eagerly, and if false return no nodes right away

env = self.state.document.settings.env

env.app.builder.tags.add('TRUE')

#print(repr(self.arguments[0]))

if not env.app.builder.tags.eval_condition(self.arguments[0]):

return []

# Otherwise, do the usual processing

nodes = super(EagerOnly, self).run()

if len(nodes) == 1:

nodes[0]['expr'] = 'TRUE'

return nodes

def setup(app):

directives.register_directive('only', EagerOnly)

import sphinx

org_PyObject_add_target_and_index = None

org_PyModule_run = None

EXCLUDES = {}

def PythonModuleIndex_generate(self, docnames=None):

docnames = []

excludes = self.domain.env.config['modindex_exclude']

for modname, (docname, synopsis, platforms, deprecated) in self.domain.data['modules'].items():

#print(docname)

if modname not in excludes:

docnames.append(docname)

return org_PythonModuleIndex_generate(self, docnames)

def PyObject_add_target_and_index(self, name_cls, sig, signode):

if hasattr(self.env, "ref_context"):

# Sphinx 1.4

ref_context = self.env.ref_context

else:

# Sphinx 1.2

ref_context = self.env.temp_data

modname = self.options.get(

'module', ref_context.get('py:module'))

#print("*", modname, name_cls)

if modname in self.env.config['modindex_exclude']:

return None

return org_PyObject_add_target_and_index(self, name_cls, sig, signode)

def PyModule_run(self):

env = self.state.document.settings.env

modname = self.arguments[0].strip()

excl = env.config['modindex_exclude']

if modname in excl:

self.options['noindex'] = True

EXCLUDES.setdefault(modname, []).append(env.docname)

return org_PyModule_run(self)

def setup(app):

app.add_config_value('modindex_exclude', [], 'html')

global org_PyObject_add_target_and_index

org_PyObject_add_target_and_index = sphinx.domains.python.PyObject.add_target_and_index

sphinx.domains.python.PyObject.add_target_and_index = PyObject_add_target_and_index

global org_PyModule_run

org_PyModule_run = sphinx.domains.python.PyModule.run

sphinx.domains.python.PyModule.run = PyModule_run

import sphinx

org_StandaloneHTMLBuilder_index_page = None

def StandaloneHTMLBuilder_index_page(self, pagename, doctree, title):

if pagename not in self.env.files_to_rebuild:

if pagename != self.env.config.master_doc and 'orphan' not in self.env.metadata[pagename]:

print("Excluding %s from full-text index because it's not referenced in ToC" % pagename)

return

return org_StandaloneHTMLBuilder_index_page(self, pagename, doctree, title)

def setup(app):

global org_StandaloneHTMLBuilder_index_page

org_StandaloneHTMLBuilder_index_page = sphinx.builders.html.StandaloneHTMLBuilder.index_page

sphinx.builders.html.StandaloneHTMLBuilder.index_page = StandaloneHTMLBuilder_index_page