Documentation builders

Note

If you are a developer and want to build the SageMath documentation from source, refer to developer’s guide.

This module is the starting point for building documentation, and is responsible to figure out what to build and with which options. The actual documentation build for each individual document is then done in a subprocess call to Sphinx, see builder_helper(). Note that

  • The builders are configured with build_options.py;

  • The Sphinx subprocesses are configured in conf.py.

DocBuilder is the base class of all Builders. It has builder helpers html(), latex(), pdf(), inventory(), etc, which are invoked depending on the output type. Each type corresponds with the Sphinx builder format, except that pdf() is Sphinx latex builder plus compiling latex to pdf. Note that Sphinx inventory builder is not native to Sphinx but provided by Sage. See sage_docbuild.ext.inventory_builder. The Sphinx inventory builder is a dummy builder with no actual output but produces doctree files in $SAGE_DOC/doctrees and objects.inv inventory files in $SAGE_DOC/inventory.

The reference manual is built in two passes, first by ReferenceBuilder with inventory output type and secondly with html output type. The ReferenceBuilder itself uses ReferenceTopBuilder and ReferenceSubBuilder to build subcomponents of the reference manual. The ReferenceSubBuilder examines the modules included in the subcomponent by comparing the modification times of the module files with the times saved in local/share/doctree/reference.pickle from the previous build. Then new rst files are generated for new and updated modules. See get_new_and_updated_modules().

After Issue #31948, when Sage is built, ReferenceBuilder is not used and its responsibility is now taken by the Makefile in $SAGE_ROOT/src/doc.

class sage_docbuild.builders.DocBuilder(name: str, options: BuildOptions)[source]

Bases: object

INPUT:

  • name – the name of a document directory below doc, such as ‘en/tutorial’ or ‘fr/tutorial’

changes(*args, **kwds)[source]

Build the documentation for output type changes.

clean(*args)[source]
documents_single_file = False

Whether this builder documents one file of its own rather than a manual; see SingleFileBuilder and builder_helper().

html(*args, **kwds)[source]

Build the documentation for output type html.

htmlhelp(*args, **kwds)[source]

Build the documentation for output type htmlhelp.

inventory(*args, **kwds)[source]

Build the documentation for output type inventory.

json(*args, **kwds)[source]

Build the documentation for output type json.

latex(*args, **kwds)[source]

Build the documentation for output type latex.

linkcheck(*args, **kwds)[source]

Build the documentation for output type linkcheck.

pdf()[source]

Build the PDF files for this document.

This is done by first (re)-building the LaTeX output, going into that LaTeX directory, and running ‘make all-pdf’ there.

EXAMPLES:

sage: from sage_docbuild.builders import DocBuilder
sage: from sage_docbuild.build_options import BuildOptions
sage: options = BuildOptions(source_dir = Path('src/doc'))
sage: builder = DocBuilder('tutorial', options)
sage: builder.pdf() #not tested

>>> from sage.all import *
>>> from sage_docbuild.builders import DocBuilder
>>> from sage_docbuild.build_options import BuildOptions
>>> options = BuildOptions(source_dir = Path('src/doc'))
>>> builder = DocBuilder('tutorial', options)
>>> builder.pdf() #not tested

from sage_docbuild.builders import DocBuilder
from sage_docbuild.build_options import BuildOptions
options = BuildOptions(source_dir = Path('src/doc'))
builder = DocBuilder('tutorial', options)
builder.pdf() #not tested
pickle(*args, **kwds)[source]

Build the documentation for output type pickle.

web(*args, **kwds)[source]

alias of pickle().

class sage_docbuild.builders.ReferenceBuilder(name: str, options: BuildOptions)[source]

Bases: object

This class builds the reference manual. It uses DocBuilder to build the top-level page and ReferenceSubBuilder for each sub-component.

print_unincluded_modules()[source]

Print the modules of the Sage library that the reference manual does not include.

A sub-document knows the modules of its own toctrees only, so running the command of ReferenceSubBuilder over each of them in turn would report every module that any other sub-document documents. The answer is the complement of their union.

class sage_docbuild.builders.ReferenceSubBuilder(name: str, options: BuildOptions)[source]

Bases: DocBuilder

This class builds sub-components of the reference manual. It is responsible for making sure that the auto generated reST files for the Sage library are up to date.

When building any output, we must first go through and check to see if we need to update any of the autogenerated reST files. There are two cases where this would happen:

  1. A new module gets added to one of the toctrees.

  2. The actual module gets updated and possibly contains a new title.

auto_rest_filename(module_name)[source]

Return the name of the file associated to a given module

EXAMPLES:

sage: from sage_docbuild.builders import ReferenceSubBuilder
sage: from sage_docbuild.build_options import BuildOptions
sage: options = BuildOptions(source_dir = Path('src/doc'))
sage: ReferenceSubBuilder("reference", options).auto_rest_filename("sage.combinat.partition")
...Path('src/doc/en/reference/sage/combinat/partition.rst')

>>> from sage.all import *
>>> from sage_docbuild.builders import ReferenceSubBuilder
>>> from sage_docbuild.build_options import BuildOptions
>>> options = BuildOptions(source_dir = Path('src/doc'))
>>> ReferenceSubBuilder("reference", options).auto_rest_filename("sage.combinat.partition")
...Path('src/doc/en/reference/sage/combinat/partition.rst')

from sage_docbuild.builders import ReferenceSubBuilder
from sage_docbuild.build_options import BuildOptions
options = BuildOptions(source_dir = Path('src/doc'))
ReferenceSubBuilder("reference", options).auto_rest_filename("sage.combinat.partition")
cache_file()[source]

Return the filename where the pickle of the reference cache is stored.

clean_auto()[source]

Remove all autogenerated reST files.

get_all_included_modules()[source]

Return an iterator for all modules which are included in the reference manual.

get_all_rst_files()[source]

Return an iterator for all rst files which are not autogenerated.

get_cache()[source]

Retrieve the reference cache which contains the options previously used by the reference builder.

If it doesn’t exist, then we just return an empty dictionary. If it is corrupted, return an empty dictionary.

get_modified_modules()[source]

Return an iterator for all the modules that have been modified since the documentation was last built.

get_module_docstring_title(module_name)[source]

Return the title of the module from its docstring.

get_modules(file)[source]

Given a reST file, return an iterator for all of the autogenerated reST files that it includes.

get_new_and_updated_modules()[source]

Return an iterator for all new and updated modules that appear in the toctrees, and remove obsolete old modules.

get_sphinx_environment()[source]

Return the Sphinx environment for this project.

get_unincluded_modules()[source]

Return an iterator for all the modules in the Sage library which are not included in this document.

Beware that a module the reference manual documents elsewhere is not included here; ReferenceBuilder.print_unincluded_modules() answers for the manual as a whole.

print_included_modules()[source]

Print all of the modules that are included in the Sage reference manual.

print_modified_modules()[source]

Print a list of all the modules that have been modified since the documentation was last built.

print_new_and_updated_modules()[source]

Print all the modules that appear in the toctrees that are newly included or updated.

print_unincluded_modules()[source]

Print all of the modules which are not included in the Sage reference manual.

save_cache()[source]

Pickle the current reference cache for later retrieval.

update_mtimes()[source]

Update the modification times for reST files in the Sphinx environment for this project.

write_auto_rest_file(module_name)[source]

Write the autogenerated reST file for module_name.

class sage_docbuild.builders.ReferenceTopBuilder(name: str, options: BuildOptions)[source]

Bases: DocBuilder

This class builds the top-level page of the reference manual.

html()[source]

Build the top-level document.

class sage_docbuild.builders.SingleFileBuilder(path: str, options: BuildOptions)[source]

Bases: DocBuilder

This is the class used to build the documentation for a single user-specified file. If the file is called ‘foo.py’, then the documentation is built in DIR/foo/ if the user passes the command line option “-o DIR”, or in DOT_SAGE/docbuild/foo/ otherwise.

documents_single_file = True

Whether this builder documents one file of its own rather than a manual; see SingleFileBuilder and builder_helper().

pdf()[source]

Build a PDF, recording success only once the whole build ends.

class sage_docbuild.builders.WebsiteBuilder(name: str, options: BuildOptions)[source]

Bases: DocBuilder

clean()[source]

When we clean the output for the website index, we need to remove all of the HTML that were placed in the parent directory.

In addition, remove the index file installed into the root doc directory.

html()[source]

After we have finished building the website index page, we copy everything one directory up, that is, to the base diectory html/en.

In addition, an index file is installed into the root doc directory.

Thus we have three index.html files:

html/en/website/index.html (not used) html/en/index.html (base directory) index.html (root doc directory)

pdf()[source]

Build the website hosting pdf docs.

sage_docbuild.builders.build_many(target, args, processes=None)[source]

Thin wrapper around sage_docbuild.utils.build_many() which uses the docbuild settings NUM_THREADS and ABORT_ON_ERROR.

sage_docbuild.builders.build_ref_doc(args)[source]
sage_docbuild.builders.builder_helper(type)[source]

Return a function which builds the documentation for output type type.

sage_docbuild.builders.get_all_documents(source)[source]

Return a list of all of the documents, relative to the source directory.

A document is a directory within one of the language subdirectories of doc.

EXAMPLES:

sage: from sage_docbuild.builders import get_all_documents
sage: from sage.env import SAGE_DOC_SRC
sage: documents = get_all_documents(Path(SAGE_DOC_SRC))
sage: Path('en/tutorial') in documents
True

>>> from sage.all import *
>>> from sage_docbuild.builders import get_all_documents
>>> from sage.env import SAGE_DOC_SRC
>>> documents = get_all_documents(Path(SAGE_DOC_SRC))
>>> Path('en/tutorial') in documents
True

from sage_docbuild.builders import get_all_documents
from sage.env import SAGE_DOC_SRC
documents = get_all_documents(Path(SAGE_DOC_SRC))
Path('en/tutorial') in documents

A directory without the root source that Sphinx would build is not a document:

sage: import tempfile
sage: with tempfile.TemporaryDirectory() as directory:
....:     source = Path(directory)
....:     empty = source / 'en' / 'empty'
....:     complete = source / 'en' / 'complete'
....:     empty.mkdir(parents=True)
....:     complete.mkdir()
....:     _ = (complete / 'index.rst').write_text('Complete\n========\n')
....:     get_all_documents(source) == [Path('en/complete')]
True

>>> from sage.all import *
>>> import tempfile
>>> with tempfile.TemporaryDirectory() as directory:
...     source = Path(directory)
...     empty = source / 'en' / 'empty'
...     complete = source / 'en' / 'complete'
...     empty.mkdir(parents=True)
...     complete.mkdir()
...     _ = (complete / 'index.rst').write_text('Complete\n========\n')
...     get_all_documents(source) == [Path('en/complete')]
True

import tempfile
with tempfile.TemporaryDirectory() as directory:
    source = Path(directory)
    empty = source / 'en' / 'empty'
    complete = source / 'en' / 'complete'
    empty.mkdir(parents=True)
    complete.mkdir()
    _ = (complete / 'index.rst').write_text('Complete\n========\n')
    get_all_documents(source) == [Path('en/complete')]
sage_docbuild.builders.get_all_reference_documents(source)[source]

Return a list of all reference manual documents to build, relative to the specified source directory.

We add a document if it’s a subdirectory of the manual’s directory and contains a file named ‘index.rst’.

The order corresponds to the order in which the documents should be built.

EXAMPLES:

sage: from sage_docbuild.builders import get_all_reference_documents
sage: from sage.env import SAGE_DOC_SRC
sage: documents = get_all_reference_documents(Path(SAGE_DOC_SRC) / 'en')
sage: Path('reference/algebras') in documents
True

>>> from sage.all import *
>>> from sage_docbuild.builders import get_all_reference_documents
>>> from sage.env import SAGE_DOC_SRC
>>> documents = get_all_reference_documents(Path(SAGE_DOC_SRC) / 'en')
>>> Path('reference/algebras') in documents
True

from sage_docbuild.builders import get_all_reference_documents
from sage.env import SAGE_DOC_SRC
documents = get_all_reference_documents(Path(SAGE_DOC_SRC) / 'en')
Path('reference/algebras') in documents
sage_docbuild.builders.get_builder(name, options)[source]

Return an appropriate Builder object for the document name.

DocBuilder and its subclasses do all the real work in building the documentation.

The name is one of those that get_documents() lists, so an English document may be named without its en/ prefix.

EXAMPLES:

sage: from sage.env import SAGE_DOC_SRC
sage: from sage_docbuild.build_options import BuildOptions
sage: from sage_docbuild.builders import get_builder
sage: options = BuildOptions(source_dir=Path(SAGE_DOC_SRC))
sage: get_builder('developer', options).name
'en/developer'
sage: get_builder('en/developer', options).name
'en/developer'
sage: get_builder('fr/tutorial', options).name
'fr/tutorial'

>>> from sage.all import *
>>> from sage.env import SAGE_DOC_SRC
>>> from sage_docbuild.build_options import BuildOptions
>>> from sage_docbuild.builders import get_builder
>>> options = BuildOptions(source_dir=Path(SAGE_DOC_SRC))
>>> get_builder('developer', options).name
'en/developer'
>>> get_builder('en/developer', options).name
'en/developer'
>>> get_builder('fr/tutorial', options).name
'fr/tutorial'

from sage.env import SAGE_DOC_SRC
from sage_docbuild.build_options import BuildOptions
from sage_docbuild.builders import get_builder
options = BuildOptions(source_dir=Path(SAGE_DOC_SRC))
get_builder('developer', options).name
get_builder('en/developer', options).name
get_builder('fr/tutorial', options).name

The website has a builder of its own, and reaches it under either name:

sage: type(get_builder('website', options)).__name__
'WebsiteBuilder'
sage: get_builder('website', options).name
'en/website'

>>> from sage.all import *
>>> type(get_builder('website', options)).__name__
'WebsiteBuilder'
>>> get_builder('website', options).name
'en/website'

type(get_builder('website', options)).__name__
get_builder('website', options).name

A document merely ending in reference is not mistaken for the Sage reference manual:

sage: import contextlib, io, tempfile
sage: with tempfile.TemporaryDirectory() as directory:
....:     source = Path(directory)
....:     document = source / 'fr' / 'myreference'
....:     document.mkdir(parents=True)
....:     _ = (document / 'index.rst').write_text('Title\n=====\n')
....:     english = source / 'en'
....:     english.mkdir()
....:     _ = (english / 'index.rst').write_text('English\n=======\n')
....:     local = BuildOptions(source_dir=source)
....:     answer = get_builder('fr/myreference', local)
....:     with contextlib.redirect_stdout(io.StringIO()):
....:         try:
....:             get_builder('reference/..', local)
....:         except SystemExit:
....:             traversal_rejected = True
sage: type(answer).__name__, answer.name, traversal_rejected
('DocBuilder', 'fr/myreference', True)

>>> from sage.all import *
>>> import contextlib, io, tempfile
>>> with tempfile.TemporaryDirectory() as directory:
...     source = Path(directory)
...     document = source / 'fr' / 'myreference'
...     document.mkdir(parents=True)
...     _ = (document / 'index.rst').write_text('Title\n=====\n')
...     english = source / 'en'
...     english.mkdir()
...     _ = (english / 'index.rst').write_text('English\n=======\n')
...     local = BuildOptions(source_dir=source)
...     answer = get_builder('fr/myreference', local)
...     with contextlib.redirect_stdout(io.StringIO()):
...         try:
...             get_builder('reference/..', local)
...         except SystemExit:
...             traversal_rejected = True
>>> type(answer).__name__, answer.name, traversal_rejected
('DocBuilder', 'fr/myreference', True)

import contextlib, io, tempfile
with tempfile.TemporaryDirectory() as directory:
    source = Path(directory)
    document = source / 'fr' / 'myreference'
    document.mkdir(parents=True)
    _ = (document / 'index.rst').write_text('Title\n=====\n')
    english = source / 'en'
    english.mkdir()
    _ = (english / 'index.rst').write_text('English\n=======\n')
    local = BuildOptions(source_dir=source)
    answer = get_builder('fr/myreference', local)
    with contextlib.redirect_stdout(io.StringIO()):
        try:
            get_builder('reference/..', local)
        except SystemExit:
            traversal_rejected = True
type(answer).__name__, answer.name, traversal_rejected
sage_docbuild.builders.get_documents(source)[source]

Return the documents that the documentation builder accepts as command-line arguments, in the order in which they are built.

These are the documents of get_all_documents(), named the way the command line names them: the English ones without their language prefix, and the reference manual - which get_all_documents() leaves out, its top level having a builder of its own - in front.

EXAMPLES:

sage: from sage.env import SAGE_DOC_SRC
sage: from sage_docbuild.builders import get_documents
sage: documents = get_documents(Path(SAGE_DOC_SRC))
sage: documents[0]
'reference'
sage: 'tutorial' in documents
True
sage: 'en/tutorial' in documents
False

>>> from sage.all import *
>>> from sage.env import SAGE_DOC_SRC
>>> from sage_docbuild.builders import get_documents
>>> documents = get_documents(Path(SAGE_DOC_SRC))
>>> documents[Integer(0)]
'reference'
>>> 'tutorial' in documents
True
>>> 'en/tutorial' in documents
False

from sage.env import SAGE_DOC_SRC
from sage_docbuild.builders import get_documents
documents = get_documents(Path(SAGE_DOC_SRC))
documents[0]
'tutorial' in documents
'en/tutorial' in documents
sage_docbuild.builders.load_single_file(*args, **kwds)[source]

Import the file path under the module name name, and return it.

This is what the configuration of a single-file build runs when importing name would read another file - the same module of a second checkout, say - or no file at all. The modules beside path come before the ones of that other copy, so that what the file imports is what sits next to it; a module of that copy which is loaded already is dropped, since an import would answer with it and never read this directory.

Modules that path reaches by any other route are the ones already loaded, which is what the caller warns about.

For an ordinary module, the already-active containing package is retained; its initializer is not re-executed from the alternate tree. When path is itself __init__.py, that package initializer is the explicit target and is replaced together with its children.

EXAMPLES:

sage: import os, sys, tempfile
sage: from sage_docbuild.builders import (load_single_file,
....:     _clear_single_file_imports)
sage: with tempfile.TemporaryDirectory() as directory:
....:     package = os.path.join(directory, 'pkg_2718')
....:     os.mkdir(package)
....:     _ = open(os.path.join(package, '__init__.py'), 'w').close()
....:     with open(os.path.join(package, 'helper.py'), 'w') as f:
....:         _ = f.write('answer = 42\n')
....:     with open(os.path.join(package, 'main.py'), 'w') as f:
....:         _ = f.write('import _imp\nfrom .helper import answer\n'
....:                     'lock_held = _imp.lock_held()\n')
....:     sys.path.insert(0, directory)
....:     module = load_single_file('pkg_2718.main',
....:                               os.path.join(package, 'main.py'))
....:     _clear_single_file_imports()
....:     sys.path.remove(directory)
sage: module.answer
42
sage: module.__name__
'pkg_2718.main'
sage: module.lock_held
False

>>> from sage.all import *
>>> import os, sys, tempfile
>>> from sage_docbuild.builders import (load_single_file,
...     _clear_single_file_imports)
>>> with tempfile.TemporaryDirectory() as directory:
...     package = os.path.join(directory, 'pkg_2718')
...     os.mkdir(package)
...     _ = open(os.path.join(package, '__init__.py'), 'w').close()
...     with open(os.path.join(package, 'helper.py'), 'w') as f:
...         _ = f.write('answer = 42\n')
...     with open(os.path.join(package, 'main.py'), 'w') as f:
...         _ = f.write('import _imp\nfrom .helper import answer\n'
...                     'lock_held = _imp.lock_held()\n')
...     sys.path.insert(Integer(0), directory)
...     module = load_single_file('pkg_2718.main',
...                               os.path.join(package, 'main.py'))
...     _clear_single_file_imports()
...     sys.path.remove(directory)
>>> module.answer
42
>>> module.__name__
'pkg_2718.main'
>>> module.lock_held
False

import os, sys, tempfile
from sage_docbuild.builders import (load_single_file,
    _clear_single_file_imports)
with tempfile.TemporaryDirectory() as directory:
    package = os.path.join(directory, 'pkg_2718')
    os.mkdir(package)
    _ = open(os.path.join(package, '__init__.py'), 'w').close()
    with open(os.path.join(package, 'helper.py'), 'w') as f:
        _ = f.write('answer = 42\n')
    with open(os.path.join(package, 'main.py'), 'w') as f:
        _ = f.write('import _imp\nfrom .helper import answer\n'
                    'lock_held = _imp.lock_held()\n')
    sys.path.insert(0, directory)
    module = load_single_file('pkg_2718.main',
                              os.path.join(package, 'main.py'))
    _clear_single_file_imports()
    sys.path.remove(directory)
module.answer
module.__name__
module.lock_held

Already-loaded children and descendants of another copy are replaced, and a failed replacement restores them:

sage: import importlib
sage: from sage_docbuild.builders import (_SiblingFinder,
....:     _module_path)
sage: with tempfile.TemporaryDirectory() as active, \
....:      tempfile.TemporaryDirectory() as alternate:
....:     active_pkg = Path(active) / 'pkg_shadow_2718'
....:     alternate_pkg = Path(alternate) / 'pkg_shadow_2718'
....:     (active_pkg / 'nested').mkdir(parents=True)
....:     (alternate_pkg / 'nested').mkdir(parents=True)
....:     for package in (active_pkg, alternate_pkg,
....:                     active_pkg / 'nested', alternate_pkg / 'nested'):
....:         _ = (package / '__init__.py').write_text('')
....:     _ = (active_pkg / 'helper.py').write_text("VALUE = 'active'\n")
....:     _ = (active_pkg / 'nested' / 'helper.py').write_text(
....:         "VALUE = 'active nested'\n")
....:     _ = (active_pkg / 'broken.py').write_text("VALUE = 'old'\n")
....:     _ = (active_pkg / 'fallback.py').write_text(
....:         "VALUE = 'active fallback'\n")
....:     _ = (Path(active) / 'unrelated_tx_2718.py').write_text(
....:         "VALUE = 'preserved'\n")
....:     _ = (alternate_pkg / 'helper.py').write_text(
....:         "VALUE = 'alternate'\n")
....:     _ = (alternate_pkg / 'nested' / 'helper.py').write_text(
....:         "VALUE = 'alternate nested'\n")
....:     _ = (alternate_pkg / 'main.py').write_text(
....:         'from . import helper\n'
....:         'from .nested.helper import VALUE as NESTED\n'
....:         'VALUE = helper.VALUE\n')
....:     sys.path.insert(0, active)
....:     package = importlib.import_module('pkg_shadow_2718')
....:     old_helper = importlib.import_module('pkg_shadow_2718.helper')
....:     old_nested = importlib.import_module(
....:         'pkg_shadow_2718.nested.helper')
....:     main = load_single_file(
....:         'pkg_shadow_2718.main', str(alternate_pkg / 'main.py'))
....:     values = (main.VALUE, main.NESTED,
....:               package.helper is not old_helper,
....:               sys.modules['pkg_shadow_2718.nested.helper'] is not old_nested)
....:     _clear_single_file_imports()
....:     active_restored = (
....:         package.helper is old_helper
....:         and sys.modules['pkg_shadow_2718.nested.helper'] is old_nested
....:         and _module_path('pkg_shadow_2718.helper')
....:             == os.path.realpath(active_pkg / 'helper.py')
....:         and not any(isinstance(item, _SiblingFinder)
....:                     and item.package == 'pkg_shadow_2718'
....:                     for item in sys.meta_path))
....:     old_broken = importlib.import_module('pkg_shadow_2718.broken')
....:     _ = sys.modules.pop('unrelated_tx_2718', None)
....:     _ = (alternate_pkg / 'broken.py').write_text(
....:         "import unrelated_tx_2718\n"
....:         "from . import fallback\n"
....:         "VALUE = 'partial'\nraise RuntimeError('broken')\n")
....:     meta_path = list(sys.meta_path)
....:     try:
....:         load_single_file('pkg_shadow_2718.broken',
....:                          str(alternate_pkg / 'broken.py'))
....:     except RuntimeError:
....:         restored = (sys.modules['pkg_shadow_2718.broken'] is old_broken
....:                     and package.broken is old_broken
....:                     and sys.meta_path == meta_path)
....:     unrelated_preserved = 'unrelated_tx_2718' in sys.modules
....:     fallback_preserved = (
....:         package.fallback
....:         is sys.modules['pkg_shadow_2718.fallback'])
....:     _ = sys.modules.pop('unrelated_tx_2718', None)
....:     sys.path.remove(active)
....:     for loaded in list(sys.modules):
....:         if loaded == 'pkg_shadow_2718' or loaded.startswith(
....:                 'pkg_shadow_2718.'):
....:             del sys.modules[loaded]
....:     sys.meta_path[:] = [finder for finder in sys.meta_path
....:                         if not (isinstance(finder, _SiblingFinder)
....:                                 and finder.package == 'pkg_shadow_2718')]
sage: (values, active_restored, restored, unrelated_preserved,
....:  fallback_preserved)
(('alternate', 'alternate nested', True, True), True, True, True, True)

>>> from sage.all import *
>>> import importlib
>>> from sage_docbuild.builders import (_SiblingFinder,
...     _module_path)
>>> with tempfile.TemporaryDirectory() as active,      tempfile.TemporaryDirectory() as alternate:
...     active_pkg = Path(active) / 'pkg_shadow_2718'
...     alternate_pkg = Path(alternate) / 'pkg_shadow_2718'
...     (active_pkg / 'nested').mkdir(parents=True)
...     (alternate_pkg / 'nested').mkdir(parents=True)
...     for package in (active_pkg, alternate_pkg,
...                     active_pkg / 'nested', alternate_pkg / 'nested'):
...         _ = (package / '__init__.py').write_text('')
...     _ = (active_pkg / 'helper.py').write_text("VALUE = 'active'\n")
...     _ = (active_pkg / 'nested' / 'helper.py').write_text(
...         "VALUE = 'active nested'\n")
...     _ = (active_pkg / 'broken.py').write_text("VALUE = 'old'\n")
...     _ = (active_pkg / 'fallback.py').write_text(
...         "VALUE = 'active fallback'\n")
...     _ = (Path(active) / 'unrelated_tx_2718.py').write_text(
...         "VALUE = 'preserved'\n")
...     _ = (alternate_pkg / 'helper.py').write_text(
...         "VALUE = 'alternate'\n")
...     _ = (alternate_pkg / 'nested' / 'helper.py').write_text(
...         "VALUE = 'alternate nested'\n")
...     _ = (alternate_pkg / 'main.py').write_text(
...         'from . import helper\n'
...         'from .nested.helper import VALUE as NESTED\n'
...         'VALUE = helper.VALUE\n')
...     sys.path.insert(Integer(0), active)
...     package = importlib.import_module('pkg_shadow_2718')
...     old_helper = importlib.import_module('pkg_shadow_2718.helper')
...     old_nested = importlib.import_module(
...         'pkg_shadow_2718.nested.helper')
...     main = load_single_file(
...         'pkg_shadow_2718.main', str(alternate_pkg / 'main.py'))
...     values = (main.VALUE, main.NESTED,
...               package.helper is not old_helper,
...               sys.modules['pkg_shadow_2718.nested.helper'] is not old_nested)
...     _clear_single_file_imports()
...     active_restored = (
...         package.helper is old_helper
...         and sys.modules['pkg_shadow_2718.nested.helper'] is old_nested
...         and _module_path('pkg_shadow_2718.helper')
...             == os.path.realpath(active_pkg / 'helper.py')
...         and not any(isinstance(item, _SiblingFinder)
...                     and item.package == 'pkg_shadow_2718'
...                     for item in sys.meta_path))
...     old_broken = importlib.import_module('pkg_shadow_2718.broken')
...     _ = sys.modules.pop('unrelated_tx_2718', None)
...     _ = (alternate_pkg / 'broken.py').write_text(
...         "import unrelated_tx_2718\n"
...         "from . import fallback\n"
...         "VALUE = 'partial'\nraise RuntimeError('broken')\n")
...     meta_path = list(sys.meta_path)
...     try:
...         load_single_file('pkg_shadow_2718.broken',
...                          str(alternate_pkg / 'broken.py'))
...     except RuntimeError:
...         restored = (sys.modules['pkg_shadow_2718.broken'] is old_broken
...                     and package.broken is old_broken
...                     and sys.meta_path == meta_path)
...     unrelated_preserved = 'unrelated_tx_2718' in sys.modules
...     fallback_preserved = (
...         package.fallback
...         is sys.modules['pkg_shadow_2718.fallback'])
...     _ = sys.modules.pop('unrelated_tx_2718', None)
...     sys.path.remove(active)
...     for loaded in list(sys.modules):
...         if loaded == 'pkg_shadow_2718' or loaded.startswith(
...                 'pkg_shadow_2718.'):
...             del sys.modules[loaded]
...     sys.meta_path[:] = [finder for finder in sys.meta_path
...                         if not (isinstance(finder, _SiblingFinder)
...                                 and finder.package == 'pkg_shadow_2718')]
>>> (values, active_restored, restored, unrelated_preserved,
...  fallback_preserved)
(('alternate', 'alternate nested', True, True), True, True, True, True)

import importlib
from sage_docbuild.builders import (_SiblingFinder,
    _module_path)
with tempfile.TemporaryDirectory() as active, \
     tempfile.TemporaryDirectory() as alternate:
    active_pkg = Path(active) / 'pkg_shadow_2718'
    alternate_pkg = Path(alternate) / 'pkg_shadow_2718'
    (active_pkg / 'nested').mkdir(parents=True)
    (alternate_pkg / 'nested').mkdir(parents=True)
    for package in (active_pkg, alternate_pkg,
                    active_pkg / 'nested', alternate_pkg / 'nested'):
        _ = (package / '__init__.py').write_text('')
    _ = (active_pkg / 'helper.py').write_text("VALUE = 'active'\n")
    _ = (active_pkg / 'nested' / 'helper.py').write_text(
        "VALUE = 'active nested'\n")
    _ = (active_pkg / 'broken.py').write_text("VALUE = 'old'\n")
    _ = (active_pkg / 'fallback.py').write_text(
        "VALUE = 'active fallback'\n")
    _ = (Path(active) / 'unrelated_tx_2718.py').write_text(
        "VALUE = 'preserved'\n")
    _ = (alternate_pkg / 'helper.py').write_text(
        "VALUE = 'alternate'\n")
    _ = (alternate_pkg / 'nested' / 'helper.py').write_text(
        "VALUE = 'alternate nested'\n")
    _ = (alternate_pkg / 'main.py').write_text(
        'from . import helper\n'
        'from .nested.helper import VALUE as NESTED\n'
        'VALUE = helper.VALUE\n')
    sys.path.insert(0, active)
    package = importlib.import_module('pkg_shadow_2718')
    old_helper = importlib.import_module('pkg_shadow_2718.helper')
    old_nested = importlib.import_module(
        'pkg_shadow_2718.nested.helper')
    main = load_single_file(
        'pkg_shadow_2718.main', str(alternate_pkg / 'main.py'))
    values = (main.VALUE, main.NESTED,
              package.helper is not old_helper,
              sys.modules['pkg_shadow_2718.nested.helper'] is not old_nested)
    _clear_single_file_imports()
    active_restored = (
        package.helper is old_helper
        and sys.modules['pkg_shadow_2718.nested.helper'] is old_nested
        and _module_path('pkg_shadow_2718.helper')
            == os.path.realpath(active_pkg / 'helper.py')
        and not any(isinstance(item, _SiblingFinder)
                    and item.package == 'pkg_shadow_2718'
                    for item in sys.meta_path))
    old_broken = importlib.import_module('pkg_shadow_2718.broken')
    _ = sys.modules.pop('unrelated_tx_2718', None)
    _ = (alternate_pkg / 'broken.py').write_text(
        "import unrelated_tx_2718\n"
        "from . import fallback\n"
        "VALUE = 'partial'\nraise RuntimeError('broken')\n")
    meta_path = list(sys.meta_path)
    try:
        load_single_file('pkg_shadow_2718.broken',
                         str(alternate_pkg / 'broken.py'))
    except RuntimeError:
        restored = (sys.modules['pkg_shadow_2718.broken'] is old_broken
                    and package.broken is old_broken
                    and sys.meta_path == meta_path)
    unrelated_preserved = 'unrelated_tx_2718' in sys.modules
    fallback_preserved = (
        package.fallback
        is sys.modules['pkg_shadow_2718.fallback'])
    _ = sys.modules.pop('unrelated_tx_2718', None)
    sys.path.remove(active)
    for loaded in list(sys.modules):
        if loaded == 'pkg_shadow_2718' or loaded.startswith(
                'pkg_shadow_2718.'):
            del sys.modules[loaded]
    sys.meta_path[:] = [finder for finder in sys.meta_path
                        if not (isinstance(finder, _SiblingFinder)
                                and finder.package == 'pkg_shadow_2718')]
(values, active_restored, restored, unrelated_preserved,
 fallback_preserved)

A package initializer replaces its loaded children as well:

sage: with tempfile.TemporaryDirectory() as active, \
....:      tempfile.TemporaryDirectory() as alternate:
....:     for root, value in ((active, 'active'),
....:                         (alternate, 'alternate')):
....:         package = Path(root) / 'pkg_init_2718'
....:         package.mkdir()
....:         _ = (package / '__init__.py').write_text(
....:             'from .helper import VALUE\n')
....:         _ = (package / 'helper.py').write_text(
....:             f"VALUE = {value!r}\n")
....:     sys.path.insert(0, active)
....:     old_package = importlib.import_module('pkg_init_2718')
....:     old_helper = importlib.import_module('pkg_init_2718.helper')
....:     package = load_single_file(
....:         'pkg_init_2718',
....:         str(Path(alternate) / 'pkg_init_2718' / '__init__.py'))
....:     answer = (package.VALUE,
....:               sys.modules['pkg_init_2718.helper'] is not old_helper)
....:     _clear_single_file_imports()
....:     sys.path.remove(active)
....:     for loaded in list(sys.modules):
....:         if loaded == 'pkg_init_2718' or loaded.startswith(
....:                 'pkg_init_2718.'):
....:             del sys.modules[loaded]
....:     sys.meta_path[:] = [finder for finder in sys.meta_path
....:                         if not (isinstance(finder, _SiblingFinder)
....:                                 and finder.package == 'pkg_init_2718')]
sage: answer
('alternate', True)

>>> from sage.all import *
>>> with tempfile.TemporaryDirectory() as active,      tempfile.TemporaryDirectory() as alternate:
...     for root, value in ((active, 'active'),
...                         (alternate, 'alternate')):
...         package = Path(root) / 'pkg_init_2718'
...         package.mkdir()
...         _ = (package / '__init__.py').write_text(
...             'from .helper import VALUE\n')
...         _ = (package / 'helper.py').write_text(
...             f"VALUE = {value!r}\n")
...     sys.path.insert(Integer(0), active)
...     old_package = importlib.import_module('pkg_init_2718')
...     old_helper = importlib.import_module('pkg_init_2718.helper')
...     package = load_single_file(
...         'pkg_init_2718',
...         str(Path(alternate) / 'pkg_init_2718' / '__init__.py'))
...     answer = (package.VALUE,
...               sys.modules['pkg_init_2718.helper'] is not old_helper)
...     _clear_single_file_imports()
...     sys.path.remove(active)
...     for loaded in list(sys.modules):
...         if loaded == 'pkg_init_2718' or loaded.startswith(
...                 'pkg_init_2718.'):
...             del sys.modules[loaded]
...     sys.meta_path[:] = [finder for finder in sys.meta_path
...                         if not (isinstance(finder, _SiblingFinder)
...                                 and finder.package == 'pkg_init_2718')]
>>> answer
('alternate', True)

with tempfile.TemporaryDirectory() as active, \
     tempfile.TemporaryDirectory() as alternate:
    for root, value in ((active, 'active'),
                        (alternate, 'alternate')):
        package = Path(root) / 'pkg_init_2718'
        package.mkdir()
        _ = (package / '__init__.py').write_text(
            'from .helper import VALUE\n')
        _ = (package / 'helper.py').write_text(
            f"VALUE = {value!r}\n")
    sys.path.insert(0, active)
    old_package = importlib.import_module('pkg_init_2718')
    old_helper = importlib.import_module('pkg_init_2718.helper')
    package = load_single_file(
        'pkg_init_2718',
        str(Path(alternate) / 'pkg_init_2718' / '__init__.py'))
    answer = (package.VALUE,
              sys.modules['pkg_init_2718.helper'] is not old_helper)
    _clear_single_file_imports()
    sys.path.remove(active)
    for loaded in list(sys.modules):
        if loaded == 'pkg_init_2718' or loaded.startswith(
                'pkg_init_2718.'):
            del sys.modules[loaded]
    sys.meta_path[:] = [finder for finder in sys.meta_path
                        if not (isinstance(finder, _SiblingFinder)
                                and finder.package == 'pkg_init_2718')]
answer

If importing a previously absent containing package is part of a failed load, that package and its canonical child binding are rolled back too:

sage: with tempfile.TemporaryDirectory() as directory:
....:     package = Path(directory) / 'pkg_parent_2718'
....:     package.mkdir()
....:     _ = (package / '__init__.py').write_text(
....:         'from . import side\n')
....:     _ = (package / 'side.py').write_text('VALUE = 42\n')
....:     broken = package / 'broken.py'
....:     _ = broken.write_text("raise RuntimeError('broken')\n")
....:     sys.path.insert(0, directory)
....:     try:
....:         load_single_file('pkg_parent_2718.broken', str(broken))
....:     except RuntimeError:
....:         rolled_back = not any(
....:             loaded == 'pkg_parent_2718'
....:             or loaded.startswith('pkg_parent_2718.')
....:             for loaded in sys.modules)
....:     sys.path.remove(directory)
sage: rolled_back
True

>>> from sage.all import *
>>> with tempfile.TemporaryDirectory() as directory:
...     package = Path(directory) / 'pkg_parent_2718'
...     package.mkdir()
...     _ = (package / '__init__.py').write_text(
...         'from . import side\n')
...     _ = (package / 'side.py').write_text('VALUE = 42\n')
...     broken = package / 'broken.py'
...     _ = broken.write_text("raise RuntimeError('broken')\n")
...     sys.path.insert(Integer(0), directory)
...     try:
...         load_single_file('pkg_parent_2718.broken', str(broken))
...     except RuntimeError:
...         rolled_back = not any(
...             loaded == 'pkg_parent_2718'
...             or loaded.startswith('pkg_parent_2718.')
...             for loaded in sys.modules)
...     sys.path.remove(directory)
>>> rolled_back
True

with tempfile.TemporaryDirectory() as directory:
    package = Path(directory) / 'pkg_parent_2718'
    package.mkdir()
    _ = (package / '__init__.py').write_text(
        'from . import side\n')
    _ = (package / 'side.py').write_text('VALUE = 42\n')
    broken = package / 'broken.py'
    _ = broken.write_text("raise RuntimeError('broken')\n")
    sys.path.insert(0, directory)
    try:
        load_single_file('pkg_parent_2718.broken', str(broken))
    except RuntimeError:
        rolled_back = not any(
            loaded == 'pkg_parent_2718'
            or loaded.startswith('pkg_parent_2718.')
            for loaded in sys.modules)
    sys.path.remove(directory)
rolled_back