From b7c23d3626de272b94484901071efd860d0b4637 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Timoth=C3=A9e=20Mazzucotelli?= Date: Sun, 3 Sep 2023 15:21:30 +0200 Subject: [PATCH 01/16] docs: Update watch feature mention in README --- README.md | 5 ++--- mkdocs.yml | 1 + 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 06872728..da11f957 100644 --- a/README.md +++ b/README.md @@ -55,9 +55,8 @@ Come have a chat or ask questions on our [Gitter channel](https://gitter.im/mkdo each handler can be configured globally in `mkdocs.yml`, and locally for each "autodoc" instruction. -- [**Watch source code directories:**](https://mkdocstrings.github.io/usage/#watch-directories) - you can tell *mkdocstrings* to add directories to be watched by *MkDocs* when - serving the documentation, for auto-reload. +- ~~**Watch source code directories:**~~ + this feature was removed as it is now [built in MkDocs](https://www.mkdocs.org/user-guide/configuration/#watch). - **Reasonable defaults:** you should be able to just drop the plugin in your configuration and enjoy your auto-generated docs. diff --git a/mkdocs.yml b/mkdocs.yml index fdd6898b..a8e17f73 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -110,6 +110,7 @@ markdown_extensions: case: lower - pymdownx.tasklist: custom_checkbox: true +- pymdownx.tilde - toc: permalink: "¤" From 4df74ab58bc5ba1be4dadd46fc8e7f7c0345cab5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Timoth=C3=A9e=20Mazzucotelli?= Date: Mon, 4 Sep 2023 15:41:10 +0200 Subject: [PATCH 02/16] docs: Advertise shell handler --- README.md | 3 ++- docs/insiders/index.md | 3 ++- docs/usage/handlers.md | 1 + mkdocs.yml | 1 + 4 files changed, 6 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index da11f957..ae6d0f24 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,8 @@ Come have a chat or ask questions on our [Gitter channel](https://gitter.im/mkdo It means you can use it with any programming language, as long as there is a [**handler**](https://mkdocstrings.github.io/reference/handlers/base/) for it. We currently have [handlers](https://mkdocstrings.github.io/handlers/overview/) - for the [Crystal](https://mkdocstrings.github.io/crystal/) and [Python](https://mkdocstrings.github.io/python/) languages. + for the [Crystal](https://mkdocstrings.github.io/crystal/) and [Python](https://mkdocstrings.github.io/python/) languages, + as well as for [shell scripts/libraries](https://mkdocstrings.github.io/shell/). Maybe you'd like to add another one to the list? :wink: - [**Multiple themes support:**](https://mkdocstrings.github.io/theming/) diff --git a/docs/insiders/index.md b/docs/insiders/index.md index bfb2d428..eb8feea0 100644 --- a/docs/insiders/index.md +++ b/docs/insiders/index.md @@ -58,9 +58,10 @@ a handful of them, [thanks to our awesome sponsors][sponsors]! --> ```python exec="1" session="insiders" data_source = [ "docs/insiders/goals.yml", - ("mkdocstrings-python", "https://mkdocstrings.github.io/python/", "insiders/goals.yml"), ("griffe-pydantic", "https://mkdocstrings.github.io/griffe-pydantic/", "insiders/goals.yml"), ("griffe-typing-deprecated", "https://mkdocstrings.github.io/griffe-typing-deprecated/", "insiders/goals.yml"), + ("mkdocstrings-python", "https://mkdocstrings.github.io/python/", "insiders/goals.yml"), + ("mkdocstrings-shell", "https://mkdocstrings.github.io/shell/", "insiders/goals.yml"), ] ``` diff --git a/docs/usage/handlers.md b/docs/usage/handlers.md index d2c25420..bd7e5823 100644 --- a/docs/usage/handlers.md +++ b/docs/usage/handlers.md @@ -7,6 +7,7 @@ A handler is what makes it possible to collect and render documentation for a pa - Crystal - Python - Python (Legacy) +- Shell ## About the Python handlers diff --git a/mkdocs.yml b/mkdocs.yml index a8e17f73..d22db25f 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -27,6 +27,7 @@ nav: - Crystal: https://mkdocstrings.github.io/crystal/ - Python: https://mkdocstrings.github.io/python/ - Python (Legacy): https://mkdocstrings.github.io/python-legacy/ + - Shell: https://mkdocstrings.github.io/shell/ - Guides: - Recipes: recipes.md - Troubleshooting: troubleshooting.md From ea61f3dda3a0d0b88e410bdb0134fc570d4e6076 Mon Sep 17 00:00:00 2001 From: Nick Renieris Date: Thu, 7 Sep 2023 22:14:24 +0300 Subject: [PATCH 03/16] docs: Fix link to Python CSS customization (#606) --- docs/usage/theming.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/usage/theming.md b/docs/usage/theming.md index 73b7e0b3..083d3840 100644 --- a/docs/usage/theming.md +++ b/docs/usage/theming.md @@ -82,7 +82,7 @@ Since each handler provides its own set of templates, with their own CSS classes we cannot list them all here. See the documentation about CSS classes for: - the Crystal handler: https://mkdocstrings.github.io/crystal/styling.html#custom-styles -- the Python handler: https://mkdocstrings.github.io/python/customization/#css-classes +- the Python handler: https://mkdocstrings.github.io/python/usage/customization/#css-classes ### Syntax highlighting From 2e9d352066d4877ac30422183454824378a254a3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Timoth=C3=A9e=20Mazzucotelli?= Date: Thu, 14 Sep 2023 13:33:06 +0200 Subject: [PATCH 04/16] chore: Template upgrade --- .copier-answers.yml | 2 +- duties.py | 8 ++------ mkdocs.yml | 3 +++ scripts/insiders.py | 11 +++++++---- 4 files changed, 13 insertions(+), 11 deletions(-) diff --git a/.copier-answers.yml b/.copier-answers.yml index c720007f..9977da68 100644 --- a/.copier-answers.yml +++ b/.copier-answers.yml @@ -1,5 +1,5 @@ # Changes here will be overwritten by Copier -_commit: 0.16.6 +_commit: 0.16.9 _src_path: gh:pawamoy/copier-pdm.git author_email: pawamoy@pm.me author_fullname: Timothée Mazzucotelli diff --git a/duties.py b/duties.py index 644b2ffb..93311189 100644 --- a/duties.py +++ b/duties.py @@ -4,21 +4,17 @@ import os import sys +from importlib.metadata import version as pkgversion from pathlib import Path from typing import TYPE_CHECKING, Any from duty import duty from duty.callables import black, blacken_docs, coverage, lazy, mkdocs, mypy, pytest, ruff, safety -if sys.version_info < (3, 8): - from importlib_metadata import version as pkgversion -else: - from importlib.metadata import version as pkgversion - - if TYPE_CHECKING: from duty.context import Context + PY_SRC_PATHS = (Path(_) for _ in ("src", "tests", "duties.py", "scripts")) PY_SRC_LIST = tuple(str(_) for _ in PY_SRC_PATHS) PY_SRC = " ".join(PY_SRC_LIST) diff --git a/mkdocs.yml b/mkdocs.yml index d22db25f..c5d191f9 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -131,11 +131,14 @@ plugins: - https://docs.python.org/3/objects.inv - https://installer.readthedocs.io/en/stable/objects.inv # demonstration purpose in the docs - https://mkdocstrings.github.io/autorefs/objects.inv + paths: [src] options: docstring_options: ignore_init_summary: true docstring_section_style: list + filters: ["!^_"] heading_level: 1 + inherited_members: true merge_init_into_class: true separate_signature: true show_root_heading: true diff --git a/scripts/insiders.py b/scripts/insiders.py index 6f8d0d84..28ca1c87 100644 --- a/scripts/insiders.py +++ b/scripts/insiders.py @@ -39,11 +39,13 @@ class Feature: """Class representing an Insiders feature.""" name: str - ref: str + ref: str | None since: date | None project: Project | None - def url(self, rel_base: str = "..") -> str: # noqa: D102 + def url(self, rel_base: str = "..") -> str | None: # noqa: D102 + if not self.ref: + return None if self.project: rel_base = self.project.url return posixpath.join(rel_base, self.ref.lstrip("/")) @@ -56,7 +58,8 @@ def render(self, rel_base: str = "..", *, badge: bool = False) -> None: # noqa: ft_date = self.since.strftime("%B %d, %Y") # type: ignore[union-attr] new = f' :material-alert-decagram:{{ .new-feature .vibrate title="Added on {ft_date}" }}' project = f"[{self.project.name}]({self.project.url}) — " if self.project else "" - print(f"- [{'x' if self.since else ' '}] {project}[{self.name}]({self.url(rel_base)}){new}") + feature = f"[{self.name}]({self.url(rel_base)})" if self.ref else self.name + print(f"- [{'x' if self.since else ' '}] {project}{feature}{new}") @dataclass @@ -99,7 +102,7 @@ def load_goals(data: str, funding: int = 0, project: Project | None = None) -> d features=[ Feature( name=feature_data["name"], - ref=feature_data["ref"], + ref=feature_data.get("ref"), since=feature_data.get("since") and datetime.strptime(feature_data["since"], "%Y/%m/%d").date(), # noqa: DTZ007 project=project, From e2123a935edea0abdc1b439e2c2b76e002c76e2b Mon Sep 17 00:00:00 2001 From: Perceval Wajsburt Date: Mon, 18 Sep 2023 11:38:34 +0200 Subject: [PATCH 05/16] fix: Remove duplicated headings for docstrings nested in tabs/admonitions Issue #609: https://github.com/mkdocstrings/mkdocstrings/issues/609 PR #610: https://github.com/mkdocstrings/mkdocstrings/pull/610 --- src/mkdocstrings/extension.py | 15 ++++++++++++--- tests/test_extension.py | 18 ++++++++++++++++++ 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/src/mkdocstrings/extension.py b/src/mkdocstrings/extension.py index 139030ba..f7634a28 100644 --- a/src/mkdocstrings/extension.py +++ b/src/mkdocstrings/extension.py @@ -231,17 +231,26 @@ def _process_block( class _PostProcessor(Treeprocessor): def run(self, root: Element) -> None: + self._remove_duplicated_headings(root) + + def _remove_duplicated_headings(self, parent: Element) -> bool: carry_text = "" - for el in reversed(root): # Reversed mainly for the ability to mutate during iteration. + found = False + for el in reversed(parent): # Reversed mainly for the ability to mutate during iteration. if el.tag == "div" and el.get("class") == "mkdocstrings": # Delete the duplicated headings along with their container, but keep the text (i.e. the actual HTML). carry_text = (el.text or "") + carry_text - root.remove(el) + parent.remove(el) + found = True elif carry_text: el.tail = (el.tail or "") + carry_text carry_text = "" + elif self._remove_duplicated_headings(el): + found = True + break if carry_text: - root.text = (root.text or "") + carry_text + parent.text = (parent.text or "") + carry_text + return found class MkdocstringsExtension(Extension): diff --git a/tests/test_extension.py b/tests/test_extension.py index 8c687629..2d50ef4d 100644 --- a/tests/test_extension.py +++ b/tests/test_extension.py @@ -150,3 +150,21 @@ def test_use_options_yaml_key(ext_markdown: Markdown) -> None: """Check that using the 'options' YAML key works as expected.""" assert "h1" in ext_markdown.convert("::: tests.fixtures.headings\n options:\n heading_level: 1") assert "h1" not in ext_markdown.convert("::: tests.fixtures.headings\n options:\n heading_level: 2") + + +@pytest.mark.parametrize("ext_markdown", [{"markdown_extensions": [{"pymdownx.tabbed": {"alternate_style": True}}]}], indirect=["ext_markdown"]) +def test_removing_duplicated_headings(ext_markdown: Markdown) -> None: + """Assert duplicated headings are removed from the output.""" + output = ext_markdown.convert( + dedent( + """ + === "Tab A" + + ::: tests.fixtures.headings + + """, + ), + ) + assert output.count("Foo") == 1 + assert output.count("Bar") == 1 + assert output.count("Baz") == 1 From f4a94f7d8b8eb1ac01d65bb7237f0077e320ddac Mon Sep 17 00:00:00 2001 From: Oleh Prypin Date: Mon, 18 Sep 2023 22:17:18 +0200 Subject: [PATCH 06/16] fix: Properly fix duplicated headings for nested docstrings PR #613: https://github.com/mkdocstrings/mkdocstrings/pull/613 --- src/mkdocstrings/extension.py | 17 +++++++---------- tests/fixtures/headings_many.py | 10 ++++++++++ tests/test_extension.py | 16 ++++++++++------ 3 files changed, 27 insertions(+), 16 deletions(-) create mode 100644 tests/fixtures/headings_many.py diff --git a/src/mkdocstrings/extension.py b/src/mkdocstrings/extension.py index f7634a28..8e83c62e 100644 --- a/src/mkdocstrings/extension.py +++ b/src/mkdocstrings/extension.py @@ -233,24 +233,21 @@ class _PostProcessor(Treeprocessor): def run(self, root: Element) -> None: self._remove_duplicated_headings(root) - def _remove_duplicated_headings(self, parent: Element) -> bool: + def _remove_duplicated_headings(self, parent: Element) -> None: carry_text = "" - found = False for el in reversed(parent): # Reversed mainly for the ability to mutate during iteration. if el.tag == "div" and el.get("class") == "mkdocstrings": # Delete the duplicated headings along with their container, but keep the text (i.e. the actual HTML). carry_text = (el.text or "") + carry_text parent.remove(el) - found = True - elif carry_text: - el.tail = (el.tail or "") + carry_text - carry_text = "" - elif self._remove_duplicated_headings(el): - found = True - break + else: + if carry_text: + el.tail = (el.tail or "") + carry_text + carry_text = "" + self._remove_duplicated_headings(el) + if carry_text: parent.text = (parent.text or "") + carry_text - return found class MkdocstringsExtension(Extension): diff --git a/tests/fixtures/headings_many.py b/tests/fixtures/headings_many.py new file mode 100644 index 00000000..fa643a48 --- /dev/null +++ b/tests/fixtures/headings_many.py @@ -0,0 +1,10 @@ +def heading_1(): + """## Heading one""" + + +def heading_2(): + """### Heading two""" + + +def heading_3(): + """#### Heading three""" diff --git a/tests/test_extension.py b/tests/test_extension.py index 2d50ef4d..6011be7f 100644 --- a/tests/test_extension.py +++ b/tests/test_extension.py @@ -152,19 +152,23 @@ def test_use_options_yaml_key(ext_markdown: Markdown) -> None: assert "h1" not in ext_markdown.convert("::: tests.fixtures.headings\n options:\n heading_level: 2") -@pytest.mark.parametrize("ext_markdown", [{"markdown_extensions": [{"pymdownx.tabbed": {"alternate_style": True}}]}], indirect=["ext_markdown"]) +@pytest.mark.parametrize("ext_markdown", [{"markdown_extensions": [{"admonition": {}}]}], indirect=["ext_markdown"]) def test_removing_duplicated_headings(ext_markdown: Markdown) -> None: """Assert duplicated headings are removed from the output.""" output = ext_markdown.convert( dedent( """ - === "Tab A" + ::: tests.fixtures.headings_many.heading_1 - ::: tests.fixtures.headings + !!! note + ::: tests.fixtures.headings_many.heading_2 + + ::: tests.fixtures.headings_many.heading_3 """, ), ) - assert output.count("Foo") == 1 - assert output.count("Bar") == 1 - assert output.count("Baz") == 1 + assert output.count(">Heading one<") == 1 + assert output.count(">Heading two<") == 1 + assert output.count(">Heading three<") == 1 + assert output.count('class="mkdocstrings') == 0 From 370a61d12b33f3fb61f6bddb3939eb8ff6018620 Mon Sep 17 00:00:00 2001 From: Waylan Limberg Date: Thu, 26 Oct 2023 03:06:17 -0400 Subject: [PATCH 07/16] fix: Make `custom_templates` relative to the config file Issue #477: https://github.com/mkdocstrings/mkdocstrings/issues/477 PR #627: https://github.com/mkdocstrings/mkdocstrings/pull/627 --- docs/usage/index.md | 2 +- docs/usage/theming.md | 6 +++--- src/mkdocstrings/plugin.py | 6 ++++-- tests/test_plugin.py | 36 ++++++++++++++++++++++++++++++++++++ 4 files changed, 44 insertions(+), 6 deletions(-) diff --git a/docs/usage/index.md b/docs/usage/index.md index 7599f9f1..1348b9cc 100644 --- a/docs/usage/index.md +++ b/docs/usage/index.md @@ -108,7 +108,7 @@ The above is equivalent to: - `default_handler`: The handler that is used by default when no handler is specified. - `custom_templates`: The path to a directory containing custom templates. - The path is relative to the current working directory. + The path is relative to the MkDocs configuration file. See [Theming](theming.md). - `handlers`: The handlers' global configuration. - `enable_inventory`: Whether to enable inventory file generation. diff --git a/docs/usage/theming.md b/docs/usage/theming.md index 083d3840..b5d6f7b3 100644 --- a/docs/usage/theming.md +++ b/docs/usage/theming.md @@ -17,9 +17,9 @@ so you can tweak the look and feel with extra CSS rules. ### Templates -To use custom templates and override the theme ones, -specify the relative path to your templates directory -with the `custom_templates` global configuration option: +To use custom templates and override the theme ones, specify the relative path from your +configuration file to your templates directory with the `custom_templates` global +configuration option: ```yaml title="mkdocs.yml" plugins: diff --git a/src/mkdocstrings/plugin.py b/src/mkdocstrings/plugin.py index 484d3ead..682310cc 100644 --- a/src/mkdocstrings/plugin.py +++ b/src/mkdocstrings/plugin.py @@ -23,6 +23,7 @@ from urllib import request from mkdocs.config.config_options import Type as MkType +from mkdocs.config.config_options import Dir, Optional from mkdocs.plugins import BasePlugin from mkdocs.utils import write_file from mkdocs_autorefs.plugin import AutorefsPlugin @@ -78,7 +79,7 @@ class MkdocstringsPlugin(BasePlugin): config_scheme: tuple[tuple[str, MkType]] = ( # type: ignore[assignment] ("handlers", MkType(dict, default={})), ("default_handler", MkType(str, default="python")), - ("custom_templates", MkType(str, default=None)), + ("custom_templates", Optional(Dir(exists=True))), ("enable_inventory", MkType(bool, default=None)), ("enabled", MkType(bool, default=True)), ) @@ -90,7 +91,8 @@ class MkdocstringsPlugin(BasePlugin): - **`handlers`**: Global configuration of handlers. You can set global configuration per handler, applied everywhere, but overridable in each "autodoc" instruction. Example: - **`default_handler`**: The default handler to use. The value is the name of the handler module. Default is "python". - - **`custom_templates`**: Custom templates to use when rendering API objects. + - **`custom_templates`**: Location of custom templates to use when rendering API objects. Value should be the path of + a directory relative to the MkDocs configuration file. - **`enable_inventory`**: Whether to enable object inventory creation. - **`enabled`**: Whether to enable the plugin. Default is true. If false, *mkdocstrings* will not collect or render anything. diff --git a/tests/test_plugin.py b/tests/test_plugin.py index b8e8d2a5..26c4031a 100644 --- a/tests/test_plugin.py +++ b/tests/test_plugin.py @@ -6,6 +6,7 @@ from mkdocs.commands.build import build from mkdocs.config import load_config +from mkdocstrings.plugin import MkdocstringsPlugin if TYPE_CHECKING: from pathlib import Path @@ -31,3 +32,38 @@ def test_disabling_plugin(tmp_path: Path) -> None: # make sure the instruction was not processed assert "::: mkdocstrings" in site_dir.joinpath("index.html").read_text() + + +def test_plugin_default_config(tmp_path: Path) -> None: + """Test default config options are set for Plugin.""" + config_file_path = tmp_path / "mkdocs.yml" + plugin = MkdocstringsPlugin() + errors, warnings = plugin.load_config({}, config_file_path=str(config_file_path)) + assert errors == [] + assert warnings == [] + assert plugin.config == { + "handlers": {}, + "default_handler": "python", + "custom_templates": None, + "enable_inventory": None, + "enabled": True, + } + +def test_plugin_config_custom_templates(tmp_path: Path) -> None: + """Test custom_templates option is relative to config file.""" + config_file_path = tmp_path / "mkdocs.yml" + options = {"custom_templates": "docs/templates"} + template_dir = tmp_path / options["custom_templates"] + # Path must exist or config validation will fail. + template_dir.mkdir(parents=True) + plugin = MkdocstringsPlugin() + errors, warnings = plugin.load_config(options, config_file_path=str(config_file_path)) + assert errors == [] + assert warnings == [] + assert plugin.config == { + "handlers": {}, + "default_handler": "python", + "custom_templates": str(template_dir), + "enable_inventory": None, + "enabled": True, + } From b61d4d15258c66b14266aa04b456f191f101b2c6 Mon Sep 17 00:00:00 2001 From: Oleh Prypin Date: Fri, 27 Oct 2023 00:36:02 +0200 Subject: [PATCH 08/16] refactor: Drop support for MkDocs < 1.4, modernize usages PR #629: https://github.com/mkdocstrings/mkdocstrings/pull/629 --- pyproject.toml | 2 +- src/mkdocstrings/extension.py | 3 +- src/mkdocstrings/plugin.py | 96 ++++++++++++++++++----------------- tests/conftest.py | 6 +-- 4 files changed, 54 insertions(+), 53 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index 78af6bff..7077ccf8 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -32,7 +32,7 @@ dependencies = [ "Jinja2>=2.11.1", "Markdown>=3.3", "MarkupSafe>=1.1", - "mkdocs>=1.2", + "mkdocs>=1.4", "mkdocs-autorefs>=0.3.1", "pymdown-extensions>=6.3", "importlib-metadata>=4.6; python_version < '3.10'", diff --git a/src/mkdocstrings/extension.py b/src/mkdocstrings/extension.py index 8e83c62e..a819f14b 100644 --- a/src/mkdocstrings/extension.py +++ b/src/mkdocstrings/extension.py @@ -72,8 +72,7 @@ def __init__( Arguments: parser: A `markdown.blockparser.BlockParser` instance. md: A `markdown.Markdown` instance. - config: The [configuration][mkdocstrings.plugin.MkdocstringsPlugin.config_scheme] - of the `mkdocstrings` plugin. + config: The [configuration][mkdocstrings.plugin.PluginConfig] of the `mkdocstrings` plugin. handlers: The handlers container. autorefs: The autorefs plugin instance. """ diff --git a/src/mkdocstrings/plugin.py b/src/mkdocstrings/plugin.py index 682310cc..2415cf45 100644 --- a/src/mkdocstrings/plugin.py +++ b/src/mkdocstrings/plugin.py @@ -22,8 +22,8 @@ from typing import TYPE_CHECKING, Any, BinaryIO, Callable, Iterable, List, Mapping, Tuple, TypeVar from urllib import request -from mkdocs.config.config_options import Type as MkType -from mkdocs.config.config_options import Dir, Optional +from mkdocs.config import Config +from mkdocs.config import config_options as opt from mkdocs.plugins import BasePlugin from mkdocs.utils import write_file from mkdocs_autorefs.plugin import AutorefsPlugin @@ -34,7 +34,6 @@ if TYPE_CHECKING: from jinja2.environment import Environment - from mkdocs.config import Config from mkdocs.config.defaults import MkDocsConfig if sys.version_info < (3, 10): @@ -63,38 +62,15 @@ def wrapper(*args: P.args, **kwargs: P.kwargs) -> R: return wrapper -class MkdocstringsPlugin(BasePlugin): - """An `mkdocs` plugin. - - This plugin defines the following event hooks: - - - `on_config` - - `on_env` - - `on_post_build` +class PluginConfig(Config): + """The configuration options of `mkdocstrings`, written in `mkdocs.yml`.""" - Check the [Developing Plugins](https://www.mkdocs.org/user-guide/plugins/#developing-plugins) page of `mkdocs` - for more information about its plugin system. - """ - - config_scheme: tuple[tuple[str, MkType]] = ( # type: ignore[assignment] - ("handlers", MkType(dict, default={})), - ("default_handler", MkType(str, default="python")), - ("custom_templates", Optional(Dir(exists=True))), - ("enable_inventory", MkType(bool, default=None)), - ("enabled", MkType(bool, default=True)), - ) + handlers = opt.Type(dict, default={}) """ - The configuration options of `mkdocstrings`, written in `mkdocs.yml`. + Global configuration of handlers. - Available options are: - - - **`handlers`**: Global configuration of handlers. You can set global configuration per handler, applied everywhere, - but overridable in each "autodoc" instruction. Example: - - **`default_handler`**: The default handler to use. The value is the name of the handler module. Default is "python". - - **`custom_templates`**: Location of custom templates to use when rendering API objects. Value should be the path of - a directory relative to the MkDocs configuration file. - - **`enable_inventory`**: Whether to enable object inventory creation. - - **`enabled`**: Whether to enable the plugin. Default is true. If false, *mkdocstrings* will not collect or render anything. + You can set global configuration per handler, applied everywhere, + but overridable in each "autodoc" instruction. Example: ```yaml plugins: @@ -110,6 +86,32 @@ class MkdocstringsPlugin(BasePlugin): ``` """ + default_handler = opt.Type(str, default="python") + """The default handler to use. The value is the name of the handler module. Default is "python".""" + custom_templates = opt.Optional(opt.Dir(exists=True)), + """Location of custom templates to use when rendering API objects. + + Value should be the path of a directory relative to the MkDocs configuration file. + """ + enable_inventory = opt.Optional(opt.Type(bool)) + """Whether to enable object inventory creation.""" + enabled = opt.Type(bool, default=True) + """Whether to enable the plugin. Default is true. If false, *mkdocstrings* will not collect or render anything.""" + + +class MkdocstringsPlugin(BasePlugin[PluginConfig]): + """An `mkdocs` plugin. + + This plugin defines the following event hooks: + + - `on_config` + - `on_env` + - `on_post_build` + + Check the [Developing Plugins](https://www.mkdocs.org/user-guide/plugins/#developing-plugins) page of `mkdocs` + for more information about its plugin system. + """ + css_filename = "assets/_mkdocstrings.css" def __init__(self) -> None: @@ -152,10 +154,10 @@ def on_config(self, config: MkDocsConfig) -> MkDocsConfig | None: return config log.debug("Adding extension to the list") - theme_name = config["theme"].name or os.path.dirname(config["theme"].dirs[0]) + theme_name = config.theme.name or os.path.dirname(config.theme.dirs[0]) to_import: InventoryImportType = [] - for handler_name, conf in self.config["handlers"].items(): + for handler_name, conf in self.config.handlers.items(): for import_item in conf.pop("import", ()): if isinstance(import_item, str): import_item = {"url": import_item} # noqa: PLW2901 @@ -163,8 +165,8 @@ def on_config(self, config: MkDocsConfig) -> MkDocsConfig | None: extension_config = { "theme_name": theme_name, - "mdx": config["markdown_extensions"], - "mdx_configs": config["mdx_configs"], + "mdx": config.markdown_extensions, + "mdx_configs": config.mdx_configs, "mkdocstrings": self.config, "mkdocs": config, } @@ -172,21 +174,21 @@ def on_config(self, config: MkDocsConfig) -> MkDocsConfig | None: try: # If autorefs plugin is explicitly enabled, just use it. - autorefs = config["plugins"]["autorefs"] + autorefs = config.plugins["autorefs"] log.debug(f"Picked up existing autorefs instance {autorefs!r}") except KeyError: # Otherwise, add a limited instance of it that acts only on what's added through `register_anchor`. autorefs = AutorefsPlugin() autorefs.scan_toc = False - config["plugins"]["autorefs"] = autorefs + config.plugins["autorefs"] = autorefs log.debug(f"Added a subdued autorefs instance {autorefs!r}") # Add collector-based fallback in either case. autorefs.get_fallback_anchor = self.handlers.get_anchors mkdocstrings_extension = MkdocstringsExtension(extension_config, self.handlers, autorefs) - config["markdown_extensions"].append(mkdocstrings_extension) + config.markdown_extensions.append(mkdocstrings_extension) - config["extra_css"].insert(0, self.css_filename) # So that it has lower priority than user files. + config.extra_css.insert(0, self.css_filename) # So that it has lower priority than user files. self._inv_futures = {} if to_import: @@ -210,7 +212,7 @@ def inventory_enabled(self) -> bool: Returns: Whether the inventory is enabled. """ - inventory_enabled = self.config["enable_inventory"] + inventory_enabled = self.config.enable_inventory if inventory_enabled is None: inventory_enabled = any(handler.enable_inventory for handler in self.handlers.seen_handlers) return inventory_enabled @@ -222,9 +224,9 @@ def plugin_enabled(self) -> bool: Returns: Whether the plugin is enabled. """ - return self.config["enabled"] + return self.config.enabled - def on_env(self, env: Environment, config: Config, *args: Any, **kwargs: Any) -> None: # noqa: ARG002 + def on_env(self, env: Environment, config: MkDocsConfig, *args: Any, **kwargs: Any) -> None: # noqa: ARG002 """Extra actions that need to happen after all Markdown rendering and before HTML rendering. Hook for the [`on_env` event](https://www.mkdocs.org/user-guide/plugins/#on_env). @@ -236,12 +238,12 @@ def on_env(self, env: Environment, config: Config, *args: Any, **kwargs: Any) -> return if self._handlers: css_content = "\n".join(handler.extra_css for handler in self.handlers.seen_handlers) - write_file(css_content.encode("utf-8"), os.path.join(config["site_dir"], self.css_filename)) + write_file(css_content.encode("utf-8"), os.path.join(config.site_dir, self.css_filename)) if self.inventory_enabled: log.debug("Creating inventory file objects.inv") inv_contents = self.handlers.inventory.format_sphinx() - write_file(inv_contents, os.path.join(config["site_dir"], "objects.inv")) + write_file(inv_contents, os.path.join(config.site_dir, "objects.inv")) if self._inv_futures: log.debug(f"Waiting for {len(self._inv_futures)} inventory download(s)") @@ -256,12 +258,12 @@ def on_env(self, env: Environment, config: Config, *args: Any, **kwargs: Any) -> loader_name = loader.__func__.__qualname__ log.error(f"Couldn't load inventory {import_item} through {loader_name}: {error}") # noqa: TRY400 for page, identifier in results.items(): - config["plugins"]["autorefs"].register_url(page, identifier) + config.plugins["autorefs"].register_url(page, identifier) self._inv_futures = {} def on_post_build( self, - config: Config, # noqa: ARG002 + config: MkDocsConfig, # noqa: ARG002 **kwargs: Any, # noqa: ARG002 ) -> None: """Teardown the handlers. diff --git a/tests/conftest.py b/tests/conftest.py index 2119d1f3..59bac65a 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -8,7 +8,7 @@ import pytest from markdown.core import Markdown from mkdocs import config -from mkdocs.config.defaults import get_schema +from mkdocs.config.defaults import MkDocsConfig if TYPE_CHECKING: from pathlib import Path @@ -19,7 +19,7 @@ @pytest.fixture(name="mkdocs_conf") def fixture_mkdocs_conf(request: pytest.FixtureRequest, tmp_path: Path) -> Iterator[config.Config]: """Yield a MkDocs configuration object.""" - conf = config.Config(schema=get_schema()) # type: ignore[call-arg] + conf = MkDocsConfig() while hasattr(request, "_parent_request") and hasattr(request._parent_request, "_parent_request"): request = request._parent_request @@ -53,6 +53,6 @@ def fixture_plugin(mkdocs_conf: config.Config) -> MkdocstringsPlugin: @pytest.fixture(name="ext_markdown") -def fixture_ext_markdown(mkdocs_conf: config.Config) -> Markdown: +def fixture_ext_markdown(mkdocs_conf: MkDocsConfig) -> Markdown: """Return a Markdown instance with MkdocstringsExtension.""" return Markdown(extensions=mkdocs_conf["markdown_extensions"], extension_configs=mkdocs_conf["mdx_configs"]) From afc4ea4e178d27c755528f22adb7c1a6fce736f2 Mon Sep 17 00:00:00 2001 From: Oleh Prypin Date: Sat, 28 Oct 2023 18:12:11 +0200 Subject: [PATCH 09/16] fix: `custom_templates` config was dropped in previous commit (#630) --- src/mkdocstrings/plugin.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/mkdocstrings/plugin.py b/src/mkdocstrings/plugin.py index 2415cf45..3f8ce8cd 100644 --- a/src/mkdocstrings/plugin.py +++ b/src/mkdocstrings/plugin.py @@ -88,9 +88,9 @@ class PluginConfig(Config): default_handler = opt.Type(str, default="python") """The default handler to use. The value is the name of the handler module. Default is "python".""" - custom_templates = opt.Optional(opt.Dir(exists=True)), + custom_templates = opt.Optional(opt.Dir(exists=True)) """Location of custom templates to use when rendering API objects. - + Value should be the path of a directory relative to the MkDocs configuration file. """ enable_inventory = opt.Optional(opt.Type(bool)) From 39694acbdcbb22bbd84b02c05870a8ac816a416e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Timoth=C3=A9e=20Mazzucotelli?= Date: Sun, 12 Nov 2023 01:22:13 +0100 Subject: [PATCH 10/16] chore: Template upgrade --- .copier-answers.yml | 4 +- .github/ISSUE_TEMPLATE/bug_report.md | 75 +++++++---- .github/ISSUE_TEMPLATE/config.yml | 5 + .github/ISSUE_TEMPLATE/feature_request.md | 23 ++-- CONTRIBUTING.md | 5 +- Makefile | 3 +- README.md | 4 +- config/git-changelog.toml | 8 ++ config/ruff.toml | 3 + config/vscode/launch.json | 36 ++++++ config/vscode/settings.json | 52 ++++++++ config/vscode/tasks.json | 93 ++++++++++++++ docs/credits.md | 2 + docs/css/insiders.css | 5 +- docs/insiders/index.md | 14 +-- duties.py | 146 +++++++++++----------- mkdocs.insiders.yml | 5 - mkdocs.yml | 13 +- pyproject.toml | 49 ++++---- scripts/gen_credits.py | 20 +-- scripts/gen_ref_nav.py | 10 +- scripts/insiders.py | 6 +- src/mkdocstrings/debug.py | 106 ++++++++++++++++ tests/test_plugin.py | 2 + 24 files changed, 514 insertions(+), 175 deletions(-) create mode 100644 .github/ISSUE_TEMPLATE/config.yml create mode 100644 config/git-changelog.toml create mode 100644 config/vscode/launch.json create mode 100644 config/vscode/settings.json create mode 100644 config/vscode/tasks.json delete mode 100644 mkdocs.insiders.yml create mode 100644 src/mkdocstrings/debug.py diff --git a/.copier-answers.yml b/.copier-answers.yml index 9977da68..0c51afe2 100644 --- a/.copier-answers.yml +++ b/.copier-answers.yml @@ -1,5 +1,5 @@ # Changes here will be overwritten by Copier -_commit: 0.16.9 +_commit: 1.1.3 _src_path: gh:pawamoy/copier-pdm.git author_email: pawamoy@pm.me author_fullname: Timothée Mazzucotelli @@ -9,8 +9,10 @@ copyright_holder: Timothée Mazzucotelli copyright_holder_email: pawamoy@pm.me copyright_license: ISC License insiders: true +insiders_repository_name: mkdocstrings project_description: Automatic documentation from sources, for MkDocs. project_name: mkdocstrings +public_release: true python_package_command_line_name: '' python_package_distribution_name: mkdocstrings python_package_import_name: mkdocstrings diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 149c6ce0..6ed84b16 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -1,36 +1,61 @@ --- name: Bug report -about: Create a report to help us improve -title: '' +about: Create a bug report to help us improve. +title: "bug: " labels: unconfirmed -assignees: '' - +assignees: [pawamoy] --- -**Please open an issue on [Griffe](https://github.com/mkdocstrings/griffe/issues) (new Python handler) -or [pytkdocs](https://github.com/mkdocstrings/pytkdocs/issues) (legacy Python handler) instead -if this is related to Python docstrings parsing or the collection of Python objects!** +### Description of the bug + + +### To Reproduce + + +``` +WRITE MRE / INSTRUCTIONS HERE +``` + +### Full traceback + + +
Full traceback + +```python +PASTE TRACEBACK HERE +``` -**Describe the bug** -A clear and concise description of what the bug is. +
-**To Reproduce** -Steps to reproduce the behavior: -1. Go to '...' -2. Click on '....' -3. Scroll down to '....' -4. See error +### Expected behavior + -**Expected behavior** -A clear and concise description of what you expected to happen. +### Environment information + -**Screenshots** -If applicable, add screenshots to help explain your problem. +```bash +python -m mkdocstrings.debug # | xclip -selection clipboard +``` -**Information (please complete the following information):** -- OS: [e.g. iOS] -- Browser: [e.g. chrome, safari] -- `mkdocstrings` version: [e.g. 0.10.2] +PASTE OUTPUT HERE -**Additional context** -Add any other context about the problem here. +### Additional context + diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 00000000..23000298 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,5 @@ +blank_issues_enabled: false +contact_links: +- name: I have a question / I need help + url: https://github.com/mkdocstrings/mkdocstrings/discussions/new?category=q-a + about: Ask and answer questions in the Discussions tab. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index 4fe86d5e..2df98fbc 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -1,20 +1,19 @@ --- name: Feature request -about: Suggest an idea for this project -title: '' +about: Suggest an idea for this project. +title: "feature: " labels: feature -assignees: '' - +assignees: pawamoy --- -**Is your feature request related to a problem? Please describe.** -A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] +### Is your feature request related to a problem? Please describe. + -**Describe the solution you'd like** -A clear and concise description of what you want to happen. +### Describe the solution you'd like + -**Describe alternatives you've considered** -A clear and concise description of any alternative solutions or features you've considered. +### Describe alternatives you've considered + -**Additional context** -Add any other context or screenshots about the feature request here. +### Additional context + diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0b86ff4b..ff84c305 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -44,8 +44,9 @@ on multiple Python versions, you run the task directly with `pdm run duty TASK`. The Makefile detects if a virtual environment is activated, so `make` will work the same with the virtualenv activated or not. -If you work in VSCode, -[see examples of tasks and run configurations](https://pawamoy.github.io/copier-pdm/work/#vscode-setup). +If you work in VSCode, we provide +[an action to configure VSCode](https://pawamoy.github.io/copier-pdm/work/#vscode-setup) +for the project. ## Development diff --git a/Makefile b/Makefile index 5696baac..f441a5c5 100644 --- a/Makefile +++ b/Makefile @@ -18,7 +18,8 @@ BASIC_DUTIES = \ docs \ docs-deploy \ format \ - release + release \ + vscode QUALITY_DUTIES = \ check-quality \ diff --git a/README.md b/README.md index ae6d0f24..15d41d7a 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ [![pypi version](https://img.shields.io/pypi/v/mkdocstrings.svg)](https://pypi.org/project/mkdocstrings/) [![conda version](https://img.shields.io/conda/vn/conda-forge/mkdocstrings)](https://anaconda.org/conda-forge/mkdocstrings) [![gitpod](https://img.shields.io/badge/gitpod-workspace-blue.svg?style=flat)](https://gitpod.io/#https://github.com/mkdocstrings/mkdocstrings) -[![gitter](https://badges.gitter.im/join%20chat.svg)](https://gitter.im/mkdocstrings/community) +[![gitter](https://badges.gitter.im/join%20chat.svg)](https://app.gitter.im/#/room/#mkdocstrings:gitter.im) Automatic documentation from sources, for [MkDocs](https://mkdocs.org/). Come have a chat or ask questions on our [Gitter channel](https://gitter.im/mkdocstrings/community). @@ -77,6 +77,7 @@ Come have a chat or ask questions on our [Gitter channel](https://gitter.im/mkdo ## Installation With `pip`: + ```bash pip install mkdocstrings ``` @@ -90,6 +91,7 @@ pip install 'mkdocstrings[crystal,python]' See the [available language handlers](https://mkdocstrings.github.io/handlers/overview/). With `conda`: + ```bash conda install -c conda-forge mkdocstrings ``` diff --git a/config/git-changelog.toml b/config/git-changelog.toml new file mode 100644 index 00000000..44e2b1fb --- /dev/null +++ b/config/git-changelog.toml @@ -0,0 +1,8 @@ +bump = "auto" +convention = "angular" +in-place = true +output = "CHANGELOG.md" +parse-refs = false +parse-trailers = true +sections = ["build", "deps", "feat", "fix", "refactor"] +template = "keepachangelog" diff --git a/config/ruff.toml b/config/ruff.toml index c6e4a55c..ad45b6c9 100644 --- a/config/ruff.toml +++ b/config/ruff.toml @@ -77,6 +77,9 @@ ignore = [ "src/*/cli.py" = [ "T201", # Print statement ] +"src/*/debug.py" = [ + "T201", # Print statement +] "scripts/*.py" = [ "INP001", # File is part of an implicit namespace package "T201", # Print statement diff --git a/config/vscode/launch.json b/config/vscode/launch.json new file mode 100644 index 00000000..2e0d651e --- /dev/null +++ b/config/vscode/launch.json @@ -0,0 +1,36 @@ +{ + "version": "0.2.0", + "configurations": [ + { + "name": "python (current file)", + "type": "python", + "request": "launch", + "program": "${file}", + "console": "integratedTerminal", + "justMyCode": false + }, + { + "name": "test", + "type": "python", + "request": "launch", + "module": "pytest", + "justMyCode": false, + "args": [ + "-c=config/pytest.ini", + "-vvv", + "--no-cov", + "--dist=no", + "tests", + "-k=${input:tests_selection}" + ] + } + ], + "inputs": [ + { + "id": "tests_selection", + "type": "promptString", + "description": "Tests selection", + "default": "" + } + ] +} \ No newline at end of file diff --git a/config/vscode/settings.json b/config/vscode/settings.json new file mode 100644 index 00000000..17beee4b --- /dev/null +++ b/config/vscode/settings.json @@ -0,0 +1,52 @@ +{ + "files.watcherExclude": { + "**/__pypackages__/**": true, + "**/.venv*/**": true, + "**/venv*/**": true + }, + "[python]": { + "editor.defaultFormatter": "ms-python.black-formatter" + }, + "python.autoComplete.extraPaths": [ + "__pypackages__/3.8/lib", + "__pypackages__/3.9/lib", + "__pypackages__/3.10/lib", + "__pypackages__/3.11/lib", + "__pypackages__/3.12/lib" + ], + "python.analysis.extraPaths": [ + "__pypackages__/3.8/lib", + "__pypackages__/3.9/lib", + "__pypackages__/3.10/lib", + "__pypackages__/3.11/lib", + "__pypackages__/3.12/lib" + ], + "black-formatter.args": [ + "--config=config/black.toml" + ], + "mypy-type-checker.args": [ + "--config-file=config/mypy.ini" + ], + "python.testing.unittestEnabled": false, + "python.testing.pytestEnabled": true, + "python.testing.pytestArgs": [ + "--config-file=config/pytest.ini" + ], + "ruff.format.args": [ + "--config=config/ruff.toml" + ], + "ruff.lint.args": [ + "--config=config/ruff.toml" + ], + "yaml.schemas": { + "https://squidfunk.github.io/mkdocs-material/schema.json": "mkdocs.yml" + }, + "yaml.customTags": [ + "!ENV scalar", + "!ENV sequence", + "!relative scalar", + "tag:yaml.org,2002:python/name:materialx.emoji.to_svg", + "tag:yaml.org,2002:python/name:materialx.emoji.twemoji", + "tag:yaml.org,2002:python/name:pymdownx.superfences.fence_code_format" + ] +} \ No newline at end of file diff --git a/config/vscode/tasks.json b/config/vscode/tasks.json new file mode 100644 index 00000000..80cd13d2 --- /dev/null +++ b/config/vscode/tasks.json @@ -0,0 +1,93 @@ +{ + "version": "2.0.0", + "tasks": [ + { + "label": "changelog", + "type": "shell", + "command": "pdm run duty changelog" + }, + { + "label": "check", + "type": "shell", + "command": "pdm run duty check" + }, + { + "label": "check-quality", + "type": "shell", + "command": "pdm run duty check-quality" + }, + { + "label": "check-types", + "type": "shell", + "command": "pdm run duty check-types" + }, + { + "label": "check-docs", + "type": "shell", + "command": "pdm run duty check-docs" + }, + { + "label": "check-dependencies", + "type": "shell", + "command": "pdm run duty check-dependencies" + }, + { + "label": "check-api", + "type": "shell", + "command": "pdm run duty check-api" + }, + { + "label": "clean", + "type": "shell", + "command": "pdm run duty clean" + }, + { + "label": "docs", + "type": "shell", + "command": "pdm run duty docs" + }, + { + "label": "docs-deploy", + "type": "shell", + "command": "pdm run duty docs-deploy" + }, + { + "label": "format", + "type": "shell", + "command": "pdm run duty format" + }, + { + "label": "lock", + "type": "shell", + "command": "pdm lock -G:all" + }, + { + "label": "release", + "type": "shell", + "command": "pdm run duty release ${input:version}" + }, + { + "label": "setup", + "type": "shell", + "command": "bash scripts/setup.sh" + }, + { + "label": "test", + "type": "shell", + "command": "pdm run duty test coverage", + "group": "test" + }, + { + "label": "vscode", + "type": "shell", + "command": "pdm run duty vscode" + } + ], + "inputs": [ + { + "id": "version", + "type": "promptString", + "description": "Version" + } + ] +} \ No newline at end of file diff --git a/docs/credits.md b/docs/credits.md index 9db45873..f758db87 100644 --- a/docs/credits.md +++ b/docs/credits.md @@ -3,6 +3,8 @@ hide: - toc --- + ```python exec="yes" --8<-- "scripts/gen_credits.py" ``` + diff --git a/docs/css/insiders.css b/docs/css/insiders.css index b5547bd1..e7b9c74f 100644 --- a/docs/css/insiders.css +++ b/docs/css/insiders.css @@ -53,11 +53,10 @@ a.insiders { } .sponsorship-item { - float: left; border-radius: 100%; - display: block; + display: inline-block; height: 1.6rem; - margin: .2rem; + margin: 0.1rem; overflow: hidden; width: 1.6rem; } diff --git a/docs/insiders/index.md b/docs/insiders/index.md index eb8feea0..99761b96 100644 --- a/docs/insiders/index.md +++ b/docs/insiders/index.md @@ -65,18 +65,21 @@ data_source = [ ] ``` + ```python exec="1" session="insiders" --8<-- "scripts/insiders.py" ``` -```python exec="1" session="insiders" -print(f"""The moment you become a sponsor, you'll get **immediate -access to {len(unreleased_features)} additional features** that you can start using right away, and -which are currently exclusively available to sponsors:\n""") +print( + f"""The moment you become a sponsor, you'll get **immediate + access to {len(unreleased_features)} additional features** that you can start using right away, and + which are currently exclusively available to sponsors:\n""" +) for feature in unreleased_features: feature.render(badge=True) ``` + ## How to become a sponsor @@ -127,9 +130,6 @@ You can cancel your sponsorship anytime.[^5]
-
-
- If you sponsor publicly, you're automatically added here with a link to your profile and avatar to show your support for *mkdocstrings*. diff --git a/duties.py b/duties.py index 93311189..43ae357a 100644 --- a/duties.py +++ b/duties.py @@ -4,12 +4,13 @@ import os import sys +from contextlib import contextmanager from importlib.metadata import version as pkgversion from pathlib import Path -from typing import TYPE_CHECKING, Any +from typing import TYPE_CHECKING, Iterator from duty import duty -from duty.callables import black, blacken_docs, coverage, lazy, mkdocs, mypy, pytest, ruff, safety +from duty.callables import black, coverage, lazy, mkdocs, mypy, pytest, ruff, safety if TYPE_CHECKING: from duty.context import Context @@ -31,32 +32,16 @@ def pyprefix(title: str) -> str: # noqa: D103 return title -def merge(d1: Any, d2: Any) -> Any: # noqa: D103 - basic_types = (int, float, str, bool, complex) - if isinstance(d1, dict) and isinstance(d2, dict): - for key, value in d2.items(): - if key in d1: - if isinstance(d1[key], basic_types): - d1[key] = value - else: - d1[key] = merge(d1[key], value) - else: - d1[key] = value - return d1 - if isinstance(d1, list) and isinstance(d2, list): - return d1 + d2 - return d2 - - -def mkdocs_config() -> str: # noqa: D103 - import mergedeep - - # force YAML loader to merge arrays - mergedeep.merge = merge - +@contextmanager +def material_insiders() -> Iterator[bool]: # noqa: D103 if "+insiders" in pkgversion("mkdocs-material"): - return "mkdocs.insiders.yml" - return "mkdocs.yml" + os.environ["MATERIAL_INSIDERS"] = "true" + try: + yield True + finally: + os.environ.pop("MATERIAL_INSIDERS") + else: + yield False @duty @@ -66,23 +51,9 @@ def changelog(ctx: Context) -> None: Parameters: ctx: The context instance (passed automatically). """ - from git_changelog.cli import build_and_render + from git_changelog.cli import main as git_changelog - git_changelog = lazy(build_and_render, name="git_changelog") - ctx.run( - git_changelog( - repository=".", - output="CHANGELOG.md", - convention="angular", - template="keepachangelog", - parse_trailers=True, - parse_refs=False, - sections=["build", "deps", "feat", "fix", "refactor"], - bump_latest=True, - in_place=True, - ), - title="Updating changelog", - ) + ctx.run(git_changelog, args=[[]], title="Updating changelog") @duty(pre=["check_quality", "check_types", "check_docs", "check_dependencies", "check-api"]) @@ -138,12 +109,12 @@ def check_docs(ctx: Context) -> None: """ Path("htmlcov").mkdir(parents=True, exist_ok=True) Path("htmlcov/index.html").touch(exist_ok=True) - config = mkdocs_config() - ctx.run( - mkdocs.build(strict=True, config_file=config, verbose=True), - title=pyprefix("Building documentation"), - command=f"mkdocs build -vsf {config}", - ) + with material_insiders(): + ctx.run( + mkdocs.build(strict=True, verbose=True), + title=pyprefix("Building documentation"), + command="mkdocs build -vs", + ) @duty @@ -208,11 +179,12 @@ def docs(ctx: Context, host: str = "127.0.0.1", port: int = 8000) -> None: host: The host to serve the docs from. port: The port to serve the docs on. """ - ctx.run( - mkdocs.serve(dev_addr=f"{host}:{port}", config_file=mkdocs_config()), - title="Serving documentation", - capture=False, - ) + with material_insiders(): + ctx.run( + mkdocs.serve(dev_addr=f"{host}:{port}"), + title="Serving documentation", + capture=False, + ) @duty @@ -223,22 +195,26 @@ def docs_deploy(ctx: Context) -> None: ctx: The context instance (passed automatically). """ os.environ["DEPLOY"] = "true" - config_file = mkdocs_config() - if config_file == "mkdocs.yml": - ctx.run(lambda: False, title="Not deploying docs without Material for MkDocs Insiders!") - origin = ctx.run("git config --get remote.origin.url", silent=True) - if "pawamoy-insiders/mkdocstrings" in origin: - ctx.run("git remote add org-pages git@github.com:mkdocstrings/mkdocstrings.github.io", silent=True, nofail=True) - ctx.run( - mkdocs.gh_deploy(config_file=config_file, remote_name="org-pages", force=True), - title="Deploying documentation", - ) - else: - ctx.run( - lambda: False, - title="Not deploying docs from public repository (do that from insiders instead!)", - nofail=True, - ) + with material_insiders() as insiders: + if not insiders: + ctx.run(lambda: False, title="Not deploying docs without Material for MkDocs Insiders!") + origin = ctx.run("git config --get remote.origin.url", silent=True) + if "pawamoy-insiders/mkdocstrings" in origin: + ctx.run( + "git remote add org-pages git@github.com:mkdocstrings/mkdocstrings.github.io", + silent=True, + nofail=True, + ) + ctx.run( + mkdocs.gh_deploy(remote_name="upstream", force=True), + title="Deploying documentation", + ) + else: + ctx.run( + lambda: False, + title="Not deploying docs from public repository (do that from insiders instead!)", + nofail=True, + ) @duty @@ -253,11 +229,6 @@ def format(ctx: Context) -> None: title="Auto-fixing code", ) ctx.run(black.run(*PY_SRC_LIST, config="config/black.toml"), title="Formatting code") - ctx.run( - blacken_docs.run(*PY_SRC_LIST, "docs", exts=["py", "md"], line_length=120), - title="Formatting docs", - nofail=True, - ) @duty(post=["docs-deploy"]) @@ -310,3 +281,28 @@ def test(ctx: Context, match: str = "") -> None: title=pyprefix("Running tests"), command=f"pytest -c config/pytest.ini -n auto -k{match!r} --color=yes tests", ) + + +@duty +def vscode(ctx: Context) -> None: + """Configure VSCode. + + This task will overwrite the following files, + so make sure to back them up: + + - `.vscode/launch.json` + - `.vscode/settings.json` + - `.vscode/tasks.json` + + Parameters: + ctx: The context instance (passed automatically). + """ + + def update_config(filename: str) -> None: + source_file = Path("config", "vscode", filename) + target_file = Path(".vscode", filename) + target_file.parent.mkdir(exist_ok=True) + target_file.write_text(source_file.read_text()) + + for filename in ("launch.json", "settings.json", "tasks.json"): + ctx.run(update_config, args=[filename], title=f"Update .vscode/{filename}") diff --git a/mkdocs.insiders.yml b/mkdocs.insiders.yml deleted file mode 100644 index a93edcc3..00000000 --- a/mkdocs.insiders.yml +++ /dev/null @@ -1,5 +0,0 @@ -INHERIT: mkdocs.yml - -# waiting for https://github.com/squidfunk/mkdocs-material/issues/5446 -# plugins: -# - typeset diff --git a/mkdocs.yml b/mkdocs.yml index c5d191f9..37a98cf4 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -96,12 +96,13 @@ markdown_extensions: - admonition - callouts - footnotes -- pymdownx.emoji: - emoji_index: !!python/name:materialx.emoji.twemoji - emoji_generator: !!python/name:materialx.emoji.to_svg - pymdownx.details +- pymdownx.emoji: + emoji_index: !!python/name:material.extensions.emoji.twemoji + emoji_generator: !!python/name:material.extensions.emoji.to_svg - pymdownx.magiclink - pymdownx.snippets: + base_path: [!relative $config_dir] check_paths: true - pymdownx.superfences - pymdownx.tabbed: @@ -122,7 +123,7 @@ plugins: scripts: - scripts/gen_ref_nav.py - literate-nav: - nav_file: SUMMARY.txt + nav_file: SUMMARY.md - coverage - mkdocstrings: handlers: @@ -157,6 +158,10 @@ plugins: handlers/overview.md: usage/handlers.md - minify: minify_html: !ENV [DEPLOY, false] +- group: + enabled: !ENV [MATERIAL_INSIDERS, false] + plugins: + - typeset extra: social: diff --git a/pyproject.toml b/pyproject.toml index 7077ccf8..1e687fdf 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -72,44 +72,43 @@ duty = ["duty>=0.10"] ci-quality = ["mkdocstrings[duty,docs,quality,typing,security]"] ci-tests = ["mkdocstrings[duty,docs,tests]"] docs = [ - "black>=23.1", - "markdown-callouts>=0.2", - "markdown-exec>=0.5", + "black>=23.9", + "markdown-callouts>=0.3", + "markdown-exec>=1.7", "mkdocs>=1.5", - "mkdocs-coverage>=0.2", - "mkdocs-gen-files>=0.3", - "mkdocs-git-committers-plugin-2>=1.1", - "mkdocs-literate-nav>=0.4", - "mkdocs-material>=7.3", - "mkdocs-minify-plugin>=0.6.4", - "mkdocs-redirects>=1.2.0", - "mkdocstrings-python>=0.5.1", - "toml>=0.10", + "mkdocs-coverage>=1.0", + "mkdocs-gen-files>=0.5", + "mkdocs-git-committers-plugin-2>=1.2", + "mkdocs-literate-nav>=0.6", + "mkdocs-material>=9.4", + "mkdocs-minify-plugin>=0.7", + "mkdocs-redirects>=1.2", + "mkdocstrings-python>=1.7", + "tomli>=2.0; python_version < '3.11'", ] maintain = [ - "black>=23.1", - "blacken-docs>=1.13", - "git-changelog>=1.0", + "black>=23.9", + "blacken-docs>=1.16", + "git-changelog>=2.3", ] quality = [ - "ruff>=0.0.246", + "ruff>=0.0", ] tests = [ "docutils", "pygments>=2.10", # python 3.6 - "pytest>=6.2", - "pytest-cov>=3.0", - "pytest-randomly>=3.10", - "pytest-xdist>=2.4", + "pytest>=7.4", + "pytest-cov>=4.1", + "pytest-randomly>=3.15", + "pytest-xdist>=3.3", "sphinx", ] typing = [ - "mypy>=0.911", - "types-docutils", - "types-markdown>=3.3", + "mypy>=1.5", + "types-docutils>=0.20,", + "types-markdown>=3.5", "types-pyyaml>=6.0", - "types-toml>=0.10", ] security = [ - "safety>=2", + "safety>=2.3", ] diff --git a/scripts/gen_credits.py b/scripts/gen_credits.py index bc01c0bd..bf35f0da 100644 --- a/scripts/gen_credits.py +++ b/scripts/gen_credits.py @@ -2,27 +2,31 @@ from __future__ import annotations +import os import re import sys +from importlib.metadata import PackageNotFoundError, metadata from itertools import chain from pathlib import Path from textwrap import dedent from typing import Mapping, cast -import toml from jinja2 import StrictUndefined from jinja2.sandbox import SandboxedEnvironment -if sys.version_info < (3, 8): - from importlib_metadata import PackageNotFoundError, metadata +# TODO: Remove once support for Python 3.10 is dropped. +if sys.version_info >= (3, 11): + import tomllib else: - from importlib.metadata import PackageNotFoundError, metadata + import tomli as tomllib -project_dir = Path(".") -pyproject = toml.load(project_dir / "pyproject.toml") +project_dir = Path(os.getenv("MKDOCS_CONFIG_DIR", ".")) +with project_dir.joinpath("pyproject.toml").open("rb") as pyproject_file: + pyproject = tomllib.load(pyproject_file) project = pyproject["project"] pdm = pyproject["tool"]["pdm"] -lock_data = toml.load(project_dir / "pdm.lock") +with project_dir.joinpath("pdm.lock").open("rb") as lock_file: + lock_data = tomllib.load(lock_file) lock_pkgs = {pkg["name"].lower(): pkg for pkg in lock_data["package"]} project_name = project["name"] regex = re.compile(r"(?P[\w.-]+)(?P.*)$") @@ -35,7 +39,7 @@ def _get_license(pkg_name: str) -> str: return "?" license_name = cast(dict, data).get("License", "").strip() multiple_lines = bool(license_name.count("\n")) - # TODO: remove author logic once all my packages licenses are fixed + # TODO: Remove author logic once all my packages licenses are fixed. author = "" if multiple_lines or not license_name or license_name == "UNKNOWN": for header, value in cast(dict, data).items(): diff --git a/scripts/gen_ref_nav.py b/scripts/gen_ref_nav.py index 249530b1..9c041ede 100644 --- a/scripts/gen_ref_nav.py +++ b/scripts/gen_ref_nav.py @@ -7,9 +7,11 @@ nav = mkdocs_gen_files.Nav() mod_symbol = '' -for path in sorted(Path("src").rglob("*.py")): - module_path = path.relative_to("src").with_suffix("") - doc_path = path.relative_to("src/mkdocstrings").with_suffix(".md") +src = Path(__file__).parent.parent / "src" + +for path in sorted(src.rglob("*.py")): + module_path = path.relative_to(src).with_suffix("") + doc_path = path.relative_to(src / "mkdocstrings").with_suffix(".md") full_doc_path = Path("reference", doc_path) parts = tuple(module_path.parts) @@ -30,5 +32,5 @@ mkdocs_gen_files.set_edit_path(full_doc_path, ".." / path) -with mkdocs_gen_files.open("reference/SUMMARY.txt", "w") as nav_file: +with mkdocs_gen_files.open("reference/SUMMARY.md", "w") as nav_file: nav_file.writelines(nav.build_literate_nav()) diff --git a/scripts/insiders.py b/scripts/insiders.py index 28ca1c87..8f5e215e 100644 --- a/scripts/insiders.py +++ b/scripts/insiders.py @@ -4,6 +4,7 @@ import json import logging +import os import posixpath from dataclasses import dataclass from datetime import date, datetime, timedelta @@ -115,8 +116,9 @@ def load_goals(data: str, funding: int = 0, project: Project | None = None) -> d def _load_goals_from_disk(path: str, funding: int = 0) -> dict[int, Goal]: + project_dir = os.getenv("MKDOCS_CONFIG_DIR", ".") try: - data = Path(path).read_text() + data = Path(project_dir, path).read_text() except OSError as error: raise RuntimeError(f"Could not load data from disk: {path}") from error return load_goals(data, funding) @@ -159,7 +161,7 @@ def funding_goals(source: str | list[str | tuple[str, str, str]], funding: int = goals[amount] = goal else: goals[amount].features.extend(goal.features) - return goals + return {amount: goals[amount] for amount in sorted(goals)} def feature_list(goals: Iterable[Goal]) -> list[Feature]: diff --git a/src/mkdocstrings/debug.py b/src/mkdocstrings/debug.py new file mode 100644 index 00000000..35ede743 --- /dev/null +++ b/src/mkdocstrings/debug.py @@ -0,0 +1,106 @@ +"""Debugging utilities.""" + +from __future__ import annotations + +import os +import platform +import sys +from dataclasses import dataclass +from importlib import metadata + + +@dataclass +class Variable: + """Dataclass describing an environment variable.""" + + name: str + """Variable name.""" + value: str + """Variable value.""" + + +@dataclass +class Package: + """Dataclass describing a Python package.""" + + name: str + """Package name.""" + version: str + """Package version.""" + + +@dataclass +class Environment: + """Dataclass to store environment information.""" + + interpreter_name: str + """Python interpreter name.""" + interpreter_version: str + """Python interpreter version.""" + platform: str + """Operating System.""" + packages: list[Package] + """Installed packages.""" + variables: list[Variable] + """Environment variables.""" + + +def _interpreter_name_version() -> tuple[str, str]: + if hasattr(sys, "implementation"): + impl = sys.implementation.version + version = f"{impl.major}.{impl.minor}.{impl.micro}" + kind = impl.releaselevel + if kind != "final": + version += kind[0] + str(impl.serial) + return sys.implementation.name, version + return "", "0.0.0" + + +def get_version(dist: str = "mkdocstrings") -> str: + """Get version of the given distribution. + + Parameters: + dist: A distribution name. + + Returns: + A version number. + """ + try: + return metadata.version(dist) + except metadata.PackageNotFoundError: + return "0.0.0" + + +def get_debug_info() -> Environment: + """Get debug/environment information. + + Returns: + Environment information. + """ + py_name, py_version = _interpreter_name_version() + packages = ["mkdocstrings"] + variables = ["PYTHONPATH", *[var for var in os.environ if var.startswith("MKDOCSTRINGS")]] + return Environment( + interpreter_name=py_name, + interpreter_version=py_version, + platform=platform.platform(), + variables=[Variable(var, val) for var in variables if (val := os.getenv(var))], + packages=[Package(pkg, get_version(pkg)) for pkg in packages], + ) + + +def print_debug_info() -> None: + """Print debug/environment information.""" + info = get_debug_info() + print(f"- __System__: {info.platform}") + print(f"- __Python__: {info.interpreter_name} {info.interpreter_version}") + print("- __Environment variables__:") + for var in info.variables: + print(f" - `{var.name}`: `{var.value}`") + print("- __Installed packages__:") + for pkg in info.packages: + print(f" - `{pkg.name}` v{pkg.version}") + + +if __name__ == "__main__": + print_debug_info() diff --git a/tests/test_plugin.py b/tests/test_plugin.py index 26c4031a..3342e2aa 100644 --- a/tests/test_plugin.py +++ b/tests/test_plugin.py @@ -6,6 +6,7 @@ from mkdocs.commands.build import build from mkdocs.config import load_config + from mkdocstrings.plugin import MkdocstringsPlugin if TYPE_CHECKING: @@ -49,6 +50,7 @@ def test_plugin_default_config(tmp_path: Path) -> None: "enabled": True, } + def test_plugin_config_custom_templates(tmp_path: Path) -> None: """Test custom_templates option is relative to config file.""" config_file_path = tmp_path / "mkdocs.yml" From 4dbb6d6a2579d81d65244c0ab1df7e0ee0827fb8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Timoth=C3=A9e=20Mazzucotelli?= Date: Sun, 12 Nov 2023 01:24:32 +0100 Subject: [PATCH 11/16] ci: Ruff auto-fix --- tests/conftest.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/conftest.py b/tests/conftest.py index 59bac65a..be4802a2 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -7,12 +7,13 @@ import pytest from markdown.core import Markdown -from mkdocs import config from mkdocs.config.defaults import MkDocsConfig if TYPE_CHECKING: from pathlib import Path + from mkdocs import config + from mkdocstrings.plugin import MkdocstringsPlugin From d74fada8721e366fce81d61eda06d86c15b9a8b1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Timoth=C3=A9e=20Mazzucotelli?= Date: Sun, 12 Nov 2023 01:52:51 +0100 Subject: [PATCH 12/16] tests: Stop passing config file path to MkDocsConfig --- tests/conftest.py | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/conftest.py b/tests/conftest.py index be4802a2..9bb09368 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -25,7 +25,6 @@ def fixture_mkdocs_conf(request: pytest.FixtureRequest, tmp_path: Path) -> Itera request = request._parent_request conf_dict = { - "config_file_path": "mkdocs_tests.yml", "site_name": "foo", "site_url": "https://example.org/", "site_dir": str(tmp_path), From b3edf89572db5693688bccbd9642822ef4673095 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Timoth=C3=A9e=20Mazzucotelli?= Date: Sun, 12 Nov 2023 01:55:05 +0100 Subject: [PATCH 13/16] ci: Some typing fixes/ignore --- src/mkdocstrings/handlers/base.py | 19 ++++++++++--------- src/mkdocstrings/handlers/rendering.py | 3 ++- src/mkdocstrings/plugin.py | 7 ++++--- tests/test_extension.py | 2 +- 4 files changed, 17 insertions(+), 14 deletions(-) diff --git a/src/mkdocstrings/handlers/base.py b/src/mkdocstrings/handlers/base.py index 700a0565..f52e17dc 100644 --- a/src/mkdocstrings/handlers/base.py +++ b/src/mkdocstrings/handlers/base.py @@ -8,11 +8,12 @@ import importlib import sys from pathlib import Path -from typing import Any, BinaryIO, ClassVar, Iterable, Iterator, Mapping, MutableMapping, Sequence +from typing import Any, BinaryIO, ClassVar, Iterable, Iterator, Mapping, MutableMapping, Sequence, cast from xml.etree.ElementTree import Element, tostring from jinja2 import Environment, FileSystemLoader from markdown import Markdown +from markdown.extensions.toc import TocTreeprocessor from markupsafe import Markup from mkdocstrings.handlers.rendering import ( @@ -268,15 +269,15 @@ def do_convert_markdown( An HTML string. """ treeprocessors = self._md.treeprocessors - treeprocessors[HeadingShiftingTreeprocessor.name].shift_by = heading_level - treeprocessors[IdPrependingTreeprocessor.name].id_prefix = html_id and html_id + "--" - treeprocessors[ParagraphStrippingTreeprocessor.name].strip = strip_paragraph + treeprocessors[HeadingShiftingTreeprocessor.name].shift_by = heading_level # type: ignore[attr-defined] + treeprocessors[IdPrependingTreeprocessor.name].id_prefix = html_id and html_id + "--" # type: ignore[attr-defined] + treeprocessors[ParagraphStrippingTreeprocessor.name].strip = strip_paragraph # type: ignore[attr-defined] try: return Markup(self._md.convert(text)) finally: - treeprocessors[HeadingShiftingTreeprocessor.name].shift_by = 0 - treeprocessors[IdPrependingTreeprocessor.name].id_prefix = "" - treeprocessors[ParagraphStrippingTreeprocessor.name].strip = False + treeprocessors[HeadingShiftingTreeprocessor.name].shift_by = 0 # type: ignore[attr-defined] + treeprocessors[IdPrependingTreeprocessor.name].id_prefix = "" # type: ignore[attr-defined] + treeprocessors[ParagraphStrippingTreeprocessor.name].strip = False # type: ignore[attr-defined] self._md.reset() def do_heading( @@ -319,7 +320,7 @@ def do_heading( el = Element(f"h{heading_level}", attributes) el.append(Element("mkdocstrings-placeholder")) # Tell the 'toc' extension to make its additions if configured so. - toc = self._md.treeprocessors["toc"] + toc = cast(TocTreeprocessor, self._md.treeprocessors["toc"]) if toc.use_anchors: toc.add_anchor(el, attributes["id"]) if toc.use_permalinks: @@ -388,7 +389,7 @@ def __init__(self, config: dict) -> None: self._handlers: dict[str, BaseHandler] = {} self.inventory: Inventory = Inventory(project=self._config["mkdocs"]["site_name"]) - def get_anchors(self, identifier: str) -> tuple[str, ...] | set[str]: + def get_anchors(self, identifier: str) -> tuple[str, ...]: """Return the canonical HTML anchor for the identifier, if any of the seen handlers can collect it. Arguments: diff --git a/src/mkdocstrings/handlers/rendering.py b/src/mkdocstrings/handlers/rendering.py index 6009935a..2cba2538 100644 --- a/src/mkdocstrings/handlers/rendering.py +++ b/src/mkdocstrings/handlers/rendering.py @@ -211,12 +211,13 @@ def __init__(self, md: Markdown, headings: list[Element]): self.headings = headings def run(self, root: Element) -> None: + permalink_class = self.md.treeprocessors["toc"].permalink_class # type: ignore[attr-defined] for el in root.iter(): if self.regex.fullmatch(el.tag): el = copy.copy(el) # noqa: PLW2901 # 'toc' extension's first pass (which we require to build heading stubs/ids) also edits the HTML. # Undo the permalink edit so we can pass this heading to the outer pass of the 'toc' extension. - if len(el) > 0 and el[-1].get("class") == self.md.treeprocessors["toc"].permalink_class: + if len(el) > 0 and el[-1].get("class") == permalink_class: del el[-1] self.headings.append(el) diff --git a/src/mkdocstrings/plugin.py b/src/mkdocstrings/plugin.py index 3f8ce8cd..dd720330 100644 --- a/src/mkdocstrings/plugin.py +++ b/src/mkdocstrings/plugin.py @@ -172,9 +172,10 @@ def on_config(self, config: MkDocsConfig) -> MkDocsConfig | None: } self._handlers = Handlers(extension_config) + autorefs: AutorefsPlugin try: # If autorefs plugin is explicitly enabled, just use it. - autorefs = config.plugins["autorefs"] + autorefs = config.plugins["autorefs"] # type: ignore[assignment] log.debug(f"Picked up existing autorefs instance {autorefs!r}") except KeyError: # Otherwise, add a limited instance of it that acts only on what's added through `register_anchor`. @@ -186,7 +187,7 @@ def on_config(self, config: MkDocsConfig) -> MkDocsConfig | None: autorefs.get_fallback_anchor = self.handlers.get_anchors mkdocstrings_extension = MkdocstringsExtension(extension_config, self.handlers, autorefs) - config.markdown_extensions.append(mkdocstrings_extension) + config.markdown_extensions.append(mkdocstrings_extension) # type: ignore[arg-type] config.extra_css.insert(0, self.css_filename) # So that it has lower priority than user files. @@ -258,7 +259,7 @@ def on_env(self, env: Environment, config: MkDocsConfig, *args: Any, **kwargs: A loader_name = loader.__func__.__qualname__ log.error(f"Couldn't load inventory {import_item} through {loader_name}: {error}") # noqa: TRY400 for page, identifier in results.items(): - config.plugins["autorefs"].register_url(page, identifier) + config.plugins["autorefs"].register_url(page, identifier) # type: ignore[attr-defined] self._inv_futures = {} def on_post_build( diff --git a/tests/test_extension.py b/tests/test_extension.py index 6011be7f..4b470647 100644 --- a/tests/test_extension.py +++ b/tests/test_extension.py @@ -140,7 +140,7 @@ def test_dont_register_every_identifier_as_anchor(plugin: MkdocstringsPlugin, ex ids = ("id1", "id2", "id3") handler.get_anchors = lambda _: ids # type: ignore[method-assign] ext_markdown.convert("::: tests.fixtures.headings") - autorefs = ext_markdown.parser.blockprocessors["mkdocstrings"]._autorefs + autorefs = ext_markdown.parser.blockprocessors["mkdocstrings"]._autorefs # type: ignore[attr-defined] for identifier in ids: assert identifier not in autorefs._url_map assert identifier not in autorefs._abs_url_map From 4a97755e700529b48abb8c15a85cfbae2e17a09d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Timoth=C3=A9e=20Mazzucotelli?= Date: Sun, 12 Nov 2023 17:52:37 +0100 Subject: [PATCH 14/16] docs: Make recipe work with MkDocs `-f` option --- docs/recipes.md | 143 +++++++++++++++++++++++++++--------------------- 1 file changed, 81 insertions(+), 62 deletions(-) diff --git a/docs/recipes.md b/docs/recipes.md index c33130f0..8ea849fc 100644 --- a/docs/recipes.md +++ b/docs/recipes.md @@ -17,15 +17,15 @@ Let say you have a project called `project`. This project has a lot of source files, or modules, which live in the `src` folder: -``` -📁 repo -└─╴📁 src - └─╴📁 project - ├─╴📄 lorem - ├─╴📄 ipsum - ├─╴📄 dolor - ├─╴📄 sit - └─╴📄 amet +```tree +repo/ + src/ + project/ + lorem + ipsum + dolor + sit + amet ``` Without an automatic process, you will have to manually @@ -49,10 +49,10 @@ and configure it like so: ```yaml title="mkdocs.yml" plugins: -- search # (1) +- search # (1)! - gen-files: scripts: - - docs/gen_ref_pages.py # (2) + - scripts/gen_ref_pages.py # (2)! - mkdocstrings ``` @@ -60,76 +60,91 @@ plugins: 2. The magic happens here, see below how it works. mkdocs-gen-files is able to run Python scripts at build time. -The Python script that we will execute lives in the docs folder, +The Python script that we will execute lives in a scripts folder, and is named `gen_ref_pages.py`, like "generate code reference pages". -```python title="docs/gen_ref_pages.py" +```tree +repo/ + docs/ + index.md + scripts/ + gen_ref_pages.py + src/ + project/ + mkdocs.yml +``` + +```python title="scripts/gen_ref_pages.py" """Generate the code reference pages.""" from pathlib import Path import mkdocs_gen_files -for path in sorted(Path("src").rglob("*.py")): # (1) - module_path = path.relative_to("src").with_suffix("") # (2) - doc_path = path.relative_to("src").with_suffix(".md") # (3) - full_doc_path = Path("reference", doc_path) # (4) +src = Path(__file__).parent.parent / "src" # (1)! + +for path in sorted(src.rglob("*.py")): # (2)! + module_path = path.relative_to(src).with_suffix("") # (3)! + doc_path = path.relative_to(src).with_suffix(".md") # (4)! + full_doc_path = Path("reference", doc_path) # (5)! parts = list(module_path.parts) - if parts[-1] == "__init__": # (5) + if parts[-1] == "__init__": # (6)! parts = parts[:-1] elif parts[-1] == "__main__": continue - with mkdocs_gen_files.open(full_doc_path, "w") as fd: # (6) - identifier = ".".join(parts) # (7) - print("::: " + identifier, file=fd) # (8) + with mkdocs_gen_files.open(full_doc_path, "w") as fd: # (7)! + identifier = ".".join(parts) # (8)! + print("::: " + identifier, file=fd) # (9)! - mkdocs_gen_files.set_edit_path(full_doc_path, path) # (9) + mkdocs_gen_files.set_edit_path(full_doc_path, path) # (10)! ``` -1. Here we recursively list all `.py` files, but you can adapt the code to list +1. It's important to build a path relative to the script itself, + to make it possible to build the docs with MkDocs' + [`-f` option](https://www.mkdocs.org/user-guide/cli/#mkdocs-build). +2. Here we recursively list all `.py` files, but you can adapt the code to list files with other extensions of course, supporting other languages than Python. -2. The module path will look like `project/lorem`. +3. The module path will look like `project/lorem`. It will be used to build the *mkdocstrings* autodoc identifier. -3. This is the relative path to the Markdown page. -4. This is the absolute path to the Markdown page. Here we put all reference pages - into a `reference` folder. -5. This part is only relevant for Python modules. We skip `__main__` modules and +4. This is the partial path of the Markdown page for the module. +5. This is the full path of the Markdown page within the docs. + Here we put all reference pages into a `reference` folder. +6. This part is only relevant for Python modules. We skip `__main__` modules and remove `__init__` from the module parts as it's implicit during imports. -6. Magic! Add the file to MkDocs pages, without actually writing it in the docs folder. -7. Build the autodoc identifier. Here we document Python modules, so the identifier +7. Magic! Add the file to MkDocs pages, without actually writing it in the docs folder. +8. Build the autodoc identifier. Here we document Python modules, so the identifier is a dot-separated path, like `project.lorem`. -8. Actually write to the magic file. -9. We can even set the `edit_uri` on the pages. +9. Actually write to the magic file. +10. We can even set the `edit_uri` on the pages. > NOTE: > It is important to look out for correct edit page behaviour when using generated pages. > For example, if we have `edit_uri` set to `blob/master/docs/` and the following > file structure: > -> ``` -> 📁 repo -> ├─ 📄 mkdocs.yml -> │ -> ├─ 📁 docs -> │ ├─╴📄 index.md -> │ └─╴📄 gen_ref_pages.py -> │ -> └─╴📁 src -> └─╴📁 project -> ├─╴📄 lorem.py -> ├─╴📄 ipsum.py -> ├─╴📄 dolor.py -> ├─╴📄 sit.py -> └─╴📄 amet.py +> ```tree +> repo/ +> mkdocs.yml +> docs/ +> index.md +> scripts/ +> gen_ref_pages.py +> src/ +> project/ +> lorem.py +> ipsum.py +> dolor.py +> sit.py +> amet.py > ``` > > Then we will have to change our `set_edit_path` call to: > > ```python -> mkdocs_gen_files.set_edit_path(full_doc_path, Path("../") / path) # (1) +> mkdocs_gen_files.set_edit_path(full_doc_path, Path("../") / path) # (1)! > ``` > > 1. Path can be used to traverse the structure in any way you may need, but @@ -180,7 +195,7 @@ plugins: - search - gen-files: scripts: - - docs/gen_ref_pages.py + - scripts/gen_ref_pages.py - literate-nav: nav_file: SUMMARY.md - mkdocstrings @@ -188,7 +203,7 @@ plugins: Then, the previous script is updated like so: -```python title="docs/gen_ref_pages.py" hl_lines="7 21 29 30" +```python title="scripts/gen_ref_pages.py" hl_lines="7 23 31 32" """Generate the code reference pages and navigation.""" from pathlib import Path @@ -197,9 +212,11 @@ import mkdocs_gen_files nav = mkdocs_gen_files.Nav() -for path in sorted(Path("src").rglob("*.py")): - module_path = path.relative_to("src").with_suffix("") - doc_path = path.relative_to("src").with_suffix(".md") +src = Path(__file__).parent.parent / "src" + +for path in sorted(src.rglob("*.py")): + module_path = path.relative_to(src).with_suffix("") + doc_path = path.relative_to(src).with_suffix(".md") full_doc_path = Path("reference", doc_path) parts = tuple(module_path.parts) @@ -209,7 +226,7 @@ for path in sorted(Path("src").rglob("*.py")): elif parts[-1] == "__main__": continue - nav[parts] = doc_path.as_posix() # (1) + nav[parts] = doc_path.as_posix() # (1)! with mkdocs_gen_files.open(full_doc_path, "w") as fd: ident = ".".join(parts) @@ -217,8 +234,8 @@ for path in sorted(Path("src").rglob("*.py")): mkdocs_gen_files.set_edit_path(full_doc_path, path) -with mkdocs_gen_files.open("reference/SUMMARY.md", "w") as nav_file: # (2) - nav_file.writelines(nav.build_literate_nav()) # (3) +with mkdocs_gen_files.open("reference/SUMMARY.md", "w") as nav_file: # (2)! + nav_file.writelines(nav.build_literate_nav()) # (3)! ``` 1. Progressively build the navigation object. @@ -232,7 +249,7 @@ and replace it with a single line! nav: # rest of the navigation... # defer to gen-files + literate-nav -- Code Reference: reference/ # (1) +- Code Reference: reference/ # (1)! # rest of the navigation... ``` @@ -259,7 +276,7 @@ Well, this is possible thanks to a third plugin: Update the script like this: -```python title="docs/gen_ref_pages.py" hl_lines="18 19" +```python title="scripts/gen_ref_pages.py" hl_lines="20 21" """Generate the code reference pages and navigation.""" from pathlib import Path @@ -268,9 +285,11 @@ import mkdocs_gen_files nav = mkdocs_gen_files.Nav() -for path in sorted(Path("src").rglob("*.py")): - module_path = path.relative_to("src").with_suffix("") - doc_path = path.relative_to("src").with_suffix(".md") +src = Path(__file__).parent.parent / "src" + +for path in sorted(src.rglob("*.py")): + module_path = path.relative_to(src).with_suffix("") + doc_path = path.relative_to(src).with_suffix(".md") full_doc_path = Path("reference", doc_path) parts = tuple(module_path.parts) @@ -301,7 +320,7 @@ plugins: - search - gen-files: scripts: - - docs/gen_ref_pages.py + - scripts/gen_ref_pages.py - literate-nav: nav_file: SUMMARY.md - section-index From ce84dd57dc5cd3bf3f4be9623ddaa73e1c1868f0 Mon Sep 17 00:00:00 2001 From: Oleh Prypin Date: Sun, 12 Nov 2023 18:01:10 +0100 Subject: [PATCH 15/16] feat: Cache downloaded inventories as local file PR #632: https://github.com/mkdocstrings/mkdocstrings/pull/632 --- pyproject.toml | 2 + src/mkdocstrings/_cache.py | 76 ++++++++++++++++++++++++++++++++++++++ src/mkdocstrings/plugin.py | 15 +++----- 3 files changed, 84 insertions(+), 9 deletions(-) create mode 100644 src/mkdocstrings/_cache.py diff --git a/pyproject.toml b/pyproject.toml index 1e687fdf..1be96db8 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -29,11 +29,13 @@ classifiers = [ "Typing :: Typed", ] dependencies = [ + "click>=7.0", "Jinja2>=2.11.1", "Markdown>=3.3", "MarkupSafe>=1.1", "mkdocs>=1.4", "mkdocs-autorefs>=0.3.1", + "platformdirs>=2.2.0", "pymdown-extensions>=6.3", "importlib-metadata>=4.6; python_version < '3.10'", "typing-extensions>=4.1; python_version < '3.10'", diff --git a/src/mkdocstrings/_cache.py b/src/mkdocstrings/_cache.py new file mode 100644 index 00000000..8737f317 --- /dev/null +++ b/src/mkdocstrings/_cache.py @@ -0,0 +1,76 @@ +import datetime +import gzip +import hashlib +import os +import urllib.parse +import urllib.request +from typing import BinaryIO, Callable + +import click +import platformdirs + +from mkdocstrings.loggers import get_logger + +log = get_logger(__name__) + + +def download_url_with_gz(url: str) -> bytes: + req = urllib.request.Request( # noqa: S310 + url, + headers={"Accept-Encoding": "gzip", "User-Agent": "mkdocstrings/0.15.0"}, + ) + with urllib.request.urlopen(req) as resp: # noqa: S310 + content: BinaryIO = resp + if "gzip" in resp.headers.get("content-encoding", ""): + content = gzip.GzipFile(fileobj=resp) # type: ignore[assignment] + return content.read() + + +# This is mostly a copy of https://github.com/mkdocs/mkdocs/blob/master/mkdocs/utils/cache.py +# In the future maybe they can be deduplicated. + + +def download_and_cache_url( + url: str, + download: Callable[[str], bytes], + cache_duration: datetime.timedelta, + comment: bytes = b"# ", +) -> bytes: + """Downloads a file from the URL, stores it under ~/.cache/, and returns its content. + + For tracking the age of the content, a prefix is inserted into the stored file, rather than relying on mtime. + + Args: + url: URL to use. + download: Callback that will accept the URL and actually perform the download. + cache_duration: How long to consider the URL content cached. + comment: The appropriate comment prefix for this file format. + """ + directory = os.path.join(platformdirs.user_cache_dir("mkdocs"), "mkdocstrings_url_cache") + name_hash = hashlib.sha256(url.encode()).hexdigest()[:32] + path = os.path.join(directory, name_hash + os.path.splitext(url)[1]) + + now = int(datetime.datetime.now(datetime.timezone.utc).timestamp()) + prefix = b"%s%s downloaded at timestamp " % (comment, url.encode()) + # Check for cached file and try to return it + if os.path.isfile(path): + try: + with open(path, "rb") as f: + line = f.readline() + if line.startswith(prefix): + line = line[len(prefix) :] + timestamp = int(line) + if datetime.timedelta(seconds=(now - timestamp)) <= cache_duration: + log.debug(f"Using cached '{path}' for '{url}'") + return f.read() + except (OSError, ValueError) as e: + log.debug(f"{type(e).__name__}: {e}") + + # Download and cache the file + log.debug(f"Downloading '{url}' to '{path}'") + content = download(url) + os.makedirs(directory, exist_ok=True) + with click.open_file(path, "wb", atomic=True) as f: + f.write(b"%s%d\n" % (prefix, now)) + f.write(content) + return content diff --git a/src/mkdocstrings/plugin.py b/src/mkdocstrings/plugin.py index dd720330..48a7d1ab 100644 --- a/src/mkdocstrings/plugin.py +++ b/src/mkdocstrings/plugin.py @@ -14,13 +14,13 @@ from __future__ import annotations +import datetime import functools -import gzip import os import sys from concurrent import futures -from typing import TYPE_CHECKING, Any, BinaryIO, Callable, Iterable, List, Mapping, Tuple, TypeVar -from urllib import request +from io import BytesIO +from typing import TYPE_CHECKING, Any, Callable, Iterable, List, Mapping, Tuple, TypeVar from mkdocs.config import Config from mkdocs.config import config_options as opt @@ -28,6 +28,7 @@ from mkdocs.utils import write_file from mkdocs_autorefs.plugin import AutorefsPlugin +from mkdocstrings._cache import download_and_cache_url, download_url_with_gz from mkdocstrings.extension import MkdocstringsExtension from mkdocstrings.handlers.base import BaseHandler, Handlers from mkdocstrings.loggers import get_logger @@ -317,11 +318,7 @@ def _load_inventory(cls, loader: InventoryLoaderType, url: str, **kwargs: Any) - A mapping from identifier to absolute URL. """ log.debug(f"Downloading inventory from {url!r}") - req = request.Request(url, headers={"Accept-Encoding": "gzip", "User-Agent": "mkdocstrings/0.15.0"}) - with request.urlopen(req) as resp: # noqa: S310 (URL audit OK: comes from a checked-in config) - content: BinaryIO = resp - if "gzip" in resp.headers.get("content-encoding", ""): - content = gzip.GzipFile(fileobj=resp) # type: ignore[assignment] - result = dict(loader(content, url=url, **kwargs)) + content = download_and_cache_url(url, download_url_with_gz, datetime.timedelta(days=1)) + result = dict(loader(BytesIO(content), url=url, **kwargs)) log.debug(f"Loaded inventory from {url!r}: {len(result)} items") return result From 032e4175799ab15a329a9b05b5df7b74650cea4d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Timoth=C3=A9e=20Mazzucotelli?= Date: Tue, 14 Nov 2023 18:46:07 +0100 Subject: [PATCH 16/16] chore: Prepare release 0.24.0 --- CHANGELOG.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index fc3a0fb0..c24f6724 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,23 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html). +## [0.24.0](https://github.com/mkdocstrings/mkdocstrings/releases/tag/0.24.0) - 2023-11-14 + +[Compare with 0.23.0](https://github.com/mkdocstrings/mkdocstrings/compare/0.23.0...0.24.0) + +### Features + +- Cache downloaded inventories as local file ([ce84dd5](https://github.com/mkdocstrings/mkdocstrings/commit/ce84dd57dc5cd3bf3f4be9623ddaa73e1c1868f0) by Oleh Prypin). [PR #632](https://github.com/mkdocstrings/mkdocstrings/pull/632) + +### Bug Fixes + +- Make `custom_templates` relative to the config file ([370a61d](https://github.com/mkdocstrings/mkdocstrings/commit/370a61d12b33f3fb61f6bddb3939eb8ff6018620) by Waylan Limberg). [Issue #477](https://github.com/mkdocstrings/mkdocstrings/issues/477), [PR #627](https://github.com/mkdocstrings/mkdocstrings/pull/627) +- Remove duplicated headings for docstrings nested in tabs/admonitions ([e2123a9](https://github.com/mkdocstrings/mkdocstrings/commit/e2123a935edea0abdc1b439e2c2b76e002c76e2b) by Perceval Wajsburt, [f4a94f7](https://github.com/mkdocstrings/mkdocstrings/commit/f4a94f7d8b8eb1ac01d65bb7237f0077e320ddac) by Oleh Prypin). [Issue #609](https://github.com/mkdocstrings/mkdocstrings/issues/609), [PR #610](https://github.com/mkdocstrings/mkdocstrings/pull/610), [PR #613](https://github.com/mkdocstrings/mkdocstrings/pull/613) + +### Code Refactoring + +- Drop support for MkDocs < 1.4, modernize usages ([b61d4d1](https://github.com/mkdocstrings/mkdocstrings/commit/b61d4d15258c66b14266aa04b456f191f101b2c6) by Oleh Prypin). [PR #629](https://github.com/mkdocstrings/mkdocstrings/pull/629) + ## [0.23.0](https://github.com/mkdocstrings/mkdocstrings/releases/tag/0.23.0) - 2023-08-28 [Compare with 0.22.0](https://github.com/mkdocstrings/mkdocstrings/compare/0.22.0...0.23.0)