diff --git a/.copier-answers.yml b/.copier-answers.yml index c720007f..0c51afe2 100644 --- a/.copier-answers.yml +++ b/.copier-answers.yml @@ -1,5 +1,5 @@ # Changes here will be overwritten by Copier -_commit: 0.16.6 +_commit: 1.1.3 _src_path: gh:pawamoy/copier-pdm.git author_email: pawamoy@pm.me author_fullname: TimothΓ©e Mazzucotelli @@ -9,8 +9,10 @@ copyright_holder: TimothΓ©e Mazzucotelli copyright_holder_email: pawamoy@pm.me copyright_license: ISC License insiders: true +insiders_repository_name: mkdocstrings project_description: Automatic documentation from sources, for MkDocs. project_name: mkdocstrings +public_release: true python_package_command_line_name: '' python_package_distribution_name: mkdocstrings python_package_import_name: mkdocstrings diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 149c6ce0..6ed84b16 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -1,36 +1,61 @@ --- name: Bug report -about: Create a report to help us improve -title: '' +about: Create a bug report to help us improve. +title: "bug: " labels: unconfirmed -assignees: '' - +assignees: [pawamoy] --- -**Please open an issue on [Griffe](https://github.com/mkdocstrings/griffe/issues) (new Python handler) -or [pytkdocs](https://github.com/mkdocstrings/pytkdocs/issues) (legacy Python handler) instead -if this is related to Python docstrings parsing or the collection of Python objects!** +### Description of the bug + + +### To Reproduce + + +``` +WRITE MRE / INSTRUCTIONS HERE +``` + +### Full traceback + + +
Full traceback + +```python +PASTE TRACEBACK HERE +``` -**Describe the bug** -A clear and concise description of what the bug is. +
-**To Reproduce** -Steps to reproduce the behavior: -1. Go to '...' -2. Click on '....' -3. Scroll down to '....' -4. See error +### Expected behavior + -**Expected behavior** -A clear and concise description of what you expected to happen. +### Environment information + -**Screenshots** -If applicable, add screenshots to help explain your problem. +```bash +python -m mkdocstrings.debug # | xclip -selection clipboard +``` -**Information (please complete the following information):** -- OS: [e.g. iOS] -- Browser: [e.g. chrome, safari] -- `mkdocstrings` version: [e.g. 0.10.2] +PASTE OUTPUT HERE -**Additional context** -Add any other context about the problem here. +### Additional context + diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 00000000..23000298 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,5 @@ +blank_issues_enabled: false +contact_links: +- name: I have a question / I need help + url: https://github.com/mkdocstrings/mkdocstrings/discussions/new?category=q-a + about: Ask and answer questions in the Discussions tab. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index 4fe86d5e..2df98fbc 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -1,20 +1,19 @@ --- name: Feature request -about: Suggest an idea for this project -title: '' +about: Suggest an idea for this project. +title: "feature: " labels: feature -assignees: '' - +assignees: pawamoy --- -**Is your feature request related to a problem? Please describe.** -A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] +### Is your feature request related to a problem? Please describe. + -**Describe the solution you'd like** -A clear and concise description of what you want to happen. +### Describe the solution you'd like + -**Describe alternatives you've considered** -A clear and concise description of any alternative solutions or features you've considered. +### Describe alternatives you've considered + -**Additional context** -Add any other context or screenshots about the feature request here. +### Additional context + diff --git a/CHANGELOG.md b/CHANGELOG.md index fc3a0fb0..c24f6724 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,23 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html). +## [0.24.0](https://github.com/mkdocstrings/mkdocstrings/releases/tag/0.24.0) - 2023-11-14 + +[Compare with 0.23.0](https://github.com/mkdocstrings/mkdocstrings/compare/0.23.0...0.24.0) + +### Features + +- Cache downloaded inventories as local file ([ce84dd5](https://github.com/mkdocstrings/mkdocstrings/commit/ce84dd57dc5cd3bf3f4be9623ddaa73e1c1868f0) by Oleh Prypin). [PR #632](https://github.com/mkdocstrings/mkdocstrings/pull/632) + +### Bug Fixes + +- Make `custom_templates` relative to the config file ([370a61d](https://github.com/mkdocstrings/mkdocstrings/commit/370a61d12b33f3fb61f6bddb3939eb8ff6018620) by Waylan Limberg). [Issue #477](https://github.com/mkdocstrings/mkdocstrings/issues/477), [PR #627](https://github.com/mkdocstrings/mkdocstrings/pull/627) +- Remove duplicated headings for docstrings nested in tabs/admonitions ([e2123a9](https://github.com/mkdocstrings/mkdocstrings/commit/e2123a935edea0abdc1b439e2c2b76e002c76e2b) by Perceval Wajsburt, [f4a94f7](https://github.com/mkdocstrings/mkdocstrings/commit/f4a94f7d8b8eb1ac01d65bb7237f0077e320ddac) by Oleh Prypin). [Issue #609](https://github.com/mkdocstrings/mkdocstrings/issues/609), [PR #610](https://github.com/mkdocstrings/mkdocstrings/pull/610), [PR #613](https://github.com/mkdocstrings/mkdocstrings/pull/613) + +### Code Refactoring + +- Drop support for MkDocs < 1.4, modernize usages ([b61d4d1](https://github.com/mkdocstrings/mkdocstrings/commit/b61d4d15258c66b14266aa04b456f191f101b2c6) by Oleh Prypin). [PR #629](https://github.com/mkdocstrings/mkdocstrings/pull/629) + ## [0.23.0](https://github.com/mkdocstrings/mkdocstrings/releases/tag/0.23.0) - 2023-08-28 [Compare with 0.22.0](https://github.com/mkdocstrings/mkdocstrings/compare/0.22.0...0.23.0) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0b86ff4b..ff84c305 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -44,8 +44,9 @@ on multiple Python versions, you run the task directly with `pdm run duty TASK`. The Makefile detects if a virtual environment is activated, so `make` will work the same with the virtualenv activated or not. -If you work in VSCode, -[see examples of tasks and run configurations](https://pawamoy.github.io/copier-pdm/work/#vscode-setup). +If you work in VSCode, we provide +[an action to configure VSCode](https://pawamoy.github.io/copier-pdm/work/#vscode-setup) +for the project. ## Development diff --git a/Makefile b/Makefile index 5696baac..f441a5c5 100644 --- a/Makefile +++ b/Makefile @@ -18,7 +18,8 @@ BASIC_DUTIES = \ docs \ docs-deploy \ format \ - release + release \ + vscode QUALITY_DUTIES = \ check-quality \ diff --git a/README.md b/README.md index 06872728..15d41d7a 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ [![pypi version](https://img.shields.io/pypi/v/mkdocstrings.svg)](https://pypi.org/project/mkdocstrings/) [![conda version](https://img.shields.io/conda/vn/conda-forge/mkdocstrings)](https://anaconda.org/conda-forge/mkdocstrings) [![gitpod](https://img.shields.io/badge/gitpod-workspace-blue.svg?style=flat)](https://gitpod.io/#https://github.com/mkdocstrings/mkdocstrings) -[![gitter](https://badges.gitter.im/join%20chat.svg)](https://gitter.im/mkdocstrings/community) +[![gitter](https://badges.gitter.im/join%20chat.svg)](https://app.gitter.im/#/room/#mkdocstrings:gitter.im) Automatic documentation from sources, for [MkDocs](https://mkdocs.org/). Come have a chat or ask questions on our [Gitter channel](https://gitter.im/mkdocstrings/community). @@ -23,7 +23,8 @@ Come have a chat or ask questions on our [Gitter channel](https://gitter.im/mkdo It means you can use it with any programming language, as long as there is a [**handler**](https://mkdocstrings.github.io/reference/handlers/base/) for it. We currently have [handlers](https://mkdocstrings.github.io/handlers/overview/) - for the [Crystal](https://mkdocstrings.github.io/crystal/) and [Python](https://mkdocstrings.github.io/python/) languages. + for the [Crystal](https://mkdocstrings.github.io/crystal/) and [Python](https://mkdocstrings.github.io/python/) languages, + as well as for [shell scripts/libraries](https://mkdocstrings.github.io/shell/). Maybe you'd like to add another one to the list? :wink: - [**Multiple themes support:**](https://mkdocstrings.github.io/theming/) @@ -55,9 +56,8 @@ Come have a chat or ask questions on our [Gitter channel](https://gitter.im/mkdo each handler can be configured globally in `mkdocs.yml`, and locally for each "autodoc" instruction. -- [**Watch source code directories:**](https://mkdocstrings.github.io/usage/#watch-directories) - you can tell *mkdocstrings* to add directories to be watched by *MkDocs* when - serving the documentation, for auto-reload. +- ~~**Watch source code directories:**~~ + this feature was removed as it is now [built in MkDocs](https://www.mkdocs.org/user-guide/configuration/#watch). - **Reasonable defaults:** you should be able to just drop the plugin in your configuration and enjoy your auto-generated docs. @@ -77,6 +77,7 @@ Come have a chat or ask questions on our [Gitter channel](https://gitter.im/mkdo ## Installation With `pip`: + ```bash pip install mkdocstrings ``` @@ -90,6 +91,7 @@ pip install 'mkdocstrings[crystal,python]' See the [available language handlers](https://mkdocstrings.github.io/handlers/overview/). With `conda`: + ```bash conda install -c conda-forge mkdocstrings ``` diff --git a/config/git-changelog.toml b/config/git-changelog.toml new file mode 100644 index 00000000..44e2b1fb --- /dev/null +++ b/config/git-changelog.toml @@ -0,0 +1,8 @@ +bump = "auto" +convention = "angular" +in-place = true +output = "CHANGELOG.md" +parse-refs = false +parse-trailers = true +sections = ["build", "deps", "feat", "fix", "refactor"] +template = "keepachangelog" diff --git a/config/ruff.toml b/config/ruff.toml index c6e4a55c..ad45b6c9 100644 --- a/config/ruff.toml +++ b/config/ruff.toml @@ -77,6 +77,9 @@ ignore = [ "src/*/cli.py" = [ "T201", # Print statement ] +"src/*/debug.py" = [ + "T201", # Print statement +] "scripts/*.py" = [ "INP001", # File is part of an implicit namespace package "T201", # Print statement diff --git a/config/vscode/launch.json b/config/vscode/launch.json new file mode 100644 index 00000000..2e0d651e --- /dev/null +++ b/config/vscode/launch.json @@ -0,0 +1,36 @@ +{ + "version": "0.2.0", + "configurations": [ + { + "name": "python (current file)", + "type": "python", + "request": "launch", + "program": "${file}", + "console": "integratedTerminal", + "justMyCode": false + }, + { + "name": "test", + "type": "python", + "request": "launch", + "module": "pytest", + "justMyCode": false, + "args": [ + "-c=config/pytest.ini", + "-vvv", + "--no-cov", + "--dist=no", + "tests", + "-k=${input:tests_selection}" + ] + } + ], + "inputs": [ + { + "id": "tests_selection", + "type": "promptString", + "description": "Tests selection", + "default": "" + } + ] +} \ No newline at end of file diff --git a/config/vscode/settings.json b/config/vscode/settings.json new file mode 100644 index 00000000..17beee4b --- /dev/null +++ b/config/vscode/settings.json @@ -0,0 +1,52 @@ +{ + "files.watcherExclude": { + "**/__pypackages__/**": true, + "**/.venv*/**": true, + "**/venv*/**": true + }, + "[python]": { + "editor.defaultFormatter": "ms-python.black-formatter" + }, + "python.autoComplete.extraPaths": [ + "__pypackages__/3.8/lib", + "__pypackages__/3.9/lib", + "__pypackages__/3.10/lib", + "__pypackages__/3.11/lib", + "__pypackages__/3.12/lib" + ], + "python.analysis.extraPaths": [ + "__pypackages__/3.8/lib", + "__pypackages__/3.9/lib", + "__pypackages__/3.10/lib", + "__pypackages__/3.11/lib", + "__pypackages__/3.12/lib" + ], + "black-formatter.args": [ + "--config=config/black.toml" + ], + "mypy-type-checker.args": [ + "--config-file=config/mypy.ini" + ], + "python.testing.unittestEnabled": false, + "python.testing.pytestEnabled": true, + "python.testing.pytestArgs": [ + "--config-file=config/pytest.ini" + ], + "ruff.format.args": [ + "--config=config/ruff.toml" + ], + "ruff.lint.args": [ + "--config=config/ruff.toml" + ], + "yaml.schemas": { + "https://squidfunk.github.io/mkdocs-material/schema.json": "mkdocs.yml" + }, + "yaml.customTags": [ + "!ENV scalar", + "!ENV sequence", + "!relative scalar", + "tag:yaml.org,2002:python/name:materialx.emoji.to_svg", + "tag:yaml.org,2002:python/name:materialx.emoji.twemoji", + "tag:yaml.org,2002:python/name:pymdownx.superfences.fence_code_format" + ] +} \ No newline at end of file diff --git a/config/vscode/tasks.json b/config/vscode/tasks.json new file mode 100644 index 00000000..80cd13d2 --- /dev/null +++ b/config/vscode/tasks.json @@ -0,0 +1,93 @@ +{ + "version": "2.0.0", + "tasks": [ + { + "label": "changelog", + "type": "shell", + "command": "pdm run duty changelog" + }, + { + "label": "check", + "type": "shell", + "command": "pdm run duty check" + }, + { + "label": "check-quality", + "type": "shell", + "command": "pdm run duty check-quality" + }, + { + "label": "check-types", + "type": "shell", + "command": "pdm run duty check-types" + }, + { + "label": "check-docs", + "type": "shell", + "command": "pdm run duty check-docs" + }, + { + "label": "check-dependencies", + "type": "shell", + "command": "pdm run duty check-dependencies" + }, + { + "label": "check-api", + "type": "shell", + "command": "pdm run duty check-api" + }, + { + "label": "clean", + "type": "shell", + "command": "pdm run duty clean" + }, + { + "label": "docs", + "type": "shell", + "command": "pdm run duty docs" + }, + { + "label": "docs-deploy", + "type": "shell", + "command": "pdm run duty docs-deploy" + }, + { + "label": "format", + "type": "shell", + "command": "pdm run duty format" + }, + { + "label": "lock", + "type": "shell", + "command": "pdm lock -G:all" + }, + { + "label": "release", + "type": "shell", + "command": "pdm run duty release ${input:version}" + }, + { + "label": "setup", + "type": "shell", + "command": "bash scripts/setup.sh" + }, + { + "label": "test", + "type": "shell", + "command": "pdm run duty test coverage", + "group": "test" + }, + { + "label": "vscode", + "type": "shell", + "command": "pdm run duty vscode" + } + ], + "inputs": [ + { + "id": "version", + "type": "promptString", + "description": "Version" + } + ] +} \ No newline at end of file diff --git a/docs/credits.md b/docs/credits.md index 9db45873..f758db87 100644 --- a/docs/credits.md +++ b/docs/credits.md @@ -3,6 +3,8 @@ hide: - toc --- + ```python exec="yes" --8<-- "scripts/gen_credits.py" ``` + diff --git a/docs/css/insiders.css b/docs/css/insiders.css index b5547bd1..e7b9c74f 100644 --- a/docs/css/insiders.css +++ b/docs/css/insiders.css @@ -53,11 +53,10 @@ a.insiders { } .sponsorship-item { - float: left; border-radius: 100%; - display: block; + display: inline-block; height: 1.6rem; - margin: .2rem; + margin: 0.1rem; overflow: hidden; width: 1.6rem; } diff --git a/docs/insiders/index.md b/docs/insiders/index.md index bfb2d428..99761b96 100644 --- a/docs/insiders/index.md +++ b/docs/insiders/index.md @@ -58,24 +58,28 @@ a handful of them, [thanks to our awesome sponsors][sponsors]! --> ```python exec="1" session="insiders" data_source = [ "docs/insiders/goals.yml", - ("mkdocstrings-python", "https://mkdocstrings.github.io/python/", "insiders/goals.yml"), ("griffe-pydantic", "https://mkdocstrings.github.io/griffe-pydantic/", "insiders/goals.yml"), ("griffe-typing-deprecated", "https://mkdocstrings.github.io/griffe-typing-deprecated/", "insiders/goals.yml"), + ("mkdocstrings-python", "https://mkdocstrings.github.io/python/", "insiders/goals.yml"), + ("mkdocstrings-shell", "https://mkdocstrings.github.io/shell/", "insiders/goals.yml"), ] ``` + ```python exec="1" session="insiders" --8<-- "scripts/insiders.py" ``` -```python exec="1" session="insiders" -print(f"""The moment you become a sponsor, you'll get **immediate -access to {len(unreleased_features)} additional features** that you can start using right away, and -which are currently exclusively available to sponsors:\n""") +print( + f"""The moment you become a sponsor, you'll get **immediate + access to {len(unreleased_features)} additional features** that you can start using right away, and + which are currently exclusively available to sponsors:\n""" +) for feature in unreleased_features: feature.render(badge=True) ``` + ## How to become a sponsor @@ -126,9 +130,6 @@ You can cancel your sponsorship anytime.[^5]
-
-
- If you sponsor publicly, you're automatically added here with a link to your profile and avatar to show your support for *mkdocstrings*. diff --git a/docs/recipes.md b/docs/recipes.md index c33130f0..8ea849fc 100644 --- a/docs/recipes.md +++ b/docs/recipes.md @@ -17,15 +17,15 @@ Let say you have a project called `project`. This project has a lot of source files, or modules, which live in the `src` folder: -``` -πŸ“ repo -β””β”€β•΄πŸ“ src - β””β”€β•΄πŸ“ project - β”œβ”€β•΄πŸ“„ lorem - β”œβ”€β•΄πŸ“„ ipsum - β”œβ”€β•΄πŸ“„ dolor - β”œβ”€β•΄πŸ“„ sit - β””β”€β•΄πŸ“„ amet +```tree +repo/ + src/ + project/ + lorem + ipsum + dolor + sit + amet ``` Without an automatic process, you will have to manually @@ -49,10 +49,10 @@ and configure it like so: ```yaml title="mkdocs.yml" plugins: -- search # (1) +- search # (1)! - gen-files: scripts: - - docs/gen_ref_pages.py # (2) + - scripts/gen_ref_pages.py # (2)! - mkdocstrings ``` @@ -60,76 +60,91 @@ plugins: 2. The magic happens here, see below how it works. mkdocs-gen-files is able to run Python scripts at build time. -The Python script that we will execute lives in the docs folder, +The Python script that we will execute lives in a scripts folder, and is named `gen_ref_pages.py`, like "generate code reference pages". -```python title="docs/gen_ref_pages.py" +```tree +repo/ + docs/ + index.md + scripts/ + gen_ref_pages.py + src/ + project/ + mkdocs.yml +``` + +```python title="scripts/gen_ref_pages.py" """Generate the code reference pages.""" from pathlib import Path import mkdocs_gen_files -for path in sorted(Path("src").rglob("*.py")): # (1) - module_path = path.relative_to("src").with_suffix("") # (2) - doc_path = path.relative_to("src").with_suffix(".md") # (3) - full_doc_path = Path("reference", doc_path) # (4) +src = Path(__file__).parent.parent / "src" # (1)! + +for path in sorted(src.rglob("*.py")): # (2)! + module_path = path.relative_to(src).with_suffix("") # (3)! + doc_path = path.relative_to(src).with_suffix(".md") # (4)! + full_doc_path = Path("reference", doc_path) # (5)! parts = list(module_path.parts) - if parts[-1] == "__init__": # (5) + if parts[-1] == "__init__": # (6)! parts = parts[:-1] elif parts[-1] == "__main__": continue - with mkdocs_gen_files.open(full_doc_path, "w") as fd: # (6) - identifier = ".".join(parts) # (7) - print("::: " + identifier, file=fd) # (8) + with mkdocs_gen_files.open(full_doc_path, "w") as fd: # (7)! + identifier = ".".join(parts) # (8)! + print("::: " + identifier, file=fd) # (9)! - mkdocs_gen_files.set_edit_path(full_doc_path, path) # (9) + mkdocs_gen_files.set_edit_path(full_doc_path, path) # (10)! ``` -1. Here we recursively list all `.py` files, but you can adapt the code to list +1. It's important to build a path relative to the script itself, + to make it possible to build the docs with MkDocs' + [`-f` option](https://www.mkdocs.org/user-guide/cli/#mkdocs-build). +2. Here we recursively list all `.py` files, but you can adapt the code to list files with other extensions of course, supporting other languages than Python. -2. The module path will look like `project/lorem`. +3. The module path will look like `project/lorem`. It will be used to build the *mkdocstrings* autodoc identifier. -3. This is the relative path to the Markdown page. -4. This is the absolute path to the Markdown page. Here we put all reference pages - into a `reference` folder. -5. This part is only relevant for Python modules. We skip `__main__` modules and +4. This is the partial path of the Markdown page for the module. +5. This is the full path of the Markdown page within the docs. + Here we put all reference pages into a `reference` folder. +6. This part is only relevant for Python modules. We skip `__main__` modules and remove `__init__` from the module parts as it's implicit during imports. -6. Magic! Add the file to MkDocs pages, without actually writing it in the docs folder. -7. Build the autodoc identifier. Here we document Python modules, so the identifier +7. Magic! Add the file to MkDocs pages, without actually writing it in the docs folder. +8. Build the autodoc identifier. Here we document Python modules, so the identifier is a dot-separated path, like `project.lorem`. -8. Actually write to the magic file. -9. We can even set the `edit_uri` on the pages. +9. Actually write to the magic file. +10. We can even set the `edit_uri` on the pages. > NOTE: > It is important to look out for correct edit page behaviour when using generated pages. > For example, if we have `edit_uri` set to `blob/master/docs/` and the following > file structure: > -> ``` -> πŸ“ repo -> β”œβ”€ πŸ“„ mkdocs.yml -> β”‚ -> β”œβ”€ πŸ“ docs -> β”‚ β”œβ”€β•΄πŸ“„ index.md -> β”‚ β””β”€β•΄πŸ“„ gen_ref_pages.py -> β”‚ -> β””β”€β•΄πŸ“ src -> β””β”€β•΄πŸ“ project -> β”œβ”€β•΄πŸ“„ lorem.py -> β”œβ”€β•΄πŸ“„ ipsum.py -> β”œβ”€β•΄πŸ“„ dolor.py -> β”œβ”€β•΄πŸ“„ sit.py -> β””β”€β•΄πŸ“„ amet.py +> ```tree +> repo/ +> mkdocs.yml +> docs/ +> index.md +> scripts/ +> gen_ref_pages.py +> src/ +> project/ +> lorem.py +> ipsum.py +> dolor.py +> sit.py +> amet.py > ``` > > Then we will have to change our `set_edit_path` call to: > > ```python -> mkdocs_gen_files.set_edit_path(full_doc_path, Path("../") / path) # (1) +> mkdocs_gen_files.set_edit_path(full_doc_path, Path("../") / path) # (1)! > ``` > > 1. Path can be used to traverse the structure in any way you may need, but @@ -180,7 +195,7 @@ plugins: - search - gen-files: scripts: - - docs/gen_ref_pages.py + - scripts/gen_ref_pages.py - literate-nav: nav_file: SUMMARY.md - mkdocstrings @@ -188,7 +203,7 @@ plugins: Then, the previous script is updated like so: -```python title="docs/gen_ref_pages.py" hl_lines="7 21 29 30" +```python title="scripts/gen_ref_pages.py" hl_lines="7 23 31 32" """Generate the code reference pages and navigation.""" from pathlib import Path @@ -197,9 +212,11 @@ import mkdocs_gen_files nav = mkdocs_gen_files.Nav() -for path in sorted(Path("src").rglob("*.py")): - module_path = path.relative_to("src").with_suffix("") - doc_path = path.relative_to("src").with_suffix(".md") +src = Path(__file__).parent.parent / "src" + +for path in sorted(src.rglob("*.py")): + module_path = path.relative_to(src).with_suffix("") + doc_path = path.relative_to(src).with_suffix(".md") full_doc_path = Path("reference", doc_path) parts = tuple(module_path.parts) @@ -209,7 +226,7 @@ for path in sorted(Path("src").rglob("*.py")): elif parts[-1] == "__main__": continue - nav[parts] = doc_path.as_posix() # (1) + nav[parts] = doc_path.as_posix() # (1)! with mkdocs_gen_files.open(full_doc_path, "w") as fd: ident = ".".join(parts) @@ -217,8 +234,8 @@ for path in sorted(Path("src").rglob("*.py")): mkdocs_gen_files.set_edit_path(full_doc_path, path) -with mkdocs_gen_files.open("reference/SUMMARY.md", "w") as nav_file: # (2) - nav_file.writelines(nav.build_literate_nav()) # (3) +with mkdocs_gen_files.open("reference/SUMMARY.md", "w") as nav_file: # (2)! + nav_file.writelines(nav.build_literate_nav()) # (3)! ``` 1. Progressively build the navigation object. @@ -232,7 +249,7 @@ and replace it with a single line! nav: # rest of the navigation... # defer to gen-files + literate-nav -- Code Reference: reference/ # (1) +- Code Reference: reference/ # (1)! # rest of the navigation... ``` @@ -259,7 +276,7 @@ Well, this is possible thanks to a third plugin: Update the script like this: -```python title="docs/gen_ref_pages.py" hl_lines="18 19" +```python title="scripts/gen_ref_pages.py" hl_lines="20 21" """Generate the code reference pages and navigation.""" from pathlib import Path @@ -268,9 +285,11 @@ import mkdocs_gen_files nav = mkdocs_gen_files.Nav() -for path in sorted(Path("src").rglob("*.py")): - module_path = path.relative_to("src").with_suffix("") - doc_path = path.relative_to("src").with_suffix(".md") +src = Path(__file__).parent.parent / "src" + +for path in sorted(src.rglob("*.py")): + module_path = path.relative_to(src).with_suffix("") + doc_path = path.relative_to(src).with_suffix(".md") full_doc_path = Path("reference", doc_path) parts = tuple(module_path.parts) @@ -301,7 +320,7 @@ plugins: - search - gen-files: scripts: - - docs/gen_ref_pages.py + - scripts/gen_ref_pages.py - literate-nav: nav_file: SUMMARY.md - section-index diff --git a/docs/usage/handlers.md b/docs/usage/handlers.md index d2c25420..bd7e5823 100644 --- a/docs/usage/handlers.md +++ b/docs/usage/handlers.md @@ -7,6 +7,7 @@ A handler is what makes it possible to collect and render documentation for a pa - Crystal - Python - Python (Legacy) +- Shell ## About the Python handlers diff --git a/docs/usage/index.md b/docs/usage/index.md index 7599f9f1..1348b9cc 100644 --- a/docs/usage/index.md +++ b/docs/usage/index.md @@ -108,7 +108,7 @@ The above is equivalent to: - `default_handler`: The handler that is used by default when no handler is specified. - `custom_templates`: The path to a directory containing custom templates. - The path is relative to the current working directory. + The path is relative to the MkDocs configuration file. See [Theming](theming.md). - `handlers`: The handlers' global configuration. - `enable_inventory`: Whether to enable inventory file generation. diff --git a/docs/usage/theming.md b/docs/usage/theming.md index 73b7e0b3..b5d6f7b3 100644 --- a/docs/usage/theming.md +++ b/docs/usage/theming.md @@ -17,9 +17,9 @@ so you can tweak the look and feel with extra CSS rules. ### Templates -To use custom templates and override the theme ones, -specify the relative path to your templates directory -with the `custom_templates` global configuration option: +To use custom templates and override the theme ones, specify the relative path from your +configuration file to your templates directory with the `custom_templates` global +configuration option: ```yaml title="mkdocs.yml" plugins: @@ -82,7 +82,7 @@ Since each handler provides its own set of templates, with their own CSS classes we cannot list them all here. See the documentation about CSS classes for: - the Crystal handler: https://mkdocstrings.github.io/crystal/styling.html#custom-styles -- the Python handler: https://mkdocstrings.github.io/python/customization/#css-classes +- the Python handler: https://mkdocstrings.github.io/python/usage/customization/#css-classes ### Syntax highlighting diff --git a/duties.py b/duties.py index 644b2ffb..43ae357a 100644 --- a/duties.py +++ b/duties.py @@ -4,21 +4,18 @@ import os import sys +from contextlib import contextmanager +from importlib.metadata import version as pkgversion from pathlib import Path -from typing import TYPE_CHECKING, Any +from typing import TYPE_CHECKING, Iterator from duty import duty -from duty.callables import black, blacken_docs, coverage, lazy, mkdocs, mypy, pytest, ruff, safety - -if sys.version_info < (3, 8): - from importlib_metadata import version as pkgversion -else: - from importlib.metadata import version as pkgversion - +from duty.callables import black, coverage, lazy, mkdocs, mypy, pytest, ruff, safety if TYPE_CHECKING: from duty.context import Context + PY_SRC_PATHS = (Path(_) for _ in ("src", "tests", "duties.py", "scripts")) PY_SRC_LIST = tuple(str(_) for _ in PY_SRC_PATHS) PY_SRC = " ".join(PY_SRC_LIST) @@ -35,32 +32,16 @@ def pyprefix(title: str) -> str: # noqa: D103 return title -def merge(d1: Any, d2: Any) -> Any: # noqa: D103 - basic_types = (int, float, str, bool, complex) - if isinstance(d1, dict) and isinstance(d2, dict): - for key, value in d2.items(): - if key in d1: - if isinstance(d1[key], basic_types): - d1[key] = value - else: - d1[key] = merge(d1[key], value) - else: - d1[key] = value - return d1 - if isinstance(d1, list) and isinstance(d2, list): - return d1 + d2 - return d2 - - -def mkdocs_config() -> str: # noqa: D103 - import mergedeep - - # force YAML loader to merge arrays - mergedeep.merge = merge - +@contextmanager +def material_insiders() -> Iterator[bool]: # noqa: D103 if "+insiders" in pkgversion("mkdocs-material"): - return "mkdocs.insiders.yml" - return "mkdocs.yml" + os.environ["MATERIAL_INSIDERS"] = "true" + try: + yield True + finally: + os.environ.pop("MATERIAL_INSIDERS") + else: + yield False @duty @@ -70,23 +51,9 @@ def changelog(ctx: Context) -> None: Parameters: ctx: The context instance (passed automatically). """ - from git_changelog.cli import build_and_render + from git_changelog.cli import main as git_changelog - git_changelog = lazy(build_and_render, name="git_changelog") - ctx.run( - git_changelog( - repository=".", - output="CHANGELOG.md", - convention="angular", - template="keepachangelog", - parse_trailers=True, - parse_refs=False, - sections=["build", "deps", "feat", "fix", "refactor"], - bump_latest=True, - in_place=True, - ), - title="Updating changelog", - ) + ctx.run(git_changelog, args=[[]], title="Updating changelog") @duty(pre=["check_quality", "check_types", "check_docs", "check_dependencies", "check-api"]) @@ -142,12 +109,12 @@ def check_docs(ctx: Context) -> None: """ Path("htmlcov").mkdir(parents=True, exist_ok=True) Path("htmlcov/index.html").touch(exist_ok=True) - config = mkdocs_config() - ctx.run( - mkdocs.build(strict=True, config_file=config, verbose=True), - title=pyprefix("Building documentation"), - command=f"mkdocs build -vsf {config}", - ) + with material_insiders(): + ctx.run( + mkdocs.build(strict=True, verbose=True), + title=pyprefix("Building documentation"), + command="mkdocs build -vs", + ) @duty @@ -212,11 +179,12 @@ def docs(ctx: Context, host: str = "127.0.0.1", port: int = 8000) -> None: host: The host to serve the docs from. port: The port to serve the docs on. """ - ctx.run( - mkdocs.serve(dev_addr=f"{host}:{port}", config_file=mkdocs_config()), - title="Serving documentation", - capture=False, - ) + with material_insiders(): + ctx.run( + mkdocs.serve(dev_addr=f"{host}:{port}"), + title="Serving documentation", + capture=False, + ) @duty @@ -227,22 +195,26 @@ def docs_deploy(ctx: Context) -> None: ctx: The context instance (passed automatically). """ os.environ["DEPLOY"] = "true" - config_file = mkdocs_config() - if config_file == "mkdocs.yml": - ctx.run(lambda: False, title="Not deploying docs without Material for MkDocs Insiders!") - origin = ctx.run("git config --get remote.origin.url", silent=True) - if "pawamoy-insiders/mkdocstrings" in origin: - ctx.run("git remote add org-pages git@github.com:mkdocstrings/mkdocstrings.github.io", silent=True, nofail=True) - ctx.run( - mkdocs.gh_deploy(config_file=config_file, remote_name="org-pages", force=True), - title="Deploying documentation", - ) - else: - ctx.run( - lambda: False, - title="Not deploying docs from public repository (do that from insiders instead!)", - nofail=True, - ) + with material_insiders() as insiders: + if not insiders: + ctx.run(lambda: False, title="Not deploying docs without Material for MkDocs Insiders!") + origin = ctx.run("git config --get remote.origin.url", silent=True) + if "pawamoy-insiders/mkdocstrings" in origin: + ctx.run( + "git remote add org-pages git@github.com:mkdocstrings/mkdocstrings.github.io", + silent=True, + nofail=True, + ) + ctx.run( + mkdocs.gh_deploy(remote_name="upstream", force=True), + title="Deploying documentation", + ) + else: + ctx.run( + lambda: False, + title="Not deploying docs from public repository (do that from insiders instead!)", + nofail=True, + ) @duty @@ -257,11 +229,6 @@ def format(ctx: Context) -> None: title="Auto-fixing code", ) ctx.run(black.run(*PY_SRC_LIST, config="config/black.toml"), title="Formatting code") - ctx.run( - blacken_docs.run(*PY_SRC_LIST, "docs", exts=["py", "md"], line_length=120), - title="Formatting docs", - nofail=True, - ) @duty(post=["docs-deploy"]) @@ -314,3 +281,28 @@ def test(ctx: Context, match: str = "") -> None: title=pyprefix("Running tests"), command=f"pytest -c config/pytest.ini -n auto -k{match!r} --color=yes tests", ) + + +@duty +def vscode(ctx: Context) -> None: + """Configure VSCode. + + This task will overwrite the following files, + so make sure to back them up: + + - `.vscode/launch.json` + - `.vscode/settings.json` + - `.vscode/tasks.json` + + Parameters: + ctx: The context instance (passed automatically). + """ + + def update_config(filename: str) -> None: + source_file = Path("config", "vscode", filename) + target_file = Path(".vscode", filename) + target_file.parent.mkdir(exist_ok=True) + target_file.write_text(source_file.read_text()) + + for filename in ("launch.json", "settings.json", "tasks.json"): + ctx.run(update_config, args=[filename], title=f"Update .vscode/{filename}") diff --git a/mkdocs.insiders.yml b/mkdocs.insiders.yml deleted file mode 100644 index a93edcc3..00000000 --- a/mkdocs.insiders.yml +++ /dev/null @@ -1,5 +0,0 @@ -INHERIT: mkdocs.yml - -# waiting for https://github.com/squidfunk/mkdocs-material/issues/5446 -# plugins: -# - typeset diff --git a/mkdocs.yml b/mkdocs.yml index fdd6898b..37a98cf4 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -27,6 +27,7 @@ nav: - Crystal: https://mkdocstrings.github.io/crystal/ - Python: https://mkdocstrings.github.io/python/ - Python (Legacy): https://mkdocstrings.github.io/python-legacy/ + - Shell: https://mkdocstrings.github.io/shell/ - Guides: - Recipes: recipes.md - Troubleshooting: troubleshooting.md @@ -95,12 +96,13 @@ markdown_extensions: - admonition - callouts - footnotes -- pymdownx.emoji: - emoji_index: !!python/name:materialx.emoji.twemoji - emoji_generator: !!python/name:materialx.emoji.to_svg - pymdownx.details +- pymdownx.emoji: + emoji_index: !!python/name:material.extensions.emoji.twemoji + emoji_generator: !!python/name:material.extensions.emoji.to_svg - pymdownx.magiclink - pymdownx.snippets: + base_path: [!relative $config_dir] check_paths: true - pymdownx.superfences - pymdownx.tabbed: @@ -110,6 +112,7 @@ markdown_extensions: case: lower - pymdownx.tasklist: custom_checkbox: true +- pymdownx.tilde - toc: permalink: "Β€" @@ -120,7 +123,7 @@ plugins: scripts: - scripts/gen_ref_nav.py - literate-nav: - nav_file: SUMMARY.txt + nav_file: SUMMARY.md - coverage - mkdocstrings: handlers: @@ -129,11 +132,14 @@ plugins: - https://docs.python.org/3/objects.inv - https://installer.readthedocs.io/en/stable/objects.inv # demonstration purpose in the docs - https://mkdocstrings.github.io/autorefs/objects.inv + paths: [src] options: docstring_options: ignore_init_summary: true docstring_section_style: list + filters: ["!^_"] heading_level: 1 + inherited_members: true merge_init_into_class: true separate_signature: true show_root_heading: true @@ -152,6 +158,10 @@ plugins: handlers/overview.md: usage/handlers.md - minify: minify_html: !ENV [DEPLOY, false] +- group: + enabled: !ENV [MATERIAL_INSIDERS, false] + plugins: + - typeset extra: social: diff --git a/pyproject.toml b/pyproject.toml index 78af6bff..1be96db8 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -29,11 +29,13 @@ classifiers = [ "Typing :: Typed", ] dependencies = [ + "click>=7.0", "Jinja2>=2.11.1", "Markdown>=3.3", "MarkupSafe>=1.1", - "mkdocs>=1.2", + "mkdocs>=1.4", "mkdocs-autorefs>=0.3.1", + "platformdirs>=2.2.0", "pymdown-extensions>=6.3", "importlib-metadata>=4.6; python_version < '3.10'", "typing-extensions>=4.1; python_version < '3.10'", @@ -72,44 +74,43 @@ duty = ["duty>=0.10"] ci-quality = ["mkdocstrings[duty,docs,quality,typing,security]"] ci-tests = ["mkdocstrings[duty,docs,tests]"] docs = [ - "black>=23.1", - "markdown-callouts>=0.2", - "markdown-exec>=0.5", + "black>=23.9", + "markdown-callouts>=0.3", + "markdown-exec>=1.7", "mkdocs>=1.5", - "mkdocs-coverage>=0.2", - "mkdocs-gen-files>=0.3", - "mkdocs-git-committers-plugin-2>=1.1", - "mkdocs-literate-nav>=0.4", - "mkdocs-material>=7.3", - "mkdocs-minify-plugin>=0.6.4", - "mkdocs-redirects>=1.2.0", - "mkdocstrings-python>=0.5.1", - "toml>=0.10", + "mkdocs-coverage>=1.0", + "mkdocs-gen-files>=0.5", + "mkdocs-git-committers-plugin-2>=1.2", + "mkdocs-literate-nav>=0.6", + "mkdocs-material>=9.4", + "mkdocs-minify-plugin>=0.7", + "mkdocs-redirects>=1.2", + "mkdocstrings-python>=1.7", + "tomli>=2.0; python_version < '3.11'", ] maintain = [ - "black>=23.1", - "blacken-docs>=1.13", - "git-changelog>=1.0", + "black>=23.9", + "blacken-docs>=1.16", + "git-changelog>=2.3", ] quality = [ - "ruff>=0.0.246", + "ruff>=0.0", ] tests = [ "docutils", "pygments>=2.10", # python 3.6 - "pytest>=6.2", - "pytest-cov>=3.0", - "pytest-randomly>=3.10", - "pytest-xdist>=2.4", + "pytest>=7.4", + "pytest-cov>=4.1", + "pytest-randomly>=3.15", + "pytest-xdist>=3.3", "sphinx", ] typing = [ - "mypy>=0.911", - "types-docutils", - "types-markdown>=3.3", + "mypy>=1.5", + "types-docutils>=0.20,", + "types-markdown>=3.5", "types-pyyaml>=6.0", - "types-toml>=0.10", ] security = [ - "safety>=2", + "safety>=2.3", ] diff --git a/scripts/gen_credits.py b/scripts/gen_credits.py index bc01c0bd..bf35f0da 100644 --- a/scripts/gen_credits.py +++ b/scripts/gen_credits.py @@ -2,27 +2,31 @@ from __future__ import annotations +import os import re import sys +from importlib.metadata import PackageNotFoundError, metadata from itertools import chain from pathlib import Path from textwrap import dedent from typing import Mapping, cast -import toml from jinja2 import StrictUndefined from jinja2.sandbox import SandboxedEnvironment -if sys.version_info < (3, 8): - from importlib_metadata import PackageNotFoundError, metadata +# TODO: Remove once support for Python 3.10 is dropped. +if sys.version_info >= (3, 11): + import tomllib else: - from importlib.metadata import PackageNotFoundError, metadata + import tomli as tomllib -project_dir = Path(".") -pyproject = toml.load(project_dir / "pyproject.toml") +project_dir = Path(os.getenv("MKDOCS_CONFIG_DIR", ".")) +with project_dir.joinpath("pyproject.toml").open("rb") as pyproject_file: + pyproject = tomllib.load(pyproject_file) project = pyproject["project"] pdm = pyproject["tool"]["pdm"] -lock_data = toml.load(project_dir / "pdm.lock") +with project_dir.joinpath("pdm.lock").open("rb") as lock_file: + lock_data = tomllib.load(lock_file) lock_pkgs = {pkg["name"].lower(): pkg for pkg in lock_data["package"]} project_name = project["name"] regex = re.compile(r"(?P[\w.-]+)(?P.*)$") @@ -35,7 +39,7 @@ def _get_license(pkg_name: str) -> str: return "?" license_name = cast(dict, data).get("License", "").strip() multiple_lines = bool(license_name.count("\n")) - # TODO: remove author logic once all my packages licenses are fixed + # TODO: Remove author logic once all my packages licenses are fixed. author = "" if multiple_lines or not license_name or license_name == "UNKNOWN": for header, value in cast(dict, data).items(): diff --git a/scripts/gen_ref_nav.py b/scripts/gen_ref_nav.py index 249530b1..9c041ede 100644 --- a/scripts/gen_ref_nav.py +++ b/scripts/gen_ref_nav.py @@ -7,9 +7,11 @@ nav = mkdocs_gen_files.Nav() mod_symbol = '' -for path in sorted(Path("src").rglob("*.py")): - module_path = path.relative_to("src").with_suffix("") - doc_path = path.relative_to("src/mkdocstrings").with_suffix(".md") +src = Path(__file__).parent.parent / "src" + +for path in sorted(src.rglob("*.py")): + module_path = path.relative_to(src).with_suffix("") + doc_path = path.relative_to(src / "mkdocstrings").with_suffix(".md") full_doc_path = Path("reference", doc_path) parts = tuple(module_path.parts) @@ -30,5 +32,5 @@ mkdocs_gen_files.set_edit_path(full_doc_path, ".." / path) -with mkdocs_gen_files.open("reference/SUMMARY.txt", "w") as nav_file: +with mkdocs_gen_files.open("reference/SUMMARY.md", "w") as nav_file: nav_file.writelines(nav.build_literate_nav()) diff --git a/scripts/insiders.py b/scripts/insiders.py index 6f8d0d84..8f5e215e 100644 --- a/scripts/insiders.py +++ b/scripts/insiders.py @@ -4,6 +4,7 @@ import json import logging +import os import posixpath from dataclasses import dataclass from datetime import date, datetime, timedelta @@ -39,11 +40,13 @@ class Feature: """Class representing an Insiders feature.""" name: str - ref: str + ref: str | None since: date | None project: Project | None - def url(self, rel_base: str = "..") -> str: # noqa: D102 + def url(self, rel_base: str = "..") -> str | None: # noqa: D102 + if not self.ref: + return None if self.project: rel_base = self.project.url return posixpath.join(rel_base, self.ref.lstrip("/")) @@ -56,7 +59,8 @@ def render(self, rel_base: str = "..", *, badge: bool = False) -> None: # noqa: ft_date = self.since.strftime("%B %d, %Y") # type: ignore[union-attr] new = f' :material-alert-decagram:{{ .new-feature .vibrate title="Added on {ft_date}" }}' project = f"[{self.project.name}]({self.project.url}) β€” " if self.project else "" - print(f"- [{'x' if self.since else ' '}] {project}[{self.name}]({self.url(rel_base)}){new}") + feature = f"[{self.name}]({self.url(rel_base)})" if self.ref else self.name + print(f"- [{'x' if self.since else ' '}] {project}{feature}{new}") @dataclass @@ -99,7 +103,7 @@ def load_goals(data: str, funding: int = 0, project: Project | None = None) -> d features=[ Feature( name=feature_data["name"], - ref=feature_data["ref"], + ref=feature_data.get("ref"), since=feature_data.get("since") and datetime.strptime(feature_data["since"], "%Y/%m/%d").date(), # noqa: DTZ007 project=project, @@ -112,8 +116,9 @@ def load_goals(data: str, funding: int = 0, project: Project | None = None) -> d def _load_goals_from_disk(path: str, funding: int = 0) -> dict[int, Goal]: + project_dir = os.getenv("MKDOCS_CONFIG_DIR", ".") try: - data = Path(path).read_text() + data = Path(project_dir, path).read_text() except OSError as error: raise RuntimeError(f"Could not load data from disk: {path}") from error return load_goals(data, funding) @@ -156,7 +161,7 @@ def funding_goals(source: str | list[str | tuple[str, str, str]], funding: int = goals[amount] = goal else: goals[amount].features.extend(goal.features) - return goals + return {amount: goals[amount] for amount in sorted(goals)} def feature_list(goals: Iterable[Goal]) -> list[Feature]: diff --git a/src/mkdocstrings/_cache.py b/src/mkdocstrings/_cache.py new file mode 100644 index 00000000..8737f317 --- /dev/null +++ b/src/mkdocstrings/_cache.py @@ -0,0 +1,76 @@ +import datetime +import gzip +import hashlib +import os +import urllib.parse +import urllib.request +from typing import BinaryIO, Callable + +import click +import platformdirs + +from mkdocstrings.loggers import get_logger + +log = get_logger(__name__) + + +def download_url_with_gz(url: str) -> bytes: + req = urllib.request.Request( # noqa: S310 + url, + headers={"Accept-Encoding": "gzip", "User-Agent": "mkdocstrings/0.15.0"}, + ) + with urllib.request.urlopen(req) as resp: # noqa: S310 + content: BinaryIO = resp + if "gzip" in resp.headers.get("content-encoding", ""): + content = gzip.GzipFile(fileobj=resp) # type: ignore[assignment] + return content.read() + + +# This is mostly a copy of https://github.com/mkdocs/mkdocs/blob/master/mkdocs/utils/cache.py +# In the future maybe they can be deduplicated. + + +def download_and_cache_url( + url: str, + download: Callable[[str], bytes], + cache_duration: datetime.timedelta, + comment: bytes = b"# ", +) -> bytes: + """Downloads a file from the URL, stores it under ~/.cache/, and returns its content. + + For tracking the age of the content, a prefix is inserted into the stored file, rather than relying on mtime. + + Args: + url: URL to use. + download: Callback that will accept the URL and actually perform the download. + cache_duration: How long to consider the URL content cached. + comment: The appropriate comment prefix for this file format. + """ + directory = os.path.join(platformdirs.user_cache_dir("mkdocs"), "mkdocstrings_url_cache") + name_hash = hashlib.sha256(url.encode()).hexdigest()[:32] + path = os.path.join(directory, name_hash + os.path.splitext(url)[1]) + + now = int(datetime.datetime.now(datetime.timezone.utc).timestamp()) + prefix = b"%s%s downloaded at timestamp " % (comment, url.encode()) + # Check for cached file and try to return it + if os.path.isfile(path): + try: + with open(path, "rb") as f: + line = f.readline() + if line.startswith(prefix): + line = line[len(prefix) :] + timestamp = int(line) + if datetime.timedelta(seconds=(now - timestamp)) <= cache_duration: + log.debug(f"Using cached '{path}' for '{url}'") + return f.read() + except (OSError, ValueError) as e: + log.debug(f"{type(e).__name__}: {e}") + + # Download and cache the file + log.debug(f"Downloading '{url}' to '{path}'") + content = download(url) + os.makedirs(directory, exist_ok=True) + with click.open_file(path, "wb", atomic=True) as f: + f.write(b"%s%d\n" % (prefix, now)) + f.write(content) + return content diff --git a/src/mkdocstrings/debug.py b/src/mkdocstrings/debug.py new file mode 100644 index 00000000..35ede743 --- /dev/null +++ b/src/mkdocstrings/debug.py @@ -0,0 +1,106 @@ +"""Debugging utilities.""" + +from __future__ import annotations + +import os +import platform +import sys +from dataclasses import dataclass +from importlib import metadata + + +@dataclass +class Variable: + """Dataclass describing an environment variable.""" + + name: str + """Variable name.""" + value: str + """Variable value.""" + + +@dataclass +class Package: + """Dataclass describing a Python package.""" + + name: str + """Package name.""" + version: str + """Package version.""" + + +@dataclass +class Environment: + """Dataclass to store environment information.""" + + interpreter_name: str + """Python interpreter name.""" + interpreter_version: str + """Python interpreter version.""" + platform: str + """Operating System.""" + packages: list[Package] + """Installed packages.""" + variables: list[Variable] + """Environment variables.""" + + +def _interpreter_name_version() -> tuple[str, str]: + if hasattr(sys, "implementation"): + impl = sys.implementation.version + version = f"{impl.major}.{impl.minor}.{impl.micro}" + kind = impl.releaselevel + if kind != "final": + version += kind[0] + str(impl.serial) + return sys.implementation.name, version + return "", "0.0.0" + + +def get_version(dist: str = "mkdocstrings") -> str: + """Get version of the given distribution. + + Parameters: + dist: A distribution name. + + Returns: + A version number. + """ + try: + return metadata.version(dist) + except metadata.PackageNotFoundError: + return "0.0.0" + + +def get_debug_info() -> Environment: + """Get debug/environment information. + + Returns: + Environment information. + """ + py_name, py_version = _interpreter_name_version() + packages = ["mkdocstrings"] + variables = ["PYTHONPATH", *[var for var in os.environ if var.startswith("MKDOCSTRINGS")]] + return Environment( + interpreter_name=py_name, + interpreter_version=py_version, + platform=platform.platform(), + variables=[Variable(var, val) for var in variables if (val := os.getenv(var))], + packages=[Package(pkg, get_version(pkg)) for pkg in packages], + ) + + +def print_debug_info() -> None: + """Print debug/environment information.""" + info = get_debug_info() + print(f"- __System__: {info.platform}") + print(f"- __Python__: {info.interpreter_name} {info.interpreter_version}") + print("- __Environment variables__:") + for var in info.variables: + print(f" - `{var.name}`: `{var.value}`") + print("- __Installed packages__:") + for pkg in info.packages: + print(f" - `{pkg.name}` v{pkg.version}") + + +if __name__ == "__main__": + print_debug_info() diff --git a/src/mkdocstrings/extension.py b/src/mkdocstrings/extension.py index 139030ba..a819f14b 100644 --- a/src/mkdocstrings/extension.py +++ b/src/mkdocstrings/extension.py @@ -72,8 +72,7 @@ def __init__( Arguments: parser: A `markdown.blockparser.BlockParser` instance. md: A `markdown.Markdown` instance. - config: The [configuration][mkdocstrings.plugin.MkdocstringsPlugin.config_scheme] - of the `mkdocstrings` plugin. + config: The [configuration][mkdocstrings.plugin.PluginConfig] of the `mkdocstrings` plugin. handlers: The handlers container. autorefs: The autorefs plugin instance. """ @@ -231,17 +230,23 @@ def _process_block( class _PostProcessor(Treeprocessor): def run(self, root: Element) -> None: + self._remove_duplicated_headings(root) + + def _remove_duplicated_headings(self, parent: Element) -> None: carry_text = "" - for el in reversed(root): # Reversed mainly for the ability to mutate during iteration. + for el in reversed(parent): # Reversed mainly for the ability to mutate during iteration. if el.tag == "div" and el.get("class") == "mkdocstrings": # Delete the duplicated headings along with their container, but keep the text (i.e. the actual HTML). carry_text = (el.text or "") + carry_text - root.remove(el) - elif carry_text: - el.tail = (el.tail or "") + carry_text - carry_text = "" + parent.remove(el) + else: + if carry_text: + el.tail = (el.tail or "") + carry_text + carry_text = "" + self._remove_duplicated_headings(el) + if carry_text: - root.text = (root.text or "") + carry_text + parent.text = (parent.text or "") + carry_text class MkdocstringsExtension(Extension): diff --git a/src/mkdocstrings/handlers/base.py b/src/mkdocstrings/handlers/base.py index 700a0565..f52e17dc 100644 --- a/src/mkdocstrings/handlers/base.py +++ b/src/mkdocstrings/handlers/base.py @@ -8,11 +8,12 @@ import importlib import sys from pathlib import Path -from typing import Any, BinaryIO, ClassVar, Iterable, Iterator, Mapping, MutableMapping, Sequence +from typing import Any, BinaryIO, ClassVar, Iterable, Iterator, Mapping, MutableMapping, Sequence, cast from xml.etree.ElementTree import Element, tostring from jinja2 import Environment, FileSystemLoader from markdown import Markdown +from markdown.extensions.toc import TocTreeprocessor from markupsafe import Markup from mkdocstrings.handlers.rendering import ( @@ -268,15 +269,15 @@ def do_convert_markdown( An HTML string. """ treeprocessors = self._md.treeprocessors - treeprocessors[HeadingShiftingTreeprocessor.name].shift_by = heading_level - treeprocessors[IdPrependingTreeprocessor.name].id_prefix = html_id and html_id + "--" - treeprocessors[ParagraphStrippingTreeprocessor.name].strip = strip_paragraph + treeprocessors[HeadingShiftingTreeprocessor.name].shift_by = heading_level # type: ignore[attr-defined] + treeprocessors[IdPrependingTreeprocessor.name].id_prefix = html_id and html_id + "--" # type: ignore[attr-defined] + treeprocessors[ParagraphStrippingTreeprocessor.name].strip = strip_paragraph # type: ignore[attr-defined] try: return Markup(self._md.convert(text)) finally: - treeprocessors[HeadingShiftingTreeprocessor.name].shift_by = 0 - treeprocessors[IdPrependingTreeprocessor.name].id_prefix = "" - treeprocessors[ParagraphStrippingTreeprocessor.name].strip = False + treeprocessors[HeadingShiftingTreeprocessor.name].shift_by = 0 # type: ignore[attr-defined] + treeprocessors[IdPrependingTreeprocessor.name].id_prefix = "" # type: ignore[attr-defined] + treeprocessors[ParagraphStrippingTreeprocessor.name].strip = False # type: ignore[attr-defined] self._md.reset() def do_heading( @@ -319,7 +320,7 @@ def do_heading( el = Element(f"h{heading_level}", attributes) el.append(Element("mkdocstrings-placeholder")) # Tell the 'toc' extension to make its additions if configured so. - toc = self._md.treeprocessors["toc"] + toc = cast(TocTreeprocessor, self._md.treeprocessors["toc"]) if toc.use_anchors: toc.add_anchor(el, attributes["id"]) if toc.use_permalinks: @@ -388,7 +389,7 @@ def __init__(self, config: dict) -> None: self._handlers: dict[str, BaseHandler] = {} self.inventory: Inventory = Inventory(project=self._config["mkdocs"]["site_name"]) - def get_anchors(self, identifier: str) -> tuple[str, ...] | set[str]: + def get_anchors(self, identifier: str) -> tuple[str, ...]: """Return the canonical HTML anchor for the identifier, if any of the seen handlers can collect it. Arguments: diff --git a/src/mkdocstrings/handlers/rendering.py b/src/mkdocstrings/handlers/rendering.py index 6009935a..2cba2538 100644 --- a/src/mkdocstrings/handlers/rendering.py +++ b/src/mkdocstrings/handlers/rendering.py @@ -211,12 +211,13 @@ def __init__(self, md: Markdown, headings: list[Element]): self.headings = headings def run(self, root: Element) -> None: + permalink_class = self.md.treeprocessors["toc"].permalink_class # type: ignore[attr-defined] for el in root.iter(): if self.regex.fullmatch(el.tag): el = copy.copy(el) # noqa: PLW2901 # 'toc' extension's first pass (which we require to build heading stubs/ids) also edits the HTML. # Undo the permalink edit so we can pass this heading to the outer pass of the 'toc' extension. - if len(el) > 0 and el[-1].get("class") == self.md.treeprocessors["toc"].permalink_class: + if len(el) > 0 and el[-1].get("class") == permalink_class: del el[-1] self.headings.append(el) diff --git a/src/mkdocstrings/plugin.py b/src/mkdocstrings/plugin.py index 484d3ead..48a7d1ab 100644 --- a/src/mkdocstrings/plugin.py +++ b/src/mkdocstrings/plugin.py @@ -14,26 +14,27 @@ from __future__ import annotations +import datetime import functools -import gzip import os import sys from concurrent import futures -from typing import TYPE_CHECKING, Any, BinaryIO, Callable, Iterable, List, Mapping, Tuple, TypeVar -from urllib import request +from io import BytesIO +from typing import TYPE_CHECKING, Any, Callable, Iterable, List, Mapping, Tuple, TypeVar -from mkdocs.config.config_options import Type as MkType +from mkdocs.config import Config +from mkdocs.config import config_options as opt from mkdocs.plugins import BasePlugin from mkdocs.utils import write_file from mkdocs_autorefs.plugin import AutorefsPlugin +from mkdocstrings._cache import download_and_cache_url, download_url_with_gz from mkdocstrings.extension import MkdocstringsExtension from mkdocstrings.handlers.base import BaseHandler, Handlers from mkdocstrings.loggers import get_logger if TYPE_CHECKING: from jinja2.environment import Environment - from mkdocs.config import Config from mkdocs.config.defaults import MkDocsConfig if sys.version_info < (3, 10): @@ -62,37 +63,15 @@ def wrapper(*args: P.args, **kwargs: P.kwargs) -> R: return wrapper -class MkdocstringsPlugin(BasePlugin): - """An `mkdocs` plugin. - - This plugin defines the following event hooks: - - - `on_config` - - `on_env` - - `on_post_build` +class PluginConfig(Config): + """The configuration options of `mkdocstrings`, written in `mkdocs.yml`.""" - Check the [Developing Plugins](https://www.mkdocs.org/user-guide/plugins/#developing-plugins) page of `mkdocs` - for more information about its plugin system. - """ - - config_scheme: tuple[tuple[str, MkType]] = ( # type: ignore[assignment] - ("handlers", MkType(dict, default={})), - ("default_handler", MkType(str, default="python")), - ("custom_templates", MkType(str, default=None)), - ("enable_inventory", MkType(bool, default=None)), - ("enabled", MkType(bool, default=True)), - ) + handlers = opt.Type(dict, default={}) """ - The configuration options of `mkdocstrings`, written in `mkdocs.yml`. + Global configuration of handlers. - Available options are: - - - **`handlers`**: Global configuration of handlers. You can set global configuration per handler, applied everywhere, - but overridable in each "autodoc" instruction. Example: - - **`default_handler`**: The default handler to use. The value is the name of the handler module. Default is "python". - - **`custom_templates`**: Custom templates to use when rendering API objects. - - **`enable_inventory`**: Whether to enable object inventory creation. - - **`enabled`**: Whether to enable the plugin. Default is true. If false, *mkdocstrings* will not collect or render anything. + You can set global configuration per handler, applied everywhere, + but overridable in each "autodoc" instruction. Example: ```yaml plugins: @@ -108,6 +87,32 @@ class MkdocstringsPlugin(BasePlugin): ``` """ + default_handler = opt.Type(str, default="python") + """The default handler to use. The value is the name of the handler module. Default is "python".""" + custom_templates = opt.Optional(opt.Dir(exists=True)) + """Location of custom templates to use when rendering API objects. + + Value should be the path of a directory relative to the MkDocs configuration file. + """ + enable_inventory = opt.Optional(opt.Type(bool)) + """Whether to enable object inventory creation.""" + enabled = opt.Type(bool, default=True) + """Whether to enable the plugin. Default is true. If false, *mkdocstrings* will not collect or render anything.""" + + +class MkdocstringsPlugin(BasePlugin[PluginConfig]): + """An `mkdocs` plugin. + + This plugin defines the following event hooks: + + - `on_config` + - `on_env` + - `on_post_build` + + Check the [Developing Plugins](https://www.mkdocs.org/user-guide/plugins/#developing-plugins) page of `mkdocs` + for more information about its plugin system. + """ + css_filename = "assets/_mkdocstrings.css" def __init__(self) -> None: @@ -150,10 +155,10 @@ def on_config(self, config: MkDocsConfig) -> MkDocsConfig | None: return config log.debug("Adding extension to the list") - theme_name = config["theme"].name or os.path.dirname(config["theme"].dirs[0]) + theme_name = config.theme.name or os.path.dirname(config.theme.dirs[0]) to_import: InventoryImportType = [] - for handler_name, conf in self.config["handlers"].items(): + for handler_name, conf in self.config.handlers.items(): for import_item in conf.pop("import", ()): if isinstance(import_item, str): import_item = {"url": import_item} # noqa: PLW2901 @@ -161,30 +166,31 @@ def on_config(self, config: MkDocsConfig) -> MkDocsConfig | None: extension_config = { "theme_name": theme_name, - "mdx": config["markdown_extensions"], - "mdx_configs": config["mdx_configs"], + "mdx": config.markdown_extensions, + "mdx_configs": config.mdx_configs, "mkdocstrings": self.config, "mkdocs": config, } self._handlers = Handlers(extension_config) + autorefs: AutorefsPlugin try: # If autorefs plugin is explicitly enabled, just use it. - autorefs = config["plugins"]["autorefs"] + autorefs = config.plugins["autorefs"] # type: ignore[assignment] log.debug(f"Picked up existing autorefs instance {autorefs!r}") except KeyError: # Otherwise, add a limited instance of it that acts only on what's added through `register_anchor`. autorefs = AutorefsPlugin() autorefs.scan_toc = False - config["plugins"]["autorefs"] = autorefs + config.plugins["autorefs"] = autorefs log.debug(f"Added a subdued autorefs instance {autorefs!r}") # Add collector-based fallback in either case. autorefs.get_fallback_anchor = self.handlers.get_anchors mkdocstrings_extension = MkdocstringsExtension(extension_config, self.handlers, autorefs) - config["markdown_extensions"].append(mkdocstrings_extension) + config.markdown_extensions.append(mkdocstrings_extension) # type: ignore[arg-type] - config["extra_css"].insert(0, self.css_filename) # So that it has lower priority than user files. + config.extra_css.insert(0, self.css_filename) # So that it has lower priority than user files. self._inv_futures = {} if to_import: @@ -208,7 +214,7 @@ def inventory_enabled(self) -> bool: Returns: Whether the inventory is enabled. """ - inventory_enabled = self.config["enable_inventory"] + inventory_enabled = self.config.enable_inventory if inventory_enabled is None: inventory_enabled = any(handler.enable_inventory for handler in self.handlers.seen_handlers) return inventory_enabled @@ -220,9 +226,9 @@ def plugin_enabled(self) -> bool: Returns: Whether the plugin is enabled. """ - return self.config["enabled"] + return self.config.enabled - def on_env(self, env: Environment, config: Config, *args: Any, **kwargs: Any) -> None: # noqa: ARG002 + def on_env(self, env: Environment, config: MkDocsConfig, *args: Any, **kwargs: Any) -> None: # noqa: ARG002 """Extra actions that need to happen after all Markdown rendering and before HTML rendering. Hook for the [`on_env` event](https://www.mkdocs.org/user-guide/plugins/#on_env). @@ -234,12 +240,12 @@ def on_env(self, env: Environment, config: Config, *args: Any, **kwargs: Any) -> return if self._handlers: css_content = "\n".join(handler.extra_css for handler in self.handlers.seen_handlers) - write_file(css_content.encode("utf-8"), os.path.join(config["site_dir"], self.css_filename)) + write_file(css_content.encode("utf-8"), os.path.join(config.site_dir, self.css_filename)) if self.inventory_enabled: log.debug("Creating inventory file objects.inv") inv_contents = self.handlers.inventory.format_sphinx() - write_file(inv_contents, os.path.join(config["site_dir"], "objects.inv")) + write_file(inv_contents, os.path.join(config.site_dir, "objects.inv")) if self._inv_futures: log.debug(f"Waiting for {len(self._inv_futures)} inventory download(s)") @@ -254,12 +260,12 @@ def on_env(self, env: Environment, config: Config, *args: Any, **kwargs: Any) -> loader_name = loader.__func__.__qualname__ log.error(f"Couldn't load inventory {import_item} through {loader_name}: {error}") # noqa: TRY400 for page, identifier in results.items(): - config["plugins"]["autorefs"].register_url(page, identifier) + config.plugins["autorefs"].register_url(page, identifier) # type: ignore[attr-defined] self._inv_futures = {} def on_post_build( self, - config: Config, # noqa: ARG002 + config: MkDocsConfig, # noqa: ARG002 **kwargs: Any, # noqa: ARG002 ) -> None: """Teardown the handlers. @@ -312,11 +318,7 @@ def _load_inventory(cls, loader: InventoryLoaderType, url: str, **kwargs: Any) - A mapping from identifier to absolute URL. """ log.debug(f"Downloading inventory from {url!r}") - req = request.Request(url, headers={"Accept-Encoding": "gzip", "User-Agent": "mkdocstrings/0.15.0"}) - with request.urlopen(req) as resp: # noqa: S310 (URL audit OK: comes from a checked-in config) - content: BinaryIO = resp - if "gzip" in resp.headers.get("content-encoding", ""): - content = gzip.GzipFile(fileobj=resp) # type: ignore[assignment] - result = dict(loader(content, url=url, **kwargs)) + content = download_and_cache_url(url, download_url_with_gz, datetime.timedelta(days=1)) + result = dict(loader(BytesIO(content), url=url, **kwargs)) log.debug(f"Loaded inventory from {url!r}: {len(result)} items") return result diff --git a/tests/conftest.py b/tests/conftest.py index 2119d1f3..9bb09368 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -7,24 +7,24 @@ import pytest from markdown.core import Markdown -from mkdocs import config -from mkdocs.config.defaults import get_schema +from mkdocs.config.defaults import MkDocsConfig if TYPE_CHECKING: from pathlib import Path + from mkdocs import config + from mkdocstrings.plugin import MkdocstringsPlugin @pytest.fixture(name="mkdocs_conf") def fixture_mkdocs_conf(request: pytest.FixtureRequest, tmp_path: Path) -> Iterator[config.Config]: """Yield a MkDocs configuration object.""" - conf = config.Config(schema=get_schema()) # type: ignore[call-arg] + conf = MkDocsConfig() while hasattr(request, "_parent_request") and hasattr(request._parent_request, "_parent_request"): request = request._parent_request conf_dict = { - "config_file_path": "mkdocs_tests.yml", "site_name": "foo", "site_url": "https://example.org/", "site_dir": str(tmp_path), @@ -53,6 +53,6 @@ def fixture_plugin(mkdocs_conf: config.Config) -> MkdocstringsPlugin: @pytest.fixture(name="ext_markdown") -def fixture_ext_markdown(mkdocs_conf: config.Config) -> Markdown: +def fixture_ext_markdown(mkdocs_conf: MkDocsConfig) -> Markdown: """Return a Markdown instance with MkdocstringsExtension.""" return Markdown(extensions=mkdocs_conf["markdown_extensions"], extension_configs=mkdocs_conf["mdx_configs"]) diff --git a/tests/fixtures/headings_many.py b/tests/fixtures/headings_many.py new file mode 100644 index 00000000..fa643a48 --- /dev/null +++ b/tests/fixtures/headings_many.py @@ -0,0 +1,10 @@ +def heading_1(): + """## Heading one""" + + +def heading_2(): + """### Heading two""" + + +def heading_3(): + """#### Heading three""" diff --git a/tests/test_extension.py b/tests/test_extension.py index 8c687629..4b470647 100644 --- a/tests/test_extension.py +++ b/tests/test_extension.py @@ -140,7 +140,7 @@ def test_dont_register_every_identifier_as_anchor(plugin: MkdocstringsPlugin, ex ids = ("id1", "id2", "id3") handler.get_anchors = lambda _: ids # type: ignore[method-assign] ext_markdown.convert("::: tests.fixtures.headings") - autorefs = ext_markdown.parser.blockprocessors["mkdocstrings"]._autorefs + autorefs = ext_markdown.parser.blockprocessors["mkdocstrings"]._autorefs # type: ignore[attr-defined] for identifier in ids: assert identifier not in autorefs._url_map assert identifier not in autorefs._abs_url_map @@ -150,3 +150,25 @@ def test_use_options_yaml_key(ext_markdown: Markdown) -> None: """Check that using the 'options' YAML key works as expected.""" assert "h1" in ext_markdown.convert("::: tests.fixtures.headings\n options:\n heading_level: 1") assert "h1" not in ext_markdown.convert("::: tests.fixtures.headings\n options:\n heading_level: 2") + + +@pytest.mark.parametrize("ext_markdown", [{"markdown_extensions": [{"admonition": {}}]}], indirect=["ext_markdown"]) +def test_removing_duplicated_headings(ext_markdown: Markdown) -> None: + """Assert duplicated headings are removed from the output.""" + output = ext_markdown.convert( + dedent( + """ + ::: tests.fixtures.headings_many.heading_1 + + !!! note + + ::: tests.fixtures.headings_many.heading_2 + + ::: tests.fixtures.headings_many.heading_3 + """, + ), + ) + assert output.count(">Heading one<") == 1 + assert output.count(">Heading two<") == 1 + assert output.count(">Heading three<") == 1 + assert output.count('class="mkdocstrings') == 0 diff --git a/tests/test_plugin.py b/tests/test_plugin.py index b8e8d2a5..3342e2aa 100644 --- a/tests/test_plugin.py +++ b/tests/test_plugin.py @@ -7,6 +7,8 @@ from mkdocs.commands.build import build from mkdocs.config import load_config +from mkdocstrings.plugin import MkdocstringsPlugin + if TYPE_CHECKING: from pathlib import Path @@ -31,3 +33,39 @@ def test_disabling_plugin(tmp_path: Path) -> None: # make sure the instruction was not processed assert "::: mkdocstrings" in site_dir.joinpath("index.html").read_text() + + +def test_plugin_default_config(tmp_path: Path) -> None: + """Test default config options are set for Plugin.""" + config_file_path = tmp_path / "mkdocs.yml" + plugin = MkdocstringsPlugin() + errors, warnings = plugin.load_config({}, config_file_path=str(config_file_path)) + assert errors == [] + assert warnings == [] + assert plugin.config == { + "handlers": {}, + "default_handler": "python", + "custom_templates": None, + "enable_inventory": None, + "enabled": True, + } + + +def test_plugin_config_custom_templates(tmp_path: Path) -> None: + """Test custom_templates option is relative to config file.""" + config_file_path = tmp_path / "mkdocs.yml" + options = {"custom_templates": "docs/templates"} + template_dir = tmp_path / options["custom_templates"] + # Path must exist or config validation will fail. + template_dir.mkdir(parents=True) + plugin = MkdocstringsPlugin() + errors, warnings = plugin.load_config(options, config_file_path=str(config_file_path)) + assert errors == [] + assert warnings == [] + assert plugin.config == { + "handlers": {}, + "default_handler": "python", + "custom_templates": str(template_dir), + "enable_inventory": None, + "enabled": True, + }