Docs audit [April 2026]: Full version#3341
Open
C-Achard wants to merge 58 commits into
Open
Conversation
* Add docs and notebooks audit export tool Add tools/docs_and_notebooks_audit.py: a CLI tool that scans Markdown and Jupyter notebooks for a deeplabcut metadata namespace, validates selected fields (visibility, status, recommendation, last_verified) against enums via pydantic, and exports a CSV audit register. Features include YAML frontmatter and notebook metadata parsing, alias support for recommendation, preservation of human-authored CSV columns across re-runs, configurable include/exclude scan patterns, target filtering (files, dirs, globs), and basic reporting of parse/validation issues. * Ignore tools/docs_audits in docs scan Add tools/docs_audits/** to the scan exclude list in tools/docs_and_notebooks_report_config.yml so the docs & notebooks report skips files in that directory (e.g., generated audit artifacts or temporary outputs). * Ignore docs_audits; include tools MD files Add an exclude pattern (^tools/docs_audits/) to the docs/notebooks pre-commit hook so files under tools/docs_audits/ are skipped. Also add tools/**/*.md to the scan list in tools/docs_and_notebooks_report_config.yml so Markdown files in the tools directory are included in the docs/notebooks report. * Introduce markdown-myst pre-commit hook ## Summary Adds `mdformat` v1.0 as a pre commit hook with the myst plugin. This helps automate formatting of documentation and catch any mistakes in structure or syntax, making docs maintenance easier. ## Scope As a hook with write permissions, it is meant to run locally, not in CI. * Add notes field and 'move' recommendation Add support for human-editable notes and a new Recommendation.MOVE enum. AuditMetadata now includes a notes field and FIELD_SPECS registers notes so notes are read from CSV. merged_row now preserves existing human notes, warns on conflicts between scanned and previous notes, and prefers previous notes when present. build_row no longer unconditionally writes an empty notes value (to avoid clobbering preserved notes). Also removed the file shebang line. * Document Recommendation enum values Add brief inline comments to each member of the Recommendation enum in tools/docs_and_notebooks_audit.py to clarify what each action means for auditors (keep, verify, update, move, merge, archive, remove). This is a documentation-only change and does not alter runtime behavior. * Update docs_and_notebooks_audit.py * Fix pre-commit YAML indent and update usage examples Corrects YAML indentation in .pre-commit-config.yaml so mdformat-myst is properly listed under additional_dependencies. Also updates usage examples in tools/docs_and_notebooks_audit.py to reference the docs_and_notebooks_audit.py script (replacing docs_audit_export.py) to reflect the current script name.
## Summary Adds `mdformat` v1.0 as a pre commit hook with the myst plugin. This helps automate formatting of documentation and catch any mistakes in structure or syntax, making docs maintenance easier. ## Scope As a hook with write permissions, it is meant to run locally, not in CI.
* Docs versioning: Add glob support, better validation and reporting Normalize CLI target specs (handle Windows/backslashes, ./, absolute paths) and classify them as file/dir/glob. Implement matching logic and validation (report matched files and unmatched selectors, return rc=2 on unmatched), and apply target specs in scan/update. Add helpers (normalize_target_spec, compile_target_specs, validate_requested_targets, print_target_match_summary, iter_scan_candidate_paths) and adjust main argument help. Update tests to cover directory, glob, Windows-style paths, and CLI reporting. * Add metadata sync check and CI helpers Add a fail_if_metadata_sync_needed policy flag and utilities to detect/collect files whose embedded metadata last_content_updated differs from the computed git content date. New helpers: record_needs_metadata_sync() (skips non-md/ipynb and meta.ignore), collect_metadata_sync_targets() (returns sorted unique paths), and build_metadata_sync_command() (emits a ready-to-run python command to update targets using --set-content-date-from-git and --ack-meta-commit-marker). The enforce() flow now emits violations for out-of-sync md/ipynb files when the flag is enabled. * Add metadata sync suggestions to report Introduce actionable metadata-sync guidance for maintainers: rename SCHEMA_VERSION to REPORT_SCHEMA_VERSION, restrict ToolConfig.version to 1, and add metadata_sync_targets and metadata_sync_command to the Report model. Add build_git_add_command helper and mark records requiring metadata sync (metadata_sync_needed) so summaries include that count. Extend markdown output and CLI to show which files need metadata updates and to print suggested commands (sync command, git add and commit) for applying fixes locally. * Limit docs/notebooks scan to changed files Update the GitHub Actions workflow to only run docs and notebook staleness checks for changed .md/.markdown/.ipynb files. Adds a step to collect changed docs into tmp/docs_nb_checks/changed_docs.txt and expose the count via outputs; conditionally runs the report and optional policy check only when there are changed files, passing the files as --targets to the checker. Adds a no-op step when no docs changed and uploads the changed_docs.txt alongside other artifacts. Also renames the job to reflect the new behavior to reduce unnecessary scanning and noise. * Handle invalid target specs in validation Treat target specs that fail normalization as kind "invalid" so they are not silently ignored. compile_target_specs now appends invalid specs, target_spec_matches_path returns False for invalid kinds, and validate_requested_targets records invalid raw selectors as unmatched (and safely iterates when specs may be None). This ensures malformed or non-normalizable CLI selectors are reported back to the user rather than dropped. * tests: add repo/cfg fixtures and refactor tests Introduce shared pytest fixtures (repo, cfg) and import Callable to reduce repetition of tmp_path/git init logic across tests. Refactor many tests to use the new fixtures and ToolConfig factory, and add/adjust tests covering target validation and scanning edge cases (several validate_requested_targets variations, scan_files with invalid only-targets, and main returning 2 for invalid selector). Overall this centralizes repo/config setup and adds coverage for target handling behavior. * Drop .markdown from docs CI as it is unused * Fix const import in tests * Handle missing timestamps and shell-quote paths tools/docs_and_notebooks_check.py: Return False from record_needs_metadata_sync when computed timestamp is None to avoid triggering sync for files with no computed last_content_updated. Also import shlex and quote paths in build_metadata_sync_command so generated shell commands are safe for paths with spaces/special chars. * Clarify --targets help text Expand and clarify the --targets argument help strings in tools/docs_and_notebooks_check.py for the update and normalize subcommands. The updated messages document that --targets accepts exact files, directories, and glob patterns (with examples) and note that both '/' and '\\' path separators are accepted. This is a documentation-only change to improve user guidance; no functional behavior is altered. * Update tools/docs_and_notebooks_check.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update docs_and_notebooks_check.py * Update tools/docs_and_notebooks_check.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update .github/workflows/docs_and_notebooks_checks.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Quote config path * Revert "Merge branch 'cy/docs-versioning-tweaks' of https://github.com/DeepLabCut/DeepLabCut into cy/docs-versioning-tweaks" This reverts commit 46fdb04, reversing changes made to 6d3b1be. * Use TypedDict for targets and simplify glob matching Introduce TypedDict-based types for FileKind and TargetSpec (with TargetKind) and tighten type annotations across functions (compile_target_specs, target_spec_matches_path, target_matches). Replace PurePosixPath-based glob matching with fnmatch.fnmatchcase only to ensure consistent, shell-style pattern behavior across platforms and remove unused imports. Minor cleanups: import TypedDict and update variable type hints for better static checking and readability. * Avoid mixing metadata sync needed and invalid meta * Add tests for metadata sync warnings/enforcement Add parameterized tests covering both markdown and notebook files to verify metadata sync behavior. test_metadata_sync_warning_populates_report_targets_and_command creates a file with an out-of-date embedded last_content_updated, commits a newer git date, and asserts the record gets a metadata_sync_needed warning, that metadata_sync_targets and metadata_sync_command are populated (and include the target and --set-content-date-from-git flag), and that the rendered markdown includes guidance. test_enforce_fails_when_metadata_sync_needed_is_configured asserts that when fail_if_metadata_sync_needed=True an out-of-sync file is reported as a policy violation. These tests ensure reporting and enforcement handle embedded vs git-derived content dates correctly. * Avoid metadata sync on parse errors Do not mark files as needing metadata sync when metadata could not be read/parsed/validated reliably. Add blocking error prefixes (metadata_read_failed:, markdown_frontmatter_invalid:, nbformat_invalid:) to record_needs_metadata_sync so such scan/repair issues are treated separately. Also clarify the enforcement message to mention embedded last_content_updated and the git content update date for more precise guidance. * Change schema version type to Literal * Fix mismatched test error message Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Case insensitive check for file ext Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com>
* Add docs and notebooks audit export tool Add tools/docs_and_notebooks_audit.py: a CLI tool that scans Markdown and Jupyter notebooks for a deeplabcut metadata namespace, validates selected fields (visibility, status, recommendation, last_verified) against enums via pydantic, and exports a CSV audit register. Features include YAML frontmatter and notebook metadata parsing, alias support for recommendation, preservation of human-authored CSV columns across re-runs, configurable include/exclude scan patterns, target filtering (files, dirs, globs), and basic reporting of parse/validation issues. * Ignore tools/docs_audits in docs scan Add tools/docs_audits/** to the scan exclude list in tools/docs_and_notebooks_report_config.yml so the docs & notebooks report skips files in that directory (e.g., generated audit artifacts or temporary outputs). * Ignore docs_audits; include tools MD files Add an exclude pattern (^tools/docs_audits/) to the docs/notebooks pre-commit hook so files under tools/docs_audits/ are skipped. Also add tools/**/*.md to the scan list in tools/docs_and_notebooks_report_config.yml so Markdown files in the tools directory are included in the docs/notebooks report. * Introduce markdown-myst pre-commit hook ## Summary Adds `mdformat` v1.0 as a pre commit hook with the myst plugin. This helps automate formatting of documentation and catch any mistakes in structure or syntax, making docs maintenance easier. ## Scope As a hook with write permissions, it is meant to run locally, not in CI. * Add notes field and 'move' recommendation Add support for human-editable notes and a new Recommendation.MOVE enum. AuditMetadata now includes a notes field and FIELD_SPECS registers notes so notes are read from CSV. merged_row now preserves existing human notes, warns on conflicts between scanned and previous notes, and prefers previous notes when present. build_row no longer unconditionally writes an empty notes value (to avoid clobbering preserved notes). Also removed the file shebang line. * Document Recommendation enum values Add brief inline comments to each member of the Recommendation enum in tools/docs_and_notebooks_audit.py to clarify what each action means for auditors (keep, verify, update, move, merge, archive, remove). This is a documentation-only change and does not alter runtime behavior. * chore(metadata): update docs/notebooks metadata Installation page * chore(metadata): update docs/notebooks metadata * chore(metadata): update docs/notebooks metadata * chore(metadata): update docs/notebooks metadata * chore(metadata): update docs/notebooks metadata * Update audit files * chore(metadata): update docs/notebooks metadata * Update docs_and_notebooks_audit.py * Fix pre-commit YAML indent and update usage examples Corrects YAML indentation in .pre-commit-config.yaml so mdformat-myst is properly listed under additional_dependencies. Also updates usage examples in tools/docs_and_notebooks_audit.py to reference the docs_and_notebooks_audit.py script (replacing docs_audit_export.py) to reflect the current script name. * Fix typo Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com> --------- Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com>
* Add docs and notebooks audit export tool
Add tools/docs_and_notebooks_audit.py: a CLI tool that scans Markdown and Jupyter notebooks for a deeplabcut metadata namespace, validates selected fields (visibility, status, recommendation, last_verified) against enums via pydantic, and exports a CSV audit register. Features include YAML frontmatter and notebook metadata parsing, alias support for recommendation, preservation of human-authored CSV columns across re-runs, configurable include/exclude scan patterns, target filtering (files, dirs, globs), and basic reporting of parse/validation issues.
* Ignore tools/docs_audits in docs scan
Add tools/docs_audits/** to the scan exclude list in tools/docs_and_notebooks_report_config.yml so the docs & notebooks report skips files in that directory (e.g., generated audit artifacts or temporary outputs).
* Ignore docs_audits; include tools MD files
Add an exclude pattern (^tools/docs_audits/) to the docs/notebooks pre-commit hook so files under tools/docs_audits/ are skipped. Also add tools/**/*.md to the scan list in tools/docs_and_notebooks_report_config.yml so Markdown files in the tools directory are included in the docs/notebooks report.
* Introduce markdown-myst pre-commit hook
## Summary
Adds `mdformat` v1.0 as a pre commit hook with the myst plugin.
This helps automate formatting of documentation and catch any mistakes in structure or syntax, making docs maintenance easier.
## Scope
As a hook with write permissions, it is meant to run locally, not in CI.
* Add notes field and 'move' recommendation
Add support for human-editable notes and a new Recommendation.MOVE enum. AuditMetadata now includes a notes field and FIELD_SPECS registers notes so notes are read from CSV. merged_row now preserves existing human notes, warns on conflicts between scanned and previous notes, and prefers previous notes when present. build_row no longer unconditionally writes an empty notes value (to avoid clobbering preserved notes). Also removed the file shebang line.
* Document Recommendation enum values
Add brief inline comments to each member of the Recommendation enum in tools/docs_and_notebooks_audit.py to clarify what each action means for auditors (keep, verify, update, move, merge, archive, remove). This is a documentation-only change and does not alter runtime behavior.
* chore(metadata): update docs/notebooks metadata
Installation page
* chore(metadata): update docs/notebooks metadata
* chore(metadata): update docs/notebooks metadata
* chore(metadata): update docs/notebooks metadata
* chore(metadata): update docs/notebooks metadata
* Update audit files
* chore(metadata): update docs/notebooks metadata
* Polish README content and docs metadata
Clean up and reorganize README.md: adjust headings and phrasing, remove extraneous blank lines, clarify installation and TensorFlow deprecation wording, add/rename sections (Pretrained Models, Development and Applications, Ecosystem, Code contributors, Funding), comment out outdated course/residency/roadmap links, update community badges and links, and improve news/release notes. Also add brief HTML comments to docs/README.md indicating the root README is used for main documentation and noting frontmatter metadata behavior.
* Revise installation docs and update instructions
Refactor docs/installation.md: update frontmatter metadata and restructure the installation guide. Broaden supported Python versions (3.10–3.12), clarify conda vs miniconda usage, add `uv` developer install instructions and editable `pip install -e` guidance, and reorganize sections for Conda, pip, Docker, GPU support, troubleshooting, and system/hardware considerations. Update TensorFlow deprecation timeline and GPU/CUDA guidance, improve admonitions/notes formatting, fix anchors/section refs, and add various wording and usability improvements.
* docs: clarify and reformat installation guide
Large cleanup and reorganization of docs/installation.md: rename heading, fix Markdown/admonition syntax, and normalize code fences. Clarified differences between Miniconda/Anaconda/conda, added a dedicated 'Build a conda environment' step, Windows-specific warnings, and tips for activating and managing environments. Added guidance for launching the GUI, created a simple conda env example, improved GPU/CUDA notes and troubleshooting links, and reformatted numerous paragraphs and images for readability.
* Revise UseOverviewGuide content and layout
Restructure and update the UseOverviewGuide page: fix markup (e.g. hint directive), add an Introduction and Workflow overview, embed/adjust images, and reorganize sections (What we support → Main modes; Additional learning resources; Usage advice & project types). Clarify installation reference, start/requirements guidance, and what you don't need. Add tips/warnings (recommend single-animal first, Windows admin warning), improve option headings for demos/GUI/terminal, and add brief terminal usage steps and useful links. Remove or comment out outdated/broken link references and tidy copy for clarity.
* Update installation docs: GPU/OS guidance
Rewrite and clarify installation guidance in docs/installation.md. Rephrase installation bullets (conda recommended, Docker note, GPU performance), fix image HTML closing and caption, and replace a single admonition with a tab-set providing CPU, NVIDIA GPU, and Apple M-chip GPU tabs (including CUDA/cuDNN guidance). Minor wording and link adjustments for clarity and reproducibility.
* Clarify and reformat UseOverviewGuide
Refines wording and layout in the UseOverviewGuide: changes "What scenario" to "Which scenario", normalizes "Quick start" capitalization, and restructures multi-animal guidance into clearer bullet points. Expands guidance about using multianimal mode for single-animal projects (when a skeleton helps) and warns against treating fixed-position parts (e.g., whiskers) as separate individuals. Consolidates and repositions tutorial links, adds an important note to label identifiable animals consistently, updates the conversion link phrasing, and clarifies the Windows note to state admin is required for certain usage tasks (downloading models/symlinks) but not for installation.
* Convert links to Sphinx refs and tidy docs
Replace Markdown inline links with Sphinx {ref} cross-references, standardize admonition/class syntax, and clean up spacing and punctuation throughout docs/installation.md. Also normalize casing for the DeepLabCut heading, adjust image/code block indentation, add a MacOS GPU support footnote, and make minor clarity edits to GPU/PyTorch/TensorFlow installation notes and examples.
* Fix typos and formatting in installation docs
Clean up docs/installation.md: fix an escaped Windows activate path (activate.ps1), remove stray blank lines, and separate/version-check and update encouragement into their own paragraph. Add a "Data compatibility" subsection, rename the system-wide header to "System-wide installation considerations", and make small wording/formatting tweaks (e.g., emphasize "temporarily", reword potential package conflicts, and normalize "very slow"). These changes improve clarity and readability of the installation guide.
* Fix markdown code blocks and list numbering
Adjust formatting in docs/UseOverviewGuide.md: indent fenced code blocks for proper rendering and normalize the ordered list numbering (use '1.' for sequential steps) for consistency in the instructions.
* Remove outdated installation tips links
Disable references to the outdated 'installation tips' page in docs/installation.md. Replaced direct links and related guidance with HTML comments, added a note that the git-clone section is the place for editable installs, adjusted the pro-tip wording to point users to the test video, and commented out the 'Additional tips' block. Original content is preserved in comments for review.
* Update PROJECT_GUI.md
* Clarify Project GUI wording in docs
Make minor copy edits to docs/gui/PROJECT_GUI.md for clarity: change "easy-entry point" to "easy entry point", replace "easy-to-deploy GUI interface" with "easy-to-use GUI interface", and rephrase the following sentence to more clearly state that several advanced features are not available in the Project GUI.
* Update beginner guide ref
* Revise Beginner Guide to focus on GUI
Refactor beginners-guide.md to center the guide on the DeepLabCut GUI. Updated the main heading and clarified startup wording (python -m deeplabcut) and project-creation workflow. Removed/replaced redundant installation text with a reference to the installation page and commented out an outdated course link. Added notes recommending the PyTorch engine, improved tips for video selection/copying, added guidance for defining bodyparts and multiple individuals, included a video tutorial GIF, and updated the next-steps reference to the GUI-specific manage-project page. Miscellaneous wording and screenshot tweaks for clarity.
* Update install link ref
* Refactor labeling guide and napari workflow
Restructure and clarify the labeling documentation: reorganized headings and sections, consolidated frame selection guidance (automatic vs manual), and added an example workflow, important/tip admonitions, and a "Next steps" pointer to training/evaluation. Updated phrasing for clarity, fixed formatting (headings, links, and image reference), and replaced a quoted YAML note with unquoted text. These changes improve readability and better integrate the napari-DLC labeling workflow with the rest of the docs.
* Clarify GUI config editing in manage-project
Revise the manage-project beginner guide to focus on editing the GUI configuration file. Renamed the section, added a file anchor, and reworded the intro to explain config.yaml as the central project record. Introduced a dedicated "Editing the configuration" and a step-by-step walkthrough that consolidates bodyparts and skeleton guidance (including details on entries/IDs), improved bullets and examples, and added a tip and an HTML comment. Updated the link to the labeling guide (file:labeling-gui) and made minor YAML/frontmatter and formatting fixes.
* Update GUI training/evaluation guide
Revise docs/beginner-guides/Training-Evaluation.md: add RST anchor and restructure headings (Network training, Creating a training set, Starting the training process, Network evaluation, Next steps). Clarify wording (e.g. defaults described as “good” to start), convert tip to RST directive, adjust evaluation section formatting and images, and add reference to the video-analysis GUI section. Also change YAML notes value formatting.
* Refactor video analysis guide for GUI
Rework the beginner video-analysis doc to focus on the GUI workflow: add a file anchor, rename and restructure headings (Analyzing videos with your trained model, Generating labeled videos, Next steps), reorganize and clarify step-by-step instructions, and update copy. Also remove quoting from the frontmatter notes and add a TODO link comment for future cross-references.
* Refine beginner guides and napari usage docs
Update several docs to improve formatting, clarity, and links. Convert inline notes to admonition blocks, fix spacing/indentation around images and note blocks, and bold example items for readability (docs/beginner-guides/beginners-guide.md). Clarify labeling guidance and emphasize labeling across multiple videos; update front-matter note to link to napari plugin docs (docs/beginner-guides/labeling.md). Adjust manage-project wording and next-step link to the labeling guide (docs/beginner-guides/manage-project.md). Add a brief napari usage recommendation with a link to the official napari docs (docs/gui/napari/basic_usage.md). These changes aim to make the GUI/napari workflow and labeling guidance clearer for beginners.
* Tidy beginner-guides
Minor cleanup and reorganization prep for beginner guides: remove an extra closing parenthesis from an image link in Training-Evaluation.md; update frontmatter recommendation from "update" to "move" and simplify notes to indicate moving content to a dedicated GUI section in beginners-guide.md, labeling.md, and manage-project.md; and change "Click on" to "Select" in the labeling workflow example. These edits prepare the docs for consolidation under a GUI section.
* Refine PROJECT_GUI docs layout and links
Polish docs/gui/PROJECT_GUI.md: rename headings for clarity ("Interactive Project Manager GUI" → "Project Manager GUI", "Get Started:" → "Getting started", "Video Demo" → "User guide" / "Video demos"); replace inline markdown links with Sphinx {ref} cross-references (napari and beginners guide); add an important note linking to the beginners guide and a tip to "Click on the images!". These changes improve consistency with the docs style and surface the step-by-step GUI guide.
* Fix link in getting started
* docs: update Docker docs formatting and notes
Refactor docs/docker.md to improve formatting and clarify guidance. Renamed the section heading, convert several inline notes into MyST admonitions (important/note), wrap the napari-deeplabcut install guidance into an important block, and adjust the GPU reminder into an admonition. Minor rework of the mount/example instructions and commented out a redundant labelling note to reduce duplication and improve readability.
* Apply suggestions from code review
Credita to @deruyter92
Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com>
* Separate GPU and cloud bullets; remove link
Split the combined GPU/cloud sentence into two separate bullets for clarity. Removed the technical specs (FAQ) link from the GPU item and removed the troubleshooting wiki link at the bottom. Also cleaned up trailing newline/whitespace.
* Update README: TF backend, dataset links
Simplify and update README text: streamline TensorFlow engine install instructions and add a planned deprecation target (tensorflow backend deprecated in v3.2). Replace prior dataset references with explicit Zenodo DOIs for SuperAnimal-Quadruped and SuperAnimal-TopViewMouse and add a citation to Ye et al. 2024. Clarify that models including AP-10K are available in the API/GUI, and fix minor wording in the funding acknowledgement ("following support").
* Clarify Docker usage and napari limitation
Update docs/docker.md to improve clarity and accuracy of Docker guidance. Reworded introduction to explain that DeepLabCut images are available on DockerHub, that Docker requires a local Docker installation, and that containers provide a reproducible, isolated environment for terminal and Jupyter use. Explicitly state the DeepLabCut GUI is not supported in containers and move/strengthen the important note that the napari-deeplabcut plugin cannot be run inside Docker and must be installed locally (e.g. via pip). Clarified that deeplabcut-docker is a lightweight helper that will not disrupt local TensorFlow/PyTorch or other dependencies, and fixed wording/formatting issues.
* Clarify training dataset creation steps
Add brief guidance in the Training Dataset section to outline key steps (splitting labeled data and creating shuffle folders) so beginners have a clearer workflow before training. Also include a small formatting/whitespace adjustment in the GPU/Colab note.
* Clarify labeling progress and color-scheme docs
Update docs/gui/napari/advanced_usage.md to improve clarity and usability. Reword the labeling progress note to emphasize that the displayed percentage is a rough estimate and that occluded/hidden keypoints are not counted. Expand and simplify instructions for jumping to a bodypart from the color scheme by breaking into explicit steps and clarifying behavior when the bodypart is already visible. Minor wording/formatting cleanups for readability.
* Relocate demo video link; cleanup whitespace
Move the demo video link to appear before the warning block and tidy up minor whitespace/trailing-space issues in docs/gui/napari/basic_usage.md. These are formatting-only changes and do not alter substantive content.
* Update installation.md
* Fix docs formatting, typos and image tags
Clean up documentation across multiple files: add quotes to image src attributes, correct typos (e.g. "oher"→"other", "built"→"build", "compartimentalized"→"compartmentalized"), normalize admonition/code-block syntax, adjust cross-reference/link syntax, and make minor wording/grammar improvements. Affected files include docs/README.md, UseOverviewGuide.md, beginner-guides/*, course.md, docker.md, gui/PROJECT_GUI.md, and installation.md.
* Fix typo
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Docs: add external data import recipe
Add a new recipe (docs/recipes/external_data_import.md) describing how to convert externally annotated data (CSV/H5) into the DeepLabCut format and how to merge multiple datasets (uses deeplabcut.convertcsv2h5).
Update docs/installation.md to use a local Sphinx cross-reference for the Conda section, add the section anchor (sec:installation-using-conda), and fix the uv pip install line to quote the extras ('.[gui,modelzoo,tf]') to avoid shell globbing. Also add a file anchor (file:hardware-requirements)= to docs/recipes/TechHardware.md for cross-references.
* Refine wording and links in user guides
Small editorial updates to user documentation for clarity and better Sphinx linking. In docs/UseOverviewGuide.md: use inline `help` function wording, rephrase 'jump back in' to 'resume work' for clarity, and replace the 'Option 3' Project Manager GUI link with a Sphinx {ref} cross-reference. In docs/beginner-guides/beginners-guide.md: change the tip to 'Avoid spaces' and emphasize the scorer/experimenter field. These changes improve readability and consistency.
* Convert internal docs links to Sphinx refs
Replace Markdown-style internal links with Sphinx/MyST cross-reference syntax and adjust anchors/formatting for proper rendering. Updated docs/UseOverviewGuide.md to use {ref}`...` and change the target anchor to sec:important-info-regd-usage, and fixed the user-guide list formatting. Updated docs/gui/PROJECT_GUI.md to use {ref}`Read more here <sec:important-info-regd-usage>` in two places.
* chore(metadata): update docs/notebooks metadata
* Update course.md
* Docs: copyedits, formatting, and install updates
Editorial and formatting updates across documentation to improve clarity and accuracy. Key changes:
- README.md: minor whitespace/paragraph tweak in funding section.
- docs/UseOverviewGuide.md: corrected phrasing for mode selection.
- docs/beginner-guides/beginners-guide.md: reworded intro, removed stray blank line, clarified install link.
- docs/course.md: removed trailing colon from heading.
- docs/gui/PROJECT_GUI.md: removed outdated release-note lines.
- docs/gui/napari_GUI.md: adjusted front-matter spacing and landing file anchor.
- docs/installation.md: updated PyTorch install example to pip + CUDA 12.6 wheel URL, clarified pip install deeplabcut usage (mentioning --pre), updated uv editable install extras (removed tf), added recommended clone step, improved version-check example and a few other clarifications.
- docs/pytorch/pytorch_config.md: multiple formatting fixes (lists, line wraps, links, code blocks), corrected table header alignment, minor text clarifications and punctuation fixes.
- docs/recipes/TechHardware.md: heading and subsection title adjustments and minor wording tweaks.
- docs/recipes/external_data_import.md: combined demo CSV links into one sentence and adjusted bullet formatting.
- docs/standardDeepLabCut_UserGuide.md: extensive copyedits and formatting cleanups (headings, admonitions, code blocks, escaped characters, consistent naming like add_new_videos), reflowed many paragraphs and improved examples and API-admonition formatting.
Overall these changes are non-functional documentation improvements to make instructions clearer and fix formatting/typography issues.
---------
Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Add docs and notebooks audit export tool
Add tools/docs_and_notebooks_audit.py: a CLI tool that scans Markdown and Jupyter notebooks for a deeplabcut metadata namespace, validates selected fields (visibility, status, recommendation, last_verified) against enums via pydantic, and exports a CSV audit register. Features include YAML frontmatter and notebook metadata parsing, alias support for recommendation, preservation of human-authored CSV columns across re-runs, configurable include/exclude scan patterns, target filtering (files, dirs, globs), and basic reporting of parse/validation issues.
* Ignore tools/docs_audits in docs scan
Add tools/docs_audits/** to the scan exclude list in tools/docs_and_notebooks_report_config.yml so the docs & notebooks report skips files in that directory (e.g., generated audit artifacts or temporary outputs).
* Ignore docs_audits; include tools MD files
Add an exclude pattern (^tools/docs_audits/) to the docs/notebooks pre-commit hook so files under tools/docs_audits/ are skipped. Also add tools/**/*.md to the scan list in tools/docs_and_notebooks_report_config.yml so Markdown files in the tools directory are included in the docs/notebooks report.
* Introduce markdown-myst pre-commit hook
## Summary
Adds `mdformat` v1.0 as a pre commit hook with the myst plugin.
This helps automate formatting of documentation and catch any mistakes in structure or syntax, making docs maintenance easier.
## Scope
As a hook with write permissions, it is meant to run locally, not in CI.
* Add notes field and 'move' recommendation
Add support for human-editable notes and a new Recommendation.MOVE enum. AuditMetadata now includes a notes field and FIELD_SPECS registers notes so notes are read from CSV. merged_row now preserves existing human notes, warns on conflicts between scanned and previous notes, and prefers previous notes when present. build_row no longer unconditionally writes an empty notes value (to avoid clobbering preserved notes). Also removed the file shebang line.
* Document Recommendation enum values
Add brief inline comments to each member of the Recommendation enum in tools/docs_and_notebooks_audit.py to clarify what each action means for auditors (keep, verify, update, move, merge, archive, remove). This is a documentation-only change and does not alter runtime behavior.
* chore(metadata): update docs/notebooks metadata
Installation page
* chore(metadata): update docs/notebooks metadata
* chore(metadata): update docs/notebooks metadata
* chore(metadata): update docs/notebooks metadata
* chore(metadata): update docs/notebooks metadata
* Update audit files
* chore(metadata): update docs/notebooks metadata
* Polish README content and docs metadata
Clean up and reorganize README.md: adjust headings and phrasing, remove extraneous blank lines, clarify installation and TensorFlow deprecation wording, add/rename sections (Pretrained Models, Development and Applications, Ecosystem, Code contributors, Funding), comment out outdated course/residency/roadmap links, update community badges and links, and improve news/release notes. Also add brief HTML comments to docs/README.md indicating the root README is used for main documentation and noting frontmatter metadata behavior.
* Revise installation docs and update instructions
Refactor docs/installation.md: update frontmatter metadata and restructure the installation guide. Broaden supported Python versions (3.10–3.12), clarify conda vs miniconda usage, add `uv` developer install instructions and editable `pip install -e` guidance, and reorganize sections for Conda, pip, Docker, GPU support, troubleshooting, and system/hardware considerations. Update TensorFlow deprecation timeline and GPU/CUDA guidance, improve admonitions/notes formatting, fix anchors/section refs, and add various wording and usability improvements.
* docs: clarify and reformat installation guide
Large cleanup and reorganization of docs/installation.md: rename heading, fix Markdown/admonition syntax, and normalize code fences. Clarified differences between Miniconda/Anaconda/conda, added a dedicated 'Build a conda environment' step, Windows-specific warnings, and tips for activating and managing environments. Added guidance for launching the GUI, created a simple conda env example, improved GPU/CUDA notes and troubleshooting links, and reformatted numerous paragraphs and images for readability.
* Revise UseOverviewGuide content and layout
Restructure and update the UseOverviewGuide page: fix markup (e.g. hint directive), add an Introduction and Workflow overview, embed/adjust images, and reorganize sections (What we support → Main modes; Additional learning resources; Usage advice & project types). Clarify installation reference, start/requirements guidance, and what you don't need. Add tips/warnings (recommend single-animal first, Windows admin warning), improve option headings for demos/GUI/terminal, and add brief terminal usage steps and useful links. Remove or comment out outdated/broken link references and tidy copy for clarity.
* Update installation docs: GPU/OS guidance
Rewrite and clarify installation guidance in docs/installation.md. Rephrase installation bullets (conda recommended, Docker note, GPU performance), fix image HTML closing and caption, and replace a single admonition with a tab-set providing CPU, NVIDIA GPU, and Apple M-chip GPU tabs (including CUDA/cuDNN guidance). Minor wording and link adjustments for clarity and reproducibility.
* Clarify and reformat UseOverviewGuide
Refines wording and layout in the UseOverviewGuide: changes "What scenario" to "Which scenario", normalizes "Quick start" capitalization, and restructures multi-animal guidance into clearer bullet points. Expands guidance about using multianimal mode for single-animal projects (when a skeleton helps) and warns against treating fixed-position parts (e.g., whiskers) as separate individuals. Consolidates and repositions tutorial links, adds an important note to label identifiable animals consistently, updates the conversion link phrasing, and clarifies the Windows note to state admin is required for certain usage tasks (downloading models/symlinks) but not for installation.
* Convert links to Sphinx refs and tidy docs
Replace Markdown inline links with Sphinx {ref} cross-references, standardize admonition/class syntax, and clean up spacing and punctuation throughout docs/installation.md. Also normalize casing for the DeepLabCut heading, adjust image/code block indentation, add a MacOS GPU support footnote, and make minor clarity edits to GPU/PyTorch/TensorFlow installation notes and examples.
* Fix typos and formatting in installation docs
Clean up docs/installation.md: fix an escaped Windows activate path (activate.ps1), remove stray blank lines, and separate/version-check and update encouragement into their own paragraph. Add a "Data compatibility" subsection, rename the system-wide header to "System-wide installation considerations", and make small wording/formatting tweaks (e.g., emphasize "temporarily", reword potential package conflicts, and normalize "very slow"). These changes improve clarity and readability of the installation guide.
* Fix markdown code blocks and list numbering
Adjust formatting in docs/UseOverviewGuide.md: indent fenced code blocks for proper rendering and normalize the ordered list numbering (use '1.' for sequential steps) for consistency in the instructions.
* Remove outdated installation tips links
Disable references to the outdated 'installation tips' page in docs/installation.md. Replaced direct links and related guidance with HTML comments, added a note that the git-clone section is the place for editable installs, adjusted the pro-tip wording to point users to the test video, and commented out the 'Additional tips' block. Original content is preserved in comments for review.
* Update PROJECT_GUI.md
* Clarify Project GUI wording in docs
Make minor copy edits to docs/gui/PROJECT_GUI.md for clarity: change "easy-entry point" to "easy entry point", replace "easy-to-deploy GUI interface" with "easy-to-use GUI interface", and rephrase the following sentence to more clearly state that several advanced features are not available in the Project GUI.
* Update beginner guide ref
* Revise Beginner Guide to focus on GUI
Refactor beginners-guide.md to center the guide on the DeepLabCut GUI. Updated the main heading and clarified startup wording (python -m deeplabcut) and project-creation workflow. Removed/replaced redundant installation text with a reference to the installation page and commented out an outdated course link. Added notes recommending the PyTorch engine, improved tips for video selection/copying, added guidance for defining bodyparts and multiple individuals, included a video tutorial GIF, and updated the next-steps reference to the GUI-specific manage-project page. Miscellaneous wording and screenshot tweaks for clarity.
* Update install link ref
* Refactor labeling guide and napari workflow
Restructure and clarify the labeling documentation: reorganized headings and sections, consolidated frame selection guidance (automatic vs manual), and added an example workflow, important/tip admonitions, and a "Next steps" pointer to training/evaluation. Updated phrasing for clarity, fixed formatting (headings, links, and image reference), and replaced a quoted YAML note with unquoted text. These changes improve readability and better integrate the napari-DLC labeling workflow with the rest of the docs.
* Clarify GUI config editing in manage-project
Revise the manage-project beginner guide to focus on editing the GUI configuration file. Renamed the section, added a file anchor, and reworded the intro to explain config.yaml as the central project record. Introduced a dedicated "Editing the configuration" and a step-by-step walkthrough that consolidates bodyparts and skeleton guidance (including details on entries/IDs), improved bullets and examples, and added a tip and an HTML comment. Updated the link to the labeling guide (file:labeling-gui) and made minor YAML/frontmatter and formatting fixes.
* Update GUI training/evaluation guide
Revise docs/beginner-guides/Training-Evaluation.md: add RST anchor and restructure headings (Network training, Creating a training set, Starting the training process, Network evaluation, Next steps). Clarify wording (e.g. defaults described as “good” to start), convert tip to RST directive, adjust evaluation section formatting and images, and add reference to the video-analysis GUI section. Also change YAML notes value formatting.
* Refactor video analysis guide for GUI
Rework the beginner video-analysis doc to focus on the GUI workflow: add a file anchor, rename and restructure headings (Analyzing videos with your trained model, Generating labeled videos, Next steps), reorganize and clarify step-by-step instructions, and update copy. Also remove quoting from the frontmatter notes and add a TODO link comment for future cross-references.
* Refine beginner guides and napari usage docs
Update several docs to improve formatting, clarity, and links. Convert inline notes to admonition blocks, fix spacing/indentation around images and note blocks, and bold example items for readability (docs/beginner-guides/beginners-guide.md). Clarify labeling guidance and emphasize labeling across multiple videos; update front-matter note to link to napari plugin docs (docs/beginner-guides/labeling.md). Adjust manage-project wording and next-step link to the labeling guide (docs/beginner-guides/manage-project.md). Add a brief napari usage recommendation with a link to the official napari docs (docs/gui/napari/basic_usage.md). These changes aim to make the GUI/napari workflow and labeling guidance clearer for beginners.
* Tidy beginner-guides
Minor cleanup and reorganization prep for beginner guides: remove an extra closing parenthesis from an image link in Training-Evaluation.md; update frontmatter recommendation from "update" to "move" and simplify notes to indicate moving content to a dedicated GUI section in beginners-guide.md, labeling.md, and manage-project.md; and change "Click on" to "Select" in the labeling workflow example. These edits prepare the docs for consolidation under a GUI section.
* Run mdformat on all
* mdformat all
* Refine PROJECT_GUI docs layout and links
Polish docs/gui/PROJECT_GUI.md: rename headings for clarity ("Interactive Project Manager GUI" → "Project Manager GUI", "Get Started:" → "Getting started", "Video Demo" → "User guide" / "Video demos"); replace inline markdown links with Sphinx {ref} cross-references (napari and beginners guide); add an important note linking to the beginners guide and a tip to "Click on the images!". These changes improve consistency with the docs style and surface the step-by-step GUI guide.
* Fix link in getting started
* mdformat all
* docs: update Docker docs formatting and notes
Refactor docs/docker.md to improve formatting and clarify guidance. Renamed the section heading, convert several inline notes into MyST admonitions (important/note), wrap the napari-deeplabcut install guidance into an important block, and adjust the GPU reminder into an admonition. Minor rework of the mount/example instructions and commented out a redundant labelling note to reduce duplication and improve readability.
* Apply suggestions from code review
Credita to @deruyter92
Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com>
* Separate GPU and cloud bullets; remove link
Split the combined GPU/cloud sentence into two separate bullets for clarity. Removed the technical specs (FAQ) link from the GPU item and removed the troubleshooting wiki link at the bottom. Also cleaned up trailing newline/whitespace.
* Update README: TF backend, dataset links
Simplify and update README text: streamline TensorFlow engine install instructions and add a planned deprecation target (tensorflow backend deprecated in v3.2). Replace prior dataset references with explicit Zenodo DOIs for SuperAnimal-Quadruped and SuperAnimal-TopViewMouse and add a citation to Ye et al. 2024. Clarify that models including AP-10K are available in the API/GUI, and fix minor wording in the funding acknowledgement ("following support").
* Clarify Docker usage and napari limitation
Update docs/docker.md to improve clarity and accuracy of Docker guidance. Reworded introduction to explain that DeepLabCut images are available on DockerHub, that Docker requires a local Docker installation, and that containers provide a reproducible, isolated environment for terminal and Jupyter use. Explicitly state the DeepLabCut GUI is not supported in containers and move/strengthen the important note that the napari-deeplabcut plugin cannot be run inside Docker and must be installed locally (e.g. via pip). Clarified that deeplabcut-docker is a lightweight helper that will not disrupt local TensorFlow/PyTorch or other dependencies, and fixed wording/formatting issues.
* Clarify training dataset creation steps
Add brief guidance in the Training Dataset section to outline key steps (splitting labeled data and creating shuffle folders) so beginners have a clearer workflow before training. Also include a small formatting/whitespace adjustment in the GPU/Colab note.
* Clarify labeling progress and color-scheme docs
Update docs/gui/napari/advanced_usage.md to improve clarity and usability. Reword the labeling progress note to emphasize that the displayed percentage is a rough estimate and that occluded/hidden keypoints are not counted. Expand and simplify instructions for jumping to a bodypart from the color scheme by breaking into explicit steps and clarifying behavior when the bodypart is already visible. Minor wording/formatting cleanups for readability.
* Relocate demo video link; cleanup whitespace
Move the demo video link to appear before the warning block and tidy up minor whitespace/trailing-space issues in docs/gui/napari/basic_usage.md. These are formatting-only changes and do not alter substantive content.
* Update installation.md
* Fix docs formatting, typos and image tags
Clean up documentation across multiple files: add quotes to image src attributes, correct typos (e.g. "oher"→"other", "built"→"build", "compartimentalized"→"compartmentalized"), normalize admonition/code-block syntax, adjust cross-reference/link syntax, and make minor wording/grammar improvements. Affected files include docs/README.md, UseOverviewGuide.md, beginner-guides/*, course.md, docker.md, gui/PROJECT_GUI.md, and installation.md.
* Fix typo
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Docs: add external data import recipe
Add a new recipe (docs/recipes/external_data_import.md) describing how to convert externally annotated data (CSV/H5) into the DeepLabCut format and how to merge multiple datasets (uses deeplabcut.convertcsv2h5).
Update docs/installation.md to use a local Sphinx cross-reference for the Conda section, add the section anchor (sec:installation-using-conda), and fix the uv pip install line to quote the extras ('.[gui,modelzoo,tf]') to avoid shell globbing. Also add a file anchor (file:hardware-requirements)= to docs/recipes/TechHardware.md for cross-references.
* Refine wording and links in user guides
Small editorial updates to user documentation for clarity and better Sphinx linking. In docs/UseOverviewGuide.md: use inline `help` function wording, rephrase 'jump back in' to 'resume work' for clarity, and replace the 'Option 3' Project Manager GUI link with a Sphinx {ref} cross-reference. In docs/beginner-guides/beginners-guide.md: change the tip to 'Avoid spaces' and emphasize the scorer/experimenter field. These changes improve readability and consistency.
* Convert internal docs links to Sphinx refs
Replace Markdown-style internal links with Sphinx/MyST cross-reference syntax and adjust anchors/formatting for proper rendering. Updated docs/UseOverviewGuide.md to use {ref}`...` and change the target anchor to sec:important-info-regd-usage, and fixed the user-guide list formatting. Updated docs/gui/PROJECT_GUI.md to use {ref}`Read more here <sec:important-info-regd-usage>` in two places.
* chore(metadata): update docs/notebooks metadata
* Update course.md
---------
Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Move beginners-guide into the GUI workflow sections in _toc.yml so it appears under the GUI chapter, and update the document heading in docs/beginner-guides/beginners-guide.md from "Project Manager GUI - Step by step" to "Setting up a new project" for clearer wording.
Docs audit [April 2026]: Merge GUI guide entry and update guide title
* Fix conflicts * Fix conflicts * Fix conflicts * Fix conflicts * Revert lost changes * Fix TOC entry and update guide title Move beginners-guide into the GUI workflow sections in _toc.yml so it appears under the GUI chapter, and update the document heading in docs/beginner-guides/beginners-guide.md from "Project Manager GUI - Step by step" to "Setting up a new project" for clearer wording.
Add docs/dlc-utils/index.md describing DeepLabCut-Utils community contributions, example scripts, tool links, maintainers, and legacy notes, ported over from https://github.com/DeepLabCut/DLCutils/blob/master/README.md. Also update _toc.yml to include the new page
Add docs/dlc-utils/XROMM/usage.md describing the local 3-repo XROMM workflow with XROMM_DLCTools, DeepLabCut, and xmalab. Covers recommended sibling checkout layout, DeepLabCut's role in the workflow, developer setup (uv sync --group dev/dlc), example validation and end-to-end commands to run the baseline harness, and a compatibility note about lite-mode importability when GUI deps are unavailable. Co-Authored-By: homfunc_ <4338462+homfunc@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Documentation: Migrate dlc-utils docs page
Documentation: Add XROMM+DeepLabCut local integration guide
Update docker documentation
Docs [April 2026] Update TensorFlow documentation
…ooks Docs audit [April 2026] Small updates to demo notebooks
…abCut into docs/audit-2026
Update GitHub Actions docs_and_notebooks_checks workflow to only consider changed .md and .ipynb files that live under docs/, tools/, examples/COLAB/, or examples/JUPYTER/. This adds an extra grep filter before sorting, so checks are not triggered by markdown/notebook files outside the intended directories. The output still writes the unique changed paths to tmp/docs_nb_checks/changed_docs.txt. This is to ignore README,md at the top leve, which cannot have a frontmatter, but include all else
Introduce a force_overwrite_notes option to allow scanned notes to replace existing CSV notes. Updated merged_row to accept a force_overwrite_notes flag (default False) and to prefer scanned notes when it's set; added propagation through export_csv; and added a CLI flag --force-overwrite-notes with help text. Conflict handling now prints a warning and indicates when force overwrite is used. This enables bulk updates of notes while preserving the previous default behavior of keeping manually curated notes.
Replace markdown-style links with Sphinx {ref} cross-reference roles in docs/UseOverviewGuide.md and docs/maDLC_UserGuide.md to ensure proper Sphinx rendering. Also fix a malformed bracket/URL in docs/pytorch/architectures.md for the Ye et al. citation link.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documentation update
Introduces all the latest documentation updating efforts.
For review, please check the individual PRs listed below:
Content update
Require full approval before merging here/can be merged and reviewed on their own:
Metadata/audit
CI/Tooling