Skip to content

Releases: DeepLabCut/DeepLabCut

v3.0.2

Choose a tag to compare

@deruyter92 deruyter92 released this 23 Sep 10:46
4dcb9a7

DeepLabCut 3.0.2

DeepLabCut 3.0.2 is a patch release bringing several PyTorch workflows fixes,
improved config handling and export reliability, broadened compatibility for newer
NumPy / Matplotlib versions, and adding a machine-readable docs/API index for
agent-assisted interactions with the library.

Highlights

  • Forward-compatibility work for NumPy 2, pandas 3 CoW, and Matplotlib 3.9,
    including releasing core NumPy / Matplotlib upper bounds (#3420, #3498,
    #3502, #3508, #3499, #3445, #3394).
  • More reliable COCO training: model_cfg.metadata is authoritative, with
    fail-fast validation against annotation JSON (#3435).
  • Safer legacy project configs: empty / null top-level YAML fields are treated
    as unset; ignore_empty is removed (#3439).
  • Normalized export_model pickles for downstream / real-time consumers (#3473).
  • Detector null thresholds resolve to field defaults instead of crashing
    (#3470, #3475).
  • PyTorch analyze_videos / auto-track honor the overwrite argument instead of
    hard-coding overwrite=False (#3514).
  • Public LLM-friendly knowledge index (llms.txt, docs.jsonl, api.jsonl)
    plus hardening follow-up and AGENTS.md (#3453, #3494, #3483).

Important changes

NumPy / Matplotlib dependency bounds (#3420, #3498, #3502, #3508, #3499)

Core installs no longer pin numpy<2 or a Matplotlib upper bound. Also added
numba>=0.60 and an explicit OpenCV dependency. TensorFlow extras still keep a
NumPy <2 ceiling where the corresponding TensorFlow build requires it.

  • #3420 — Prepare for NumPy 2 forward compatibility (including dense SVD for
    tracklet ranking instead of randomized interpolative SVD)
  • #3498 — Fix broken NumPy 2.0 compatibility in crossvalutils and
    predict_multianimal
  • #3502 — NumPy 2 + numba: take the real part of eig eigenvectors in
    EllipseFitter
  • #3508 — Follow-up NumPy 2 fixes (PyTorch snapshot metric loading /
    Assembly.affinity typing; raise numba floor; declare OpenCV)
  • #3499 — Remove core NumPy and Matplotlib upper bounds (NumPy <2 remains
    only where TensorFlow extras still require it; exclude broken Matplotlib
    3.7 / 3.7.1)
  • #3445 — Guard pandas 3 Copy-on-Write read-only .to_numpy() / .values
    mutations
  • #3394 — Matplotlib v3.9 compatibility
  • #3509 — Tighten strict canary CI handling (strict_cells for same-repo
    pushes/PRs; fork PRs stay non-blocking)

COCOLoader (#3435)

bodyparts / individuals capacity now come from model_cfg.metadata, matching
DLCLoader. Construction validates train.json and test.json and raises
ValueError if an image exceeds configured capacity or bodyparts do not match.
Projects that previously relied on capacity inferred only from the busiest
training image should set max_individuals (and bodyparts) explicitly.

ProjectConfig empty / null fields (#3439)

Empty top-level YAML fields are treated as unset and emit an actionable warning.
DLCBaseConfig.from_yaml(..., ignore_empty=...) no longer accepts ignore_empty
(breaking for that optional parameter). Lower-level read_config(..., ignore_empty=...)
deprecates the argument.

Exported models (#3473)

export_model now calls PoseConfig.to_dict(normalize=True), so pickles store
primitives (Path → str, enums → str, etc.). Prefer re-exporting models for
consumers that load the pickled config.

overwrite in PyTorch video analysis (#3514)

analyze_videos and the auto-track path now forward the caller’s overwrite
flag instead of always behaving as overwrite=False. Existing prediction files
are skipped only when overwrite is false.

Features and enhancements

  • #3436 — Add detector_eval_interval to train_from_coco
  • #3475 — Add DefaultIfNone config validator (null → field default)
  • #3481 — Harden wandb logger error handling (validate before wandb.init)

Agentic interaction

Coding agents and LLM tools can now discover DeepLabCut knowledge without
scraping HTML.

CI builds and deploys a structured index with the docs site:

  • llms.txt — llmstxt.org-style entry point at the docs
    root (published here
    after deploy)
  • knowledge/manifest.json — which API versions are indexed and which is current
  • knowledge/main/docs.jsonl — one JSON object per user-docs page and heading
  • knowledge/<version>/api.jsonl — one JSON object per documented API symbol

Also added: AGENTS.md,
shared guidance for humans and coding agents.

  • #3453 — Knowledge indexing generator + CI deploy (tools/knowledge_indexing)
  • #3494 — Follow-up hardening for the LLM knowledge index (dependency bounds,
    stable vs main API links in llms.txt, safer skip/empty-dir publish
    behavior, anchor/regression fixes)
  • #3483 — Add AGENTS.md for AI-assisted development

Bug fixes

  • #3419 — Skip TensorFlow-backed tests cleanly when the tf extra is absent
  • #3435 — Fix COCOLoader: treat model_cfg.metadata / max_individuals as
    authoritative instead of inferring solely from train.json
  • #3439 — Update None-type validation for legacy ProjectConfig templates;
    remove ignore_empty
  • #3441 — GUI config editor crash + shared deferred reload on DefaultTab
  • #3447 — Evaluation visualization fixes, clean-up, and coverage improvements
  • #3449 — Fix frame extraction failure with pathlib video selections
  • #3452 — Fix incorrect SuperAnimal HumanBody rename mapping in ModelZoo /
    Hugging Face download
  • #3467 — Fix faulty str / Path conversions
  • #3468 — Fix latent bugs and misleading errors in GUI create_videos
  • #3469 — Fix WandbLoggerConfig so extra wandb kwargs validate (scoop into
    wandb_kwargs)
  • #3470 — Fix Faster R-CNN detector inference crashing on
    box_score_thresh: null
  • #3473 — Fix export_model dumping unnormalized objects in pickle
  • #3480 — Fix GUI tests: use config_path after rename
  • #3484 — Fix tools imports under a plain pytest invocation
  • #3486 — Docs: fix dead Model Zoo link in examples/README.md
  • #3514 — Fix hard-coded overwrite=False in PyTorch analyze_videos /
    auto-track path

Please let us know if you face any issues with this release, especially related
to the latest forward compatibility efforts, we appreciate any issues or reports!


New Contributors

Full Changelog: v3.0.1...3.0.2

v3.0.1

Choose a tag to compare

@deruyter92 deruyter92 released this 28 Jul 06:25

DeepLabCut 3.0.1 - a stability release with a major documentation overhaul and a new structured configuration system.

Highlights

  • Comprehensive documentation audit — The user guide has been thoroughly reviewed, updated, and improved. Please check out:

    • Our updated user guide, which is restructured for better readability and updated to reflect the latest API and features.
    • Our brand-new developer documentation site, which provides full API references, architecture guides, and developer onboarding material.
  • Structured & validated configurations — The config system has been migrated to typed Pydantic models. Existing config.yaml files remain fully compatible, but configs are now schema-validated on load with clear error messages for invalid entries. Some user-benefits include:

    • ProjectConfig: load and save your config.yaml with full schema validation — typos, wrong types, or invalid values are caught immediately with clear error messages.
    • Pretty-printing: call cfg.print() or pretty_print(cfg) to inspect any config in a readable tree.
    • Change-tracking: DLCVersionedConfig tracks which fields have been modified since the config was loaded from YAML, so when saving back to disk, log_changes() can report exactly what was changed.
    • Dict-like compatibility: existing code that accesses config keys like a dict still works.
    • Auto-aliasing of old field names: renamed keys are silently mapped, so older config files load without manual edits.

What's Changed

Full Changelog: v3.0.0...v3.0.1

DeepLabCut - 3.0

Choose a tag to compare

@deruyter92 deruyter92 released this 22 May 05:42
cdc8724

Familiar workflows, modern foundations, better performance

DeepLabCut 3.0 introduces a PyTorch-first training and inference stack while keeping the core DeepLabCut workflow familiar.
Projects still follow the same labeling, training, evaluation, and video-analysis pipeline used throughout the 2.x series, but the underlying engine has been substantially modernized.

For users who have already been following the release candidates, many of these changes will already feel familiar.
DeepLabCut 3.0 consolidates these incremental changes into a stable release.

Increased model performance and speed

Benchmark table Openfield benchmark results
Pose estimation performance of the 3.0 PyTorch models compared against previous TensorFlow models on the DeepLabCut Openfield dataset (see PR #2613); RMSE: root mean squared error. *Values from Mathis et al. 2018.
Trimice dataset Speed comparison: PyTorch vs TensorFlow
Speed of the current PyTorch implementation (ResNet50) compared to the TensorFlow implementation. Results were obtained using a NVIDIA GeForce RTX 2080 Ti with CUDA 12.2 on the DeepLabCut Trimice dataset.
BUCTD benchmarks
Comparison of the new BUCTD model architectures with DLCRNet and DEKR on the Marmoset, Fish and Trimice dataset. From Zhou et al., ICCV 2023.

The journey to 3.0

A quick recap of some of the major milestones leading to this release:

  • #2613 - Full initial PyTorch backend implementation
  • #2952 - New bottom-up conditional top-down (BUCTD) model architecture
  • #2795 - New RTMPose top-down architecture
  • #2868 - Updated notebooks and Colab examples for PyTorch workflows
  • #2804 - PyTorch model export

And more, find the full PR reference on GitHub!

Credits to @n-poulsen, @jeylau, @maximpavliv, @MMathisLab, @yeshaokai, @LucZot, @AlexEMG, @arashsm79 and many other contributors.

Notable features in 3.0.0

PyTorch-first, TensorFlow-compatible

DeepLabCut 3.0 adds a new PyTorch backend while retaining TensorFlow support for legacy workflows.
Project management remains the same and labeled datasets remain compatible.
PyTorch models can be trained alongside previous TensorFlow models on the same train/test splits for direct benchmarking and comparison.

Expanded architecture support

DeepLabCut 3.0 significantly broadens the supported model ecosystem beyond the classic ResNet-based workflows. The PyTorch stack includes:

  • ResNet and HRNet backbones
  • Bottom-up multi-animal approaches such as DEKR and PAF/DLCRNet variants
  • Top-down detector-plus-pose pipelines including RTMPose
  • Hybrid architectures such as BUCTD and CTD variants
  • SuperAnimal-related pretrained workflows

The documentation now includes dedicated architecture guides to help users choose models based on scene complexity and experimental needs.

Flexible PyTorch training configuration

The PyTorch engine introduces a modern training stack with expanded augmentation options, training schedules, device management, and model architectures. For each training run, the settings are stored in a pytorch_config.yaml, enabling easy reproducibility.

Improved interoperability

The new PyTorch data pipeline introduces loaders for both standard DeepLabCut projects and COCO-style datasets, making it easier to integrate DeepLabCut with broader computer-vision workflows and external annotation formats.

Model Zoo and SuperAnimal workflows

DeepLabCut 3.0 continues to expand the Model Zoo and SuperAnimal ecosystem, making pretrained models and transfer learning more accessible.
Colab notebooks and updated GUI tooling make it easier to experiment with modern architectures without extensive setup. (see the documentation)

Modernized installation and packaging

The project has been moved to a newer packaging system, and is now based around pyproject.toml. This enables the use of modern package-managers & dependency resolvers, such as uv or pdm.
Users can still install only the components they require, be it GUI support, TensorFlow compatibility, ModelZoo features, and optional experimental integrations.

Labeling GUI

DeepLabCut 3.0 is shipped with a new release of the napari-deeplabcut plugin. Our napari-based labeling GUI has undergone a major internal re-write and modernization: while preserving familiar UI and the DeepLabCut workflow, the update substantially improves stability, data handling, usability, visualization, and annotation workflows, now with automated point tracking for faster labeling. See the release notes to find out about all improvements.

Upcoming: refreshed documentation

We have updated and streamlined the documentation, with a focus on clarity and up-to-date information in core areas (installation, getting started guides, and more).
Expect the documentation to continue evolving soon after the release!

A major transition

The jump from the final DeepLabCut 2.x releases to the current codebase is best understood as a transition to more recent Python & deep learning ecosystems rather than a routine update.
Taken together, the PyTorch backend, broader architecture support, ModelZoo integration, packaging modernization, updated labeling GUI, and documentation improvements represent a major evolution of DeepLabCut, which we are happy to release as 3.0.

Closing thoughts

We hope you enjoy this new version, and we aim to keep sharing many exciting improvements in the future in all areas, be it performance and speed, codebase quality improvements, foundation models integration, user experience and documentation.


Changelog since 3.0.0rc14

  • DLC-live-GUI docs : Install from PyPI pre-release (#3238) by @C-Achard
  • Apply linting to entire codebase & add CI workflow to check linting (#3216) by @deruyter92
  • Bump requests from 2.32.5 to 2.33.0 (#3259) by @app/dependabot
  • Refactor Analyze Videos tab (#3268) by @C-Achard
  • Consolidate test workflow infrastructure in CI (#3254) by @C-Achard
  • Move protobuf requirement to pyproject.toml (#3235) by @C-Achard
  • Use pinned ffmpeg version in CI (#3276) by @C-Achard
  • Docs versioning: Add glob support, better validation and reporting (#3278) by @C-Achard
  • Bump cryptography from 46.0.5 to 46.0.7 (#3277) by @app/dependabot
  • Fix failing local Windows tests due to ruamel parsing (#3275) by @C-Achard
  • Update & de-duplicate skeleton builder (#3258) by @C-Achard
  • Bump pygments from 2.19.2 to 2.20.0 (#3262) by @app/dependabot
  • update conda yaml: add pyside6 as conda dependency (#3253) by @deruyter92
  • Bump pyasn1 from 0.6.2 to 0.6.3 (#3249) by @app/dependabot
  • Fix SuperAnimal / pretrained load for RTMPose: implement convert_weights on RTMCCHead (#3270) by @deruyter92
  • Run update check in a separate thread (#3234) by @mikhailmikhalchuk
  • Update napari-DLC docs for refactor (#3280) by @C-Achard
  • Refactor/predict multianimal (#3220) by @juan-cobos
  • Fix incorrect MultiLevel construction in outlier_frames.compute_deviations (#3247) by @C-Achard
  • Bump pillow from 12.1.1 to 12.2.0 (#3283) by @app/dependabot
  • Bump pytest from 9.0.2 to 9.0.3 (#3284) by @app/dependabot
  • CircleCI: disable hugginface xet (#3316) by @deruyter92
  • Update and diversify TensorFlow optional installations. (#3292) by @deruyter92
  • Bump urllib3 from 2.6.3 to 2.7.0 (#3325) by @app/dependabot
  • Bump gitpython from 3.1.47 to 3.1.50 (#3322) by @app/dependabot
  • make CTD GenerativeSampler visibility-aware (#3305) by @deruyter92
  • Add isatty method to StreamWriter + eval GUI fix (#3314) by @yyexela
  • Add conditional replacement of @torch.inference_mode for inference on AMD DirectML GPUs (#3295) by @deruyter92
  • Robustness fix: Annotation file not dropping likelihood column if present from machine labels (#3323) by @C-Achard
  • Remove trailing comma in models_to_framework.json (#3330) by @C-Achard
  • update list_videos_in_folder (#3303) by @deruyter92
  • Improve TrainingDatasetMetadata and get_shuffle_engine for incomplete projects (#3313) by @deruyter92
  • update RTMPose SimCCPredictor: expose apply_softmax and fix visibility thresholding (#3306) by @deruyter92
  • GUI: Add "Generate debug log" action (#3328) by @C-Achard
  • [Docker 1] Simplify and modernize Dockerfile (#3290) by @deruyter92
  • [Docker 2] Update deeplabcut-docker package (#3291) by @deruyter92
  • Add additional drop_likelihood_columns guards (#3333) by @C-Achard
  • Resolve inconsistent parameter names via aliasing + depreca...
Read more

v3.0.0rc14

Choose a tag to compare

@C-Achard C-Achard released this 23 Mar 21:21
9c74d49

Highlights

  • New installation procedure via pyproject.toml : easier, more modern installation. Enables uv, pdm and other modern package managers use.

  • Many fixes and improvements:

    • PyTorch speed improvements
    • CI improvements

Note: these changes are based on all merges since rc10

What's Changed

New Contributors

Full Changelog: v3.0.0rc10...v3.0.0rc14

v3.0.0rc10

Choose a tag to compare

@MMathisLab MMathisLab released this 01 Jul 04:04
d11f093

What's Changed

Read more

v2.3.11 🐛 fixes, package maintenance (TF only)

Choose a tag to compare

@MMathisLab MMathisLab released this 18 Feb 09:06
fa2ebee

What's Changed

New Contributors

@maximpavliv joined the software team!

Funding includes ChanZuckerberg Initiative's Essential Open Source Software for Science. Thanks!

Full Changelog: v2.3.10...v2.3.11

v2.3.10

Choose a tag to compare

@MMathisLab MMathisLab released this 07 Jun 16:32
fa89f7f

What's Changed

New Contributors

Full Changelog: v2.3.9...v2.9.10

Updates and bug fixes

Choose a tag to compare

@AlexEMG AlexEMG released this 08 Feb 18:42
a2c9b03

What's Changed

Full Changelog: v2.3.8...v2.3.9

Expanded GUI features for the DLC ModelZoo

Choose a tag to compare

@AlexEMG AlexEMG released this 07 Nov 20:27
3ba648f

What's Changed

  • better augmentation for maDLC: Fix fliplr augmentation for multi animal 2 by @biol-jsh in #2043
  • Improved transfer learning for DLC ModelZoo by @yeshaokai in #2389
  • Improvements to the model zoo's GUI tab by @jeylau in #2431
  • "deeplabcut.filterpredictions" and "deeplabcut.analyze_skeleton" return mappings by @rahulporuri in #1908
  • Let user change the search window size during corner detection refinement by @jeylau in #2414
  • Install DLC from PyPI rather than its local version in M1 Macs by @jeylau in #2416
  • Correct link to platform specifications page by @molecules in #2423
  • Automatically check for napari-deeplabcut updates (in addition to DLC) by @jeylau in #2422
  • Smartly restore pretrained model weights by @jeylau in #2426
  • Fix video adaptation by @jeylau in #2436

New Contributors

Funding includes ChanZuckerberg Initiative's Essential Open Source Software for Science. Thanks!

Full Changelog: v2.3.7...v2.3.8

Minor fixes

Choose a tag to compare

@AlexEMG AlexEMG released this 10 Oct 14:27
7d7a57b

What's Changed

Full Changelog: v2.3.6...v2.3.7

Funding includes ChanZuckerberg Initiative's Essential Open Source Software for Science. Thanks!