Visualization fixes, clean-up and coverage improvements - #3447
Open
C-Achard wants to merge 21 commits into
Open
Conversation
Add pre-validation in `plot_evaluation_results` to correctly split and compare ground-truth vs prediction individuals and bodyparts before reshaping arrays. When counts or bodypart sets mismatch, the function now logs a warning and skips that image instead of proceeding with invalid dimensions.
Adds a new test module for `deeplabcut.utils.visualization` focused on evaluation plotting behavior. The tests cover single-animal label handling across scorers, coordinate arrangement for both plotting modes, mismatch/malformed-data skip paths with expected messages, invalid mode validation, output filename conventions in `save_labeled_frame`, and bounding-box linestyle behavior in `make_multianimal_labeled_image`.
Tightened `plot_evaluation_results` by constraining `mode` with a `Literal` type and adding an explicit runtime `ValueError` for unsupported values. The plotting branch logic now treats non-`bodypart` mode consistently as `individual`, and warning messages for individual-count/reshape issues were rewritten to be clearer and more specific. Tests were updated to match the new warning text, and a matplotlib cleanup fixture was added to close figures after each test.
Use a row-scoped `plot_unique_for_row` flag in `plot_evaluation_results` so a reshape failure only disables unique-bodypart plotting for the current image. This prevents one bad row from mutating `plot_unique_bodyparts` and incorrectly skipping unique overlays for all subsequent rows.
Improve `visualization.py` with clearer type aliases, small cleanup of unused variables, and better API messaging (including a specific `ValueError` for missing `individuals`). In evaluation plotting, tighten consistency checks by comparing ordered bodypart/individual labels, skip mismatched rows with explicit warnings, simplify auto bounding-box color handling, and wrap plotting in `try/finally` to always close figures and avoid leaked matplotlib resources.
Removes the duplicated legacy license/docstring block from `deeplabcut/utils/visualization.py` and adds the standard DeepLabCut license header comments to `tests/utils/test_visualization.py` for consistent file metadata across the codebase.
Refactors unique-bodypart extraction in `plot_evaluation_results` to index the `single` individual directly and validate bodypart alignment between ground truth and predictions before reshaping. This avoids reshape failures from mismatched bodyparts and adds a clearer warning path when unique data is inconsistent.
This updates `visualization.py` to use consistent frame-dimension extraction (`shape[:2]`) across image plotting paths, making handling cleaner for both grayscale and color frames. It also renames `BoundingBoxesColor` to `BoundingBoxColor` for clearer typing and fixes a typo in the individuals-coloring error message.
Removes the evaluation-plot guard that skipped images when multi-individual ground-truth and prediction labels differed in order/content. This keeps visualization running for those cases while retaining existing count and bodypart mismatch checks.
Update evaluation plotting to skip unique-bodypart rendering when the "single" subset is empty before indexing scorer/model columns. This avoids downstream mismatch/reshape errors and keeps the existing warning-based fallback behavior for mismatched bodyparts or invalid shapes.
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes evaluation visualization for differing ground-truth and prediction labels and expands visualization coverage.
Changes:
- Separately validates and reshapes ground-truth and prediction labels.
- Improves plot validation, grayscale handling, and figure cleanup.
- Adds regression and visualization tests.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
deeplabcut/utils/visualization.py |
Updates plotting validation, reshaping, typing, and cleanup. |
tests/utils/test_visualization.py |
Adds visualization regression and behavior tests. |
Suppressed comments (1)
deeplabcut/utils/visualization.py:620
- This second plotting call restarts color lookup at index 0. Consequently, unique bodyparts reuse regular-bodypart colors in bodypart mode, and in individual mode they reuse the first individual's color even though the
+1entry is reserved for them. Pass an offset colormap/callable starting afterbodypartsorindividualsso the reserved colors are actually used.
colors=colors,
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Add a `color_offset` parameter to `make_multianimal_labeled_image` so callers can shift colormap indexing. In evaluation plotting, only swap prediction/ground-truth axes for unique rows in `bodypart` mode and apply a color offset there, keeping unique overlays aligned and preventing color reuse conflicts.
Update `plot_evaluation_results` to use the individual count as the colormap offset in non-bodypart mode, instead of always using `0`. This prevents color index overlap and keeps prediction colors aligned with the selected plotting mode. Also document the `color_offset` parameter in `make_multianimal_labeled_image` for clarity.
Extend `evaluation_dataframe_factory` to generate GT/pred columns for unique bodyparts and expose them in metadata. Add a new parametrized test that verifies `plot_evaluation_results` arranges unique bodyparts correctly in both `bodypart` and `individual` modes, including expected coordinate/probability shapes and color offset handling.
deruyter92
approved these changes
Aug 18, 2026
deruyter92
left a comment
Collaborator
There was a problem hiding this comment.
Great work, definitely needed.
All changes look good, I could not find introduced regressions. Two general remarks:
- The tests are now mostly mocking the actual plotting, should we add one or two tests for
make_multi_animal_labeled_image? If we keep it small should be fine for CI right? - We currently are assuming valid GT and non-empty dataframes. Should we add validation?
Suggestions for previously existing issues:
Line 229
one remaining h, w, numcolors = np.shape(frame) can be changed to h, w = np.shape(frame)[:2] like you did elsewhere
Lines 396-398
This can be simplified with a lookup dict that is built once
for i in trange(len(ic)):
filename = ic.files[i]
ind = images_list.index(filename)Lines 410-412
Handle the png replacement a bit more robust, e.g. :
stem = Path(filename).stem
out_name = f"{stem}_{color_by}.png"
Replace `print`-based warnings in `plot_evaluation_results` with module logger warnings so diagnostics are captured consistently. Update visualization tests to assert warning messages via `caplog` and split/expand multianimal image tests to verify bounding-box linestyle behavior and confidence-based keypoint marker styling.
Reworked `make_labeled_images_from_dataframe` for clarity and maintainability by splitting color selection, bone index construction, and image shape checks into dedicated helpers. The update preserves existing behavior (including duplicate filename indexing), simplifies skeleton handling, and streamlines figure saving and drawing paths for same-size vs mixed-size image collections.
Replace the grayscale/color conditional in `plot_and_save_labeled_frame` with `np.shape(frame)[:2]` so height and width are extracted uniformly for both image types. This removes redundant branching and the unused `numcolors` variable while preserving plotting behavior.
Update evaluation plotting to set `ax.set_ylim(h, 0)` directly instead of combining `set_ylim(0, h)` with `invert_yaxis()`. This keeps image-coordinate orientation explicit and avoids axis inversion side effects when overlaying points.
AlexEMG
approved these changes
Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
Fixes evaluation plotting when ground-truth and prediction DataFrames use different labels, and adds coverage for the visualization utilities.
Closes #3446
Changes
Motivation
The merged DataFrame previously treated labels such as
animalandindividual0as two individuals, causing valid single-animal evaluation data to fail reshaping and skip visualization.