Skip to content

Keep MM validation sample counts intact - #3200

Open
dajiaohuang wants to merge 1 commit into
activeloopai:mainfrom
dajiaohuang:fix-mm-validation-dataset-length
Open

dajiaohuang wants to merge 1 commit into
activeloopai:mainfrom
dajiaohuang:fix-mm-validation-dataset-length

Conversation

@dajiaohuang

Copy link
Copy Markdown

Fixes #3199.

MMDet/MMSeg evaluator wrappers count samples, but their validation __len__ methods divided the sample count by batch size and returned the result as a sample count. The actual validation DataLoader uses the full underlying dataset when drop_last=False, so evaluators could load only part of the ground truth and omit predictions. Return the full validation dataset length; drop behavior stays with the DataLoader configuration.

Validation:

  • Focused pytest regressions: 2 passed, exercising the actual __len__ methods from both source files with batch_size=2.
  • Black check: passed.
  • git diff --check: passed.

No full MMDet/MMSeg runtime is configured on this host.

@coderabbitai

coderabbitai Bot commented Oct 2, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 12 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 8 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: eaec5913-2ad1-4617-b2ef-f1a103b1de00

📥 Commits

Reviewing files that changed from the base of the PR and between f432041 and 4d07920.

📒 Files selected for processing (3)
  • python/deeplake/integrations/mmdet/mmdet_dataset_.py
  • python/deeplake/integrations/mmseg/mmseg_dataset_.py
  • python/tests/test_mm_validation_dataset_length.py
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@khustup2

khustup2 commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

This changes behavior in a maintenance-only integration (MM/Labelbox/COCO) rather than fixing a defect, and it has no owner review or integration run behind it. Holding it: it needs the integration owner to decide whether the new behavior is wanted. No action needed from you now.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MM validation wrappers truncate ground truth when batch_size exceeds one

2 participants