Problem
In some checkerboard image pairs, both cameras successfully detect a full grid of corners, and the corner overlays can each look visually plausible on their own, but the within-pair indexing direction is inconsistent across cameras.
Concretely, one camera may index a given checkerboard axis in one direction, while the other camera indexes the same physical axis in the opposite direction. This can happen even when corner detection looks good by eye.
If those inconsistent pairs are included in stereo calibration, the resulting stereo extrinsics can become badly distorted.
What we observed
In our case:
the original stereo calibration produced a checkerboard reconstruction with strong anisotropy,
many individual checkerboard reconstructions looked implausible in 3D,
screening out within-pair index-direction mismatches and recalibrating stereo parameters fixed the issue,
after recalibration, retained checkerboards became close to isotropic and planar across the dataset.
In other words, the calibration itself was not necessarily failing because the checkerboard was poorly detected; it was failing because some image pairs disagreed about the corner ordering.
Why this seems worth guarding against
The current workflow already expects users to manually inspect checkerboard detections and resulting 3D geometry. However, this specific failure mode can be subtle:
both corner detections may look reasonable,
the bad pairs may be a minority,
the resulting calibration can still complete successfully,
the failure only becomes obvious downstream when inspecting the triangulated checkerboard or stereo parameters.
So this seems like a good candidate for an explicit pre-calibration guard.
Suggested feature
Add an optional checkerboard-order consistency screening step before stereo calibration.
For each matched checkerboard image pair:
detect the ordered corner grid in each camera,
choose the checkerboard axis with the larger image-space span,
compare the direction of that axis between the two cameras,
flag pairs whose ordering directions disagree.
Proposed modes
A simple interface could be something like:
order_guard_mode="report" — detect and report inconsistent pairs, but do not exclude them automatically,
order_guard_mode="exclude" — exclude inconsistent pairs before stereo calibration.
Useful outputs
It would also help to save:
a CSV listing kept/rejected frame pairs,
a short text summary,
optional diagnostic visualizations for rejected pairs.
Why I think dominant-axis inference matters
A hardcoded left-right test is probably too brittle, because some users may orient the checkerboard so that its more informative indexed axis is mostly vertical in the image. The guard should therefore infer which checkerboard axis is best resolved in the image pair rather than assuming a fixed horizontal axis.
Question
Would a feature like this be useful if implemented as a small optional guard in the 3D calibration pipeline?
If helpful, I can also open a PR with a minimal implementation using report and exclude modes plus CSV/summary outputs.
Problem
In some checkerboard image pairs, both cameras successfully detect a full grid of corners, and the corner overlays can each look visually plausible on their own, but the within-pair indexing direction is inconsistent across cameras.
Concretely, one camera may index a given checkerboard axis in one direction, while the other camera indexes the same physical axis in the opposite direction. This can happen even when corner detection looks good by eye.
If those inconsistent pairs are included in stereo calibration, the resulting stereo extrinsics can become badly distorted.
What we observed
In our case:
the original stereo calibration produced a checkerboard reconstruction with strong anisotropy,
many individual checkerboard reconstructions looked implausible in 3D,
screening out within-pair index-direction mismatches and recalibrating stereo parameters fixed the issue,
after recalibration, retained checkerboards became close to isotropic and planar across the dataset.
In other words, the calibration itself was not necessarily failing because the checkerboard was poorly detected; it was failing because some image pairs disagreed about the corner ordering.
Why this seems worth guarding against
The current workflow already expects users to manually inspect checkerboard detections and resulting 3D geometry. However, this specific failure mode can be subtle:
both corner detections may look reasonable,
the bad pairs may be a minority,
the resulting calibration can still complete successfully,
the failure only becomes obvious downstream when inspecting the triangulated checkerboard or stereo parameters.
So this seems like a good candidate for an explicit pre-calibration guard.
Suggested feature
Add an optional checkerboard-order consistency screening step before stereo calibration.
For each matched checkerboard image pair:
detect the ordered corner grid in each camera,
choose the checkerboard axis with the larger image-space span,
compare the direction of that axis between the two cameras,
flag pairs whose ordering directions disagree.
Proposed modes
A simple interface could be something like:
order_guard_mode="report" — detect and report inconsistent pairs, but do not exclude them automatically,
order_guard_mode="exclude" — exclude inconsistent pairs before stereo calibration.
Useful outputs
It would also help to save:
a CSV listing kept/rejected frame pairs,
a short text summary,
optional diagnostic visualizations for rejected pairs.
Why I think dominant-axis inference matters
A hardcoded left-right test is probably too brittle, because some users may orient the checkerboard so that its more informative indexed axis is mostly vertical in the image. The guard should therefore infer which checkerboard axis is best resolved in the image pair rather than assuming a fixed horizontal axis.
Question
Would a feature like this be useful if implemented as a small optional guard in the 3D calibration pipeline?
If helpful, I can also open a PR with a minimal implementation using report and exclude modes plus CSV/summary outputs.