Hello!

I am trying to train a model on rat cylinder test videos, and the eventual goal is to score forepaw cylinder wall contacts automatically. The videos are recorded with a mirror, so each frame contains both the real rat and its mirror reflection. Here is a screenshot of one of the videos.

I initially tried two different project designs with small number of frames(69) just as a preliminary experiment to compare the two approaches:

1. Label the following body parts as a single animal:
   real_nose
   real_left_forepaw
   real_right_forepaw
   real_tailbase
   mirror_nose
   mirror_left_forepaw
   mirror_right_forepaw
   mirror_tailbase

2. Treat the real rat and mirror rat as two individuals and used multi-animal training:
   Individual 1 = real rat
   Individual 2 = mirror rat

Body parts:
nose
left_forepaw
right_forepaw
tailbase

After training the two models with the exact same frames and number of epochs, the multi-animal model perfromed much better, with train and test rmse about 14, whereas for the single animal model the rmses are around 200. However, the multi-animal model encountered problems at analyze_videos step: when running the code shell in colab, I consistently get the warning /usr/local/lib/python3.12/dist-packages/deeplabcut/pose_estimation_pytorch/data/postprocessor.py:507:
RuntimeWarning: invalid value encountered in cast for all my videos.
As a result, there is a large numbers of blanks in the output CSV. And correspond to that, in the created labeled videos, many frames contain predictions for only the real rat or only the mirror rat and some frames contain neither. Lowering the pcutoff to 0, disable auto_track, changing tracking method or changing batch size did not help.

Is this problem is likely due to:

Too few/diverse training frames, and would likely improve with 200–500+ labeled frames?
A limitation of the multi-animal assembly/tracking pipeline when the two "individuals" are actually a real animal and its mirror reflection?
A DLC 3.0.0 issue?

Would you recommend continuing with the multi-animal approach and collecting many more training frames, or switching back to the single-animal approach and focusing on improving that model instead?

Any advice would be greatly appreciated.

Thanks,
Emily