Is there an existing issue for this?
Bug description
My apologies. This is somewhat in between a feature request and a bug report. There's a few problems here. I've been using deeplabcut since nearly the initial release (thank you!). I was really hoping that ease of use would improve but am instead disappointed every 2 or 3 years when I need to reinstall on a new machine or otherwise change things. I'm currently exploring SLEAP solely out of perpetual frustration with DLC usage.
At a meta level, can I request/suggest fewer features that are better maintained and documented?
I expected:
- tensorflow and pytorch implementations to be either compatible or modularized
- to be able to follow the pytorch documentation to adapt my existing project and files created with the 2.3.5 docker image on dockerhub to train new models, analyze a new video, and run the other (currently supported) code
- some degree of backwards compatibility at least within an engine type
- a mature install approach with proper documentation
Operating System
Output of uname -a: Linux [hostname] 5.14.0-427.35.1.el9_4.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 18:24:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
DeepLabCut version
Wrt general comments: I've been using DLC since 2019.
Wrt more specific bugs: docker container for v2.3.5 upon which a single animal resnet 50 model was trained and used. I'm now trying to use at least some of those files on a fresh install on my department's server. I tried matching 2.3.5 but I ran into tensorflow dependency issues (even though I wasn't trying to train in tensorflow), so I'm using 2.3.10.
DeepLabCut mode
single animal
Device type
nvidia-smi
Mon Jan 6 17:19:30 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 Quadro P6000 Off | 00000000:3B:00.0 Off | Off |
| 26% 35C P8 9W / 250W | 2MiB / 24576MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 Quadro P6000 Off | 00000000:AF:00.0 Off | Off |
| 26% 29C P8 8W / 250W | 2MiB / 24576MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
Steps To Reproduce
My usage might be unusual, so I'll keep this a bit brief unless others have similar issues and want to compare notes or the DLC team explicitly request more info.
-
Versions before 2022 had poor tensorflow and other python dependency documentation, particularly for wx iirc. Why not just give list of working configurations with version numbers and a setup.py file? Maybe this information already exists, but I didn't find it.
-
A year or so ago, I managed to get the docker container to work with v2.3.2 and later v2.3.5.
Problems encountered and working solutions:
- I could never get the documented docker pull command to work --> successfully pulled a specific tag
- Since you bothered to make a docker container and some minimal documentation, why don't you give example pull and run commands. It took me hours to get the proper flags...
If helpful to others, for me this was:
- sudo docker pull deeplabcut/deeplabcut:2.3.5-core-cuda11.4.3-cudnn8-runtime-ubuntu20.04-latest”
- sudo docker run --gpus all -v ~/host_data:/host_data -it deeplabcut/deeplabcut:2.3.5-core-cuda11.4.3-cudnn8-runtime-ubuntu20.04-latest
- This month, I became unsatisfied with the noise/misdetections in my existing tracking. I've been exploring: a) training a fresh model, b) gridsearching the filterpredictions parameters, and c) (attempting unsuccessfully) to evaluate (yes, I understand this isn't yet implemented for pytorch) or create labeled video.
-
2a: the proper integration (ie config generation, population, and shuffle sharing) was extremely confusing to me. I have no confidence that I did it as intended despite attempting to follow all documentation (which doesn't appear consolidated). I ultimately resorted to a mixture of moving files to where they were expected or the DLC source code directly. If it provides a clue to my mistake or the bug, often the paths have repeating segments in them (e.g something_valid/a/b/c/a/b/c/something_else_that_looks_reasonable). I hacked the following files: analyze_videos.py, and /pose-estimation/utils.py. Inference is currently running, albeit at only 7 it/s which is slower than the old tensorflow inference which got ~ 30 it/s, presumably due to lack of cropping.
-
2b: feature request: it would be nice to support an approach that enables gridsearching over the various parameter options. I accomplished this by hacking filtering.py to name filtered files uniquely and not abort if a file with "filt" is found. Files have at least been created successfully, although I haven't yet inspected them in detail.
-
2c) There's potentially two issues here: a path issue and tensorflow dependency issue (even for pytorch or if not running training or inference). It seems like the new pytorch approach might still require a working tensorflow install to run create_labeled_video. I believe I tried both 2.3.5 and 2.3.10 on the new server installation. I will retry the function on my old docker install soon and edit this issue and my other comments. @slagathor69 just replied my comment mentioning the same "No full data found" issue. I was unable to find that string anywhere. If someone could just tell me where it is, I'll hack it (and post/pull request my hack, if desirable).
Update: create_labeled_video appears to work in my 2.3.5 docker container. My current hypothesis is that there is still a path issue or complexity to the configs that would be best to further document. Likely independent of the path issue is that dependencies for older versions like 2.3.5 expect a working tensorflow install even if no tensorflow commands are called.
Feature request: it would be create to have a minimal install approach that allows running non-tensorflow-dependent commands without the tensorflow installations/dependencies. This is particularly important given that a lot of functionality is not yet ready for the pytorch engine.
Again, my apologies for multiplexing several issues that may not be related in a single issue.
Thanks again for your time and energy providing this valuable tool!
Relevant log output
Probably not useful and too specific to my potentially unusual usage. Available upon request.
Anything else?
No response
Code of Conduct
Is there an existing issue for this?
Bug description
My apologies. This is somewhat in between a feature request and a bug report. There's a few problems here. I've been using deeplabcut since nearly the initial release (thank you!). I was really hoping that ease of use would improve but am instead disappointed every 2 or 3 years when I need to reinstall on a new machine or otherwise change things. I'm currently exploring SLEAP solely out of perpetual frustration with DLC usage.
At a meta level, can I request/suggest fewer features that are better maintained and documented?
I expected:
Operating System
Output of uname -a: Linux [hostname] 5.14.0-427.35.1.el9_4.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 18:24:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
DeepLabCut version
Wrt general comments: I've been using DLC since 2019.
Wrt more specific bugs: docker container for v2.3.5 upon which a single animal resnet 50 model was trained and used. I'm now trying to use at least some of those files on a fresh install on my department's server. I tried matching 2.3.5 but I ran into tensorflow dependency issues (even though I wasn't trying to train in tensorflow), so I'm using 2.3.10.
DeepLabCut mode
single animal
Device type
nvidia-smi
Mon Jan 6 17:19:30 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 Quadro P6000 Off | 00000000:3B:00.0 Off | Off |
| 26% 35C P8 9W / 250W | 2MiB / 24576MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 Quadro P6000 Off | 00000000:AF:00.0 Off | Off |
| 26% 29C P8 8W / 250W | 2MiB / 24576MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
Steps To Reproduce
My usage might be unusual, so I'll keep this a bit brief unless others have similar issues and want to compare notes or the DLC team explicitly request more info.
Versions before 2022 had poor tensorflow and other python dependency documentation, particularly for wx iirc. Why not just give list of working configurations with version numbers and a setup.py file? Maybe this information already exists, but I didn't find it.
A year or so ago, I managed to get the docker container to work with v2.3.2 and later v2.3.5.
Problems encountered and working solutions:
If helpful to others, for me this was:
2a: the proper integration (ie config generation, population, and shuffle sharing) was extremely confusing to me. I have no confidence that I did it as intended despite attempting to follow all documentation (which doesn't appear consolidated). I ultimately resorted to a mixture of moving files to where they were expected or the DLC source code directly. If it provides a clue to my mistake or the bug, often the paths have repeating segments in them (e.g something_valid/a/b/c/a/b/c/something_else_that_looks_reasonable). I hacked the following files: analyze_videos.py, and /pose-estimation/utils.py. Inference is currently running, albeit at only 7 it/s which is slower than the old tensorflow inference which got ~ 30 it/s, presumably due to lack of cropping.
2b: feature request: it would be nice to support an approach that enables gridsearching over the various parameter options. I accomplished this by hacking filtering.py to name filtered files uniquely and not abort if a file with "filt" is found. Files have at least been created successfully, although I haven't yet inspected them in detail.
2c) There's potentially two issues here: a path issue and tensorflow dependency issue (even for pytorch or if not running training or inference). It seems like the new pytorch approach might still require a working tensorflow install to run create_labeled_video. I believe I tried both 2.3.5 and 2.3.10 on the new server installation. I will retry the function on my old docker install soon and edit this issue and my other comments. @slagathor69 just replied my comment mentioning the same "No full data found" issue. I was unable to find that string anywhere. If someone could just tell me where it is, I'll hack it (and post/pull request my hack, if desirable).
Update: create_labeled_video appears to work in my 2.3.5 docker container. My current hypothesis is that there is still a path issue or complexity to the configs that would be best to further document. Likely independent of the path issue is that dependencies for older versions like 2.3.5 expect a working tensorflow install even if no tensorflow commands are called.
Feature request: it would be create to have a minimal install approach that allows running non-tensorflow-dependent commands without the tensorflow installations/dependencies. This is particularly important given that a lot of functionality is not yet ready for the pytorch engine.
Again, my apologies for multiplexing several issues that may not be related in a single issue.
Thanks again for your time and energy providing this valuable tool!
Relevant log output
Anything else?
No response
Code of Conduct