update list_videos_in_folder#3303
Conversation
…iles if filter `video_type` is set. - Accept files without extension - Default folder searching is kept as is (using valid video extensions)
|
Note: CI seems to be failing due to tf-macos, merging #3292 may potentially help |
…set of extensions.
|
@deruyter92 Great to add deprecations, definitely agree. Do you think a separate PR for that specifically would be useful, or is it more efficient to merge this directly ? |
There was a problem hiding this comment.
Really nice work overall! Definitely much better to have the video loading centralized, and deprecations are nice.
I added quite a few comments that I hope will help future us avoid mistakes and make things easier to use, happy to discuss if there are any concerns.
I will open a PR for the deprecations, I have a small design suggestion to ensure we can greatly extend and automate the system later if needed, while keeping current design and lightweight code.
* Add structured deprecation info and warnings Introduce a DLCDeprecationWarning and a DeprecationInfo pydantic model to standardize deprecation metadata (kind, target, replacement, since, removed_in, renamed params) with parsing and validation of versions. Revamp deprecated and renamed_parameter decorators to build messages from DeprecationInfo, emit DLCDeprecationWarning, attach metadata to wrapped callables (__deprecated_info__, __deprecated_params__), use ParamSpec/TypeVar typing for wrappers, and enforce error when both old and new kwargs are passed. Switch to packaging.version for version parsing. * Use DLCDeprecationWarning and add metadata tests Replace generic DeprecationWarning checks with DLCDeprecationWarning and import packaging.version.Version. Add tests verifying deprecated decorators attach metadata (including since/removed_in parsed as Version), validate invalid version inputs, and ensure removed_in > since. Also add tests for renamed_parameter behavior (conflicting old+new raises, metadata attachment, and invalid since handling) and small docstring/name preservation assertions. * Add packaging as core dep
…`list_videos_in_folder`
Summary
Currently there exist different functions for collecting video paths from a list of directories and files, with slightly different behavior. See #3300.
Old behavior
list_videos_in_folderalways filters the input files by extension: either the defaultSUPPORTED_VIDEOSor custom specifiedvideo_type.get_video_listonly filters directories bySUPPORTED_VIDEOS, but not files.get_list_of_videosonly filters directories bySUPPORTED_VIDEOS, but not files.Issues
video_typeandvideotype: the meanings of""andNonediffer per-case. List of strings is not always accepted.Changes
list_videos_in_folderfunction, to have a single centralized helper for this purposeextensionsisNone; only filtered whenextensionsis explicitly set.SUPPORTED_VIDEOSwhenextensionsisNone, or by the given value(s) otherwise.exclude_patternsare always applied to both files and directory contents.solves #3300
Details
PR Status
list_videos_in_folderto auxfun_videos: more centralized instead of pytorch-specificcollect_video_paths, which more accurately reflects function. feedback on naming is welcome!SUPPORTED_VIDEOSby default or with specified extensions filter if set.exclude_patternswhich is defaults to ["_labeled.", "_full."] to match prior DLC outputs.list_videos_in_folderand map to newcollect_video_pathsget_list_of_videosand map to newcollect_video_pathsget_video_listand map to newcollect_video_pathsgrab_files_in_folder,get_camerawise_videos(different implementation)New function signature