From 97729d25a4d554dcd1a64761f4235341c1f3a7be Mon Sep 17 00:00:00 2001 From: imilev Date: Fri, 4 Jul 2025 17:04:29 +0300 Subject: [PATCH] Added high-level diagrams --- .../Advanced_Analysis_Post_processing.md | 133 ++++++++ .codeboarding/Core_Deep_Learning_Engine.md | 115 +++++++ .../Project_Data_Workflow_Management.md | 299 ++++++++++++++++++ .../System_Utilities_Benchmarking.md | 243 ++++++++++++++ .codeboarding/User_Interface.md | 263 +++++++++++++++ .codeboarding/on_boarding.md | 177 +++++++++++ 6 files changed, 1230 insertions(+) create mode 100644 .codeboarding/Advanced_Analysis_Post_processing.md create mode 100644 .codeboarding/Core_Deep_Learning_Engine.md create mode 100644 .codeboarding/Project_Data_Workflow_Management.md create mode 100644 .codeboarding/System_Utilities_Benchmarking.md create mode 100644 .codeboarding/User_Interface.md create mode 100644 .codeboarding/on_boarding.md diff --git a/.codeboarding/Advanced_Analysis_Post_processing.md b/.codeboarding/Advanced_Analysis_Post_processing.md new file mode 100644 index 0000000000..9d2c91d23f --- /dev/null +++ b/.codeboarding/Advanced_Analysis_Post_processing.md @@ -0,0 +1,133 @@ +```mermaid + +graph LR + + Multi_Animal_Tracking["Multi-Animal Tracking"] + + Refinement["Refinement"] + + Post_processing["Post-processing"] + + Pose_Estimation_3D["Pose Estimation 3D"] + + Multi_Animal_Tracking -- "provides to" --> Refinement + + Refinement -- "provides to" --> Post_processing + + Refinement -- "provides to" --> Pose_Estimation_3D + + Pose_Estimation_3D -- "uses" --> Post_processing + + click Post_processing href "https://github.com/DeepLabCut/DeepLabCut/blob/main/.codeboarding//Post_processing.md" "Details" + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +The `Advanced Analysis & Post-processing` subsystem is crucial for transforming raw pose estimation outputs into refined, accurate, and insightful behavioral data. It encompasses several key stages, including multi-animal identity management, data cleaning, 3D reconstruction, and general data enhancement. This modular design aligns with the project's emphasis on a pipeline-driven and data-centric architecture, ensuring that each step of data refinement is handled by specialized, decoupled components. + + + +### Multi-Animal Tracking + +This component is responsible for maintaining individual identities across frames in multi-animal scenarios. It includes functionalities for training and inference of re-identification models (e.g., DLCTrans), creating tracking datasets, and preprocessing tracking data. Its purpose is to enable the analysis of complex social behaviors by ensuring consistent individual identification. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut/pose_tracking_pytorch/apis.py` + +- `deeplabcut/pose_tracking_pytorch/create_dataset.py` + +- `deeplabcut/pose_tracking_pytorch/tracking_utils/preprocessing.py` + +- `deeplabcut/pose_tracking_pytorch/train_dlctransreid.py` + +- `deeplabcut/pose_tracking_pytorch/inference.py` + + + + + +### Refinement + +This component focuses on improving the quality and continuity of pose estimation and tracking data. It identifies and corrects outlier frames (e.g., using statistical models like SARIMAX) and stitches fragmented tracklets to ensure smooth and accurate trajectories for each individual. This is vital for downstream analysis accuracy. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut/refine_training_dataset/outlier_frames.py` + +- `deeplabcut/refine_training_dataset/tracklets.py` + +- `deeplabcut/refine_training_dataset/stitch.py` + + + + + +### Post-processing [[Expand]](./Post_processing.md) + +This component provides general post-processing functionalities for the pose estimation results. This includes applying various filters to smooth trajectories, correcting minor errors, and analyzing the skeleton (e.g., calculating bone lengths or angles) to derive higher-level behavioral metrics. It serves as a versatile tool for data enhancement. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut/post_processing/analyze_skeleton.py` + +- `deeplabcut/post_processing/filtering.py` + + + + + +### Pose Estimation 3D + +This component specializes in reconstructing 3D poses from 2D estimations, which is essential for depth perception and more accurate spatial analysis. It handles multi-view camera calibration, triangulating 2D keypoints into 3D coordinates, and visualizing the reconstructed 3D poses. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut/pose_estimation_3d/camera_calibration.py` + +- `deeplabcut/pose_estimation_3d/plotting3D.py` + +- `deeplabcut/pose_estimation_3d/triangulation.py` + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Core_Deep_Learning_Engine.md b/.codeboarding/Core_Deep_Learning_Engine.md new file mode 100644 index 0000000000..9d20bff6fe --- /dev/null +++ b/.codeboarding/Core_Deep_Learning_Engine.md @@ -0,0 +1,115 @@ +```mermaid + +graph LR + + Framework_Compatibility_Layer["Framework Compatibility Layer"] + + TensorFlow_Deep_Learning_Engine["TensorFlow Deep Learning Engine"] + + PyTorch_Deep_Learning_Engine["PyTorch Deep Learning Engine"] + + Framework_Compatibility_Layer -- "Uses" --> TensorFlow_Deep_Learning_Engine + + Framework_Compatibility_Layer -- "Uses" --> PyTorch_Deep_Learning_Engine + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +The `Core Deep Learning Engine` subsystem in DeepLabCut is designed to abstract the complexities of underlying deep learning frameworks (TensorFlow/PyTorch) while providing robust capabilities for neural network model definition, training, inference (pose prediction), and internal evaluation. This modular design adheres to the project's emphasis on decoupling high-level application flow from specific deep learning implementations. + + + +### Framework Compatibility Layer + +This component acts as a crucial abstraction layer, offering a unified interface for the rest of the DeepLabCut application to interact seamlessly with either the TensorFlow or PyTorch deep learning backends. It encapsulates framework-specific calls, allowing for flexible switching between implementations without altering higher-level logic. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.compat.analyze_videos` (657:975) + +- `deeplabcut.compat.create_tracking_dataset` (978:1108) + +- `deeplabcut.compat.return_train_network_path` (310:373) + + + + + +### TensorFlow Deep Learning Engine + +This component is the dedicated computational engine for all deep learning operations performed using the TensorFlow framework. It encompasses the definition, training, and inference of neural network models for pose estimation, along with utilities for dataset management and model configuration specific to TensorFlow. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.pose_estimation_tensorflow.nnets.base.BasePoseNet` (18:245) + +- `deeplabcut.pose_estimation_tensorflow.datasets.pose_base.BasePoseDataset` (16:36) + +- `deeplabcut.pose_estimation_tensorflow.core.train` (146:313) + +- `deeplabcut.pose_estimation_tensorflow.core.predict` + + + + + +### PyTorch Deep Learning Engine + +This component provides the core deep learning functionalities for model definition, training, and inference within the PyTorch framework. It includes a comprehensive set of APIs for various deep learning tasks, modules for data handling (preprocessing, post-processing), diverse model architectures (backbones, heads, loss functions), and runners to orchestrate training and inference workflows. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.pose_estimation_pytorch.models.model.PoseModel` (32:239) + +- `deeplabcut.pose_estimation_pytorch.models.backbones.base.BaseBackbone` (26:83) + +- `deeplabcut.pose_estimation_pytorch.models.heads.base.BaseHead` (32:140) + +- `deeplabcut.pose_estimation_pytorch.models.criterions.base.BaseCriterion` (23:40) + +- `deeplabcut.pose_estimation_pytorch.data.base.Loader` (36:372) + +- `deeplabcut.pose_estimation_pytorch.runners.base.Runner` (51:117) + +- `deeplabcut.pose_estimation_pytorch.apis.training` + +- `deeplabcut.pose_estimation_pytorch.apis.analyze_images` (217:382) + +- `deeplabcut.pose_estimation_pytorch.apis.videos` + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Project_Data_Workflow_Management.md b/.codeboarding/Project_Data_Workflow_Management.md new file mode 100644 index 0000000000..fb1173243d --- /dev/null +++ b/.codeboarding/Project_Data_Workflow_Management.md @@ -0,0 +1,299 @@ +```mermaid + +graph LR + + Project_Creation_Configuration["Project Creation & Configuration"] + + Data_Preparation_Augmentation["Data Preparation & Augmentation"] + + Model_Management_Loading["Model Management & Loading"] + + Core_Utilities["Core Utilities"] + + General_Utilities["General Utilities"] + + Graphical_User_Interface_GUI_["Graphical User Interface (GUI)"] + + Command_Line_Interface_CLI_["Command-Line Interface (CLI)"] + + Pose_Estimation_PyTorch_["Pose Estimation (PyTorch)"] + + Pose_Estimation_TensorFlow_["Pose Estimation (TensorFlow)"] + + Compatibility_Layer["Compatibility Layer"] + + Project_Creation_Configuration -- "uses" --> Core_Utilities + + Project_Creation_Configuration -- "provides data to" --> Data_Preparation_Augmentation + + Data_Preparation_Augmentation -- "uses" --> General_Utilities + + Data_Preparation_Augmentation -- "provides data to" --> Model_Management_Loading + + Model_Management_Loading -- "integrates with" --> Pose_Estimation_PyTorch_ + + Model_Management_Loading -- "integrates with" --> Pose_Estimation_TensorFlow_ + + Model_Management_Loading -- "uses" --> Data_Preparation_Augmentation + + Graphical_User_Interface_GUI_ -- "uses" --> Core_Utilities + + Graphical_User_Interface_GUI_ -- "uses" --> General_Utilities + + Graphical_User_Interface_GUI_ -- "integrates with" --> Project_Creation_Configuration + + Graphical_User_Interface_GUI_ -- "integrates with" --> Data_Preparation_Augmentation + + Command_Line_Interface_CLI_ -- "integrates with" --> Project_Creation_Configuration + + Command_Line_Interface_CLI_ -- "integrates with" --> Data_Preparation_Augmentation + + Pose_Estimation_PyTorch_ -- "consumes data from" --> Data_Preparation_Augmentation + + Pose_Estimation_PyTorch_ -- "uses" --> Model_Management_Loading + + Pose_Estimation_TensorFlow_ -- "consumes data from" --> Data_Preparation_Augmentation + + Pose_Estimation_TensorFlow_ -- "uses" --> Model_Management_Loading + + Project_Creation_Configuration -- "uses" --> Compatibility_Layer + + Pose_Estimation_PyTorch_ -- "uses" --> Compatibility_Layer + + Pose_Estimation_TensorFlow_ -- "uses" --> Compatibility_Layer + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +This component is central to managing the entire lifecycle of a DeepLabCut project, from initial setup to data preparation for model training. It adheres to the project's architectural biases by providing a modular, pipeline-driven, and data-centric approach, ensuring a clear separation of concerns and configurability. + + + +### Project Creation & Configuration + +This component handles the initial setup of new DeepLabCut projects, including both 2D and 3D variants. It manages project-specific configurations, such as defining body parts, creating configuration files, and adding video files to the project. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.create_project` (1:1) + +- `deeplabcut.create_project.new` (1:1) + +- `deeplabcut.create_project.new_3d` (1:1) + +- `deeplabcut.create_project.add` (1:1) + + + + + +### Data Preparation & Augmentation + +This component is responsible for preparing the data for model training. This includes extracting frames from videos, managing and manipulating training datasets (e.g., splitting data, shuffling), and handling metadata associated with the datasets. It also supports multi-individual training set manipulation. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.generate_training_dataset` (1:1) + +- `deeplabcut.generate_training_dataset.frame_extraction` (1:1) + +- `deeplabcut.generate_training_dataset.trainingsetmanipulation` (1:1) + +- `deeplabcut.generate_training_dataset.metadata` (1:1) + +- `deeplabcut.generate_training_dataset.multiple_individuals_trainingsetmanipulation` (1:1) + + + + + +### Model Management & Loading + +This component provides functionalities for managing and loading pre-trained models, including utilities for model configuration and data conversion to a generalized format. It acts as a centralized repository for various pose estimation models. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.modelzoo` (1:1) + +- `deeplabcut.modelzoo.utils` (1:1) + +- `deeplabcut.modelzoo.model_configs` (1:1) + +- `deeplabcut.modelzoo.generalized_data_converter` (1:1) + + + + + +### Core Utilities + +This component encompasses core functionalities and utilities that are fundamental to the entire DeepLabCut application. This includes configuration management, engine functionalities, and basic tracking utilities. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.core.config` (1:1) + +- `deeplabcut.core.engine` (1:1) + +- `deeplabcut.core.trackingutils` (1:1) + + + + + +### General Utilities + +This component provides a collection of general-purpose utility functions, including tools for video manipulation (reading, writing), frame selection, and other auxiliary functions that support various parts of the DeepLabCut workflow. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.utils.auxfun_videos` (1:1) + +- `deeplabcut.utils.frameselectiontools` (1:1) + +- `deeplabcut.utils.video_processor` (1:1) + + + + + +### Graphical User Interface (GUI) + +This component provides the graphical user interface for DeepLabCut, allowing users to interact with the application through various tabs and widgets. It encapsulates the visual presentation and user interaction logic. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.gui.window` (1:1) + +- `deeplabcut.gui.tabs` (1:1) + +- `deeplabcut.gui.components` (1:1) + +- `deeplabcut.gui.utils` (1:1) + + + + + +### Command-Line Interface (CLI) + +This component provides a command-line interface for DeepLabCut, enabling users to automate workflows and integrate DeepLabCut into scripts. It exposes core functionalities as command-line commands. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.cli` (1:1) + + + + + +### Pose Estimation (PyTorch) + +This component contains the core logic for pose estimation using the PyTorch deep learning framework. It includes models, data loaders, preprocessors, postprocessors, and training/inference runners specific to PyTorch. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.pose_estimation_pytorch` (1:1) + + + + + +### Pose Estimation (TensorFlow) + +Similar to the PyTorch component, this handles pose estimation using the TensorFlow deep learning framework. It includes TensorFlow-specific models, datasets, and network architectures. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.pose_estimation_tensorflow` (1:1) + + + + + +### Compatibility Layer + +This component (though not explicitly listed in the initial getClassHierarchy output, it's a common pattern in such frameworks) would handle compatibility issues between different versions of libraries or operating systems, ensuring smooth operation across various environments. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.compat` (1:1) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/System_Utilities_Benchmarking.md b/.codeboarding/System_Utilities_Benchmarking.md new file mode 100644 index 0000000000..ae27e1f37b --- /dev/null +++ b/.codeboarding/System_Utilities_Benchmarking.md @@ -0,0 +1,243 @@ +```mermaid + +graph LR + + Configuration_Manager["Configuration Manager"] + + Video_I_O_Processing["Video I/O & Processing"] + + General_Purpose_Utilities["General Purpose Utilities"] + + Metric_Calculation_Engine["Metric Calculation Engine"] + + Benchmarking_Orchestrator["Benchmarking Orchestrator"] + + Visualization_Plotting_Tools["Visualization & Plotting Tools"] + + Model_Weight_Management["Model Weight Management"] + + Data_Conversion_Adaptation["Data Conversion & Adaptation"] + + Benchmarking_Orchestrator -- "uses" --> Metric_Calculation_Engine + + Benchmarking_Orchestrator -- "uses" --> General_Purpose_Utilities + + Benchmarking_Orchestrator -- "reads from" --> Configuration_Manager + + Metric_Calculation_Engine -- "uses" --> General_Purpose_Utilities + + Video_I_O_Processing -- "uses" --> General_Purpose_Utilities + + Configuration_Manager -- "uses" --> General_Purpose_Utilities + + Visualization_Plotting_Tools -- "uses" --> General_Purpose_Utilities + + Visualization_Plotting_Tools -- "visualizes results from" --> Metric_Calculation_Engine + + Model_Weight_Management -- "uses" --> General_Purpose_Utilities + + Data_Conversion_Adaptation -- "uses" --> General_Purpose_Utilities + + Data_Conversion_Adaptation -- "reads from" --> Configuration_Manager + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +This foundational component provides a comprehensive set of reusable helper functions, common data structures, video I/O capabilities, file system interactions, configuration parsing, plotting functionalities, and dedicated tools for quantitatively assessing model performance. It underpins many other components by offering essential low-level services and critical evaluation mechanisms. + + + +### Configuration Manager + +Centralizes the management of project configuration, providing utilities for reading, writing, and modifying YAML-based settings. It ensures that the application's behavior is driven by external, persistent parameters, promoting flexibility and maintainability. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.core.config` + +- `deeplabcut.utils.auxiliaryfunctions:read_config` (199:240) + +- `deeplabcut.utils.auxiliaryfunctions:write_config` (243:258) + +- `deeplabcut.utils.auxiliaryfunctions:edit_config` (261:300) + + + + + +### Video I/O & Processing + +Handles all video-related operations, including efficient reading of frames, writing new videos, and performing basic manipulations like shortening, cropping, and resizing. It abstracts the complexities of underlying video file formats and codecs. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.utils.auxfun_videos` + +- `deeplabcut.utils.video_processor` + + + + + +### General Purpose Utilities + +A foundational collection of widely applicable helper functions for file system interactions (e.g., path handling, directory creation), general data manipulation (e.g., array operations, data validation), multiprocessing, and common data structures (e.g., skeleton definitions). These utilities support various parts of the application, preventing code duplication. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.utils.auxiliaryfunctions` + +- `deeplabcut.utils.frameselectiontools` + +- `deeplabcut.utils.multiprocessing` + +- `deeplabcut.utils.skeleton` + +- `deeplabcut.utils.pseudo_label` + + + + + +### Metric Calculation Engine + +Provides a comprehensive set of functions for quantitatively assessing model performance. It includes implementations for various metrics such as Root Mean Squared Error (RMSE), Object Keypoint Similarity (OKS), and mean Average Precision (mAP), which are crucial for evaluating pose estimation and tracking accuracy. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.benchmark.metrics` + +- `deeplabcut.core.metrics.api` + +- `deeplabcut.core.metrics.distance_metrics` + +- `deeplabcut.core.metrics.matching` + +- `deeplabcut.core.metrics.bbox` + +- `deeplabcut.core.metrics.identity` + + + + + +### Benchmarking Orchestrator + +Manages the end-to-end process of evaluating pose estimation models. It defines the workflow for loading predictions, validating data, invoking metric calculations, and managing benchmark results, serving as the central control for performance assessment. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.benchmark.base` + +- `deeplabcut.benchmark.benchmarks` + +- `deeplabcut.benchmark.cli` + + + + + +### Visualization & Plotting Tools + +Offers a versatile set of tools for generating visual representations of data and results. This includes plotting trajectories, score maps, and general evaluation outcomes, aiding in debugging, analysis, and understanding model behavior. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.utils.plotting` + +- `deeplabcut.utils.visualization` + +- `deeplabcut.core.visualization` + + + + + +### Model Weight Management + +Handles the lifecycle of model weights, including loading pre-trained models, checking their integrity, and facilitating their download. It ensures that the correct model parameters are available for inference and training. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.utils.auxfun_models` + +- `deeplabcut.core.weight_init` + + + + + +### Data Conversion & Adaptation + +Provides utilities for transforming data between different internal and external formats (e.g., CSV to HDF5) and adapting labeled datasets to new project structures. This ensures data compatibility and flexibility within the DeepLabCut ecosystem. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.utils.conversioncode` + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/User_Interface.md b/.codeboarding/User_Interface.md new file mode 100644 index 0000000000..b748438ee7 --- /dev/null +++ b/.codeboarding/User_Interface.md @@ -0,0 +1,263 @@ +```mermaid + +graph LR + + CLI_Entry_Point["CLI Entry Point"] + + GUI_Application_Launcher["GUI Application Launcher"] + + Main_GUI_Window["Main GUI Window"] + + GUI_Tabs["GUI Tabs"] + + GUI_Components["GUI Components"] + + GUI_Widgets["GUI Widgets"] + + GUI_Utilities["GUI Utilities"] + + Core_Engine["Core Engine"] + + Project_Management["Project Management"] + + Training_Data_Generation["Training Data Generation"] + + CLI_Entry_Point -- "initiates workflows in" --> Core_Engine + + GUI_Application_Launcher -- "starts" --> Main_GUI_Window + + Main_GUI_Window -- "manages and displays" --> GUI_Tabs + + Main_GUI_Window -- "interacts with" --> Core_Engine + + GUI_Tabs -- "utilizes" --> GUI_Components + + GUI_Tabs -- "utilizes" --> GUI_Widgets + + GUI_Tabs -- "interacts with" --> Core_Engine + + Main_GUI_Window -- "leverages" --> GUI_Utilities + + GUI_Tabs -- "leverages" --> GUI_Utilities + + Project_Management -- "configures" --> Core_Engine + + Training_Data_Generation -- "provides data to" --> Core_Engine + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +Abstract Components Overview for DeepLabCut + + + +### CLI Entry Point + +This component provides the command-line interface for DeepLabCut. It is responsible for parsing command-line arguments and dispatching calls to the appropriate core functionalities of the DeepLabCut engine. It allows users to automate workflows and integrate DeepLabCut into scripts. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.cli` + + + + + +### GUI Application Launcher + +This component is the main entry point for the graphical user interface. It initializes the GUI environment and launches the MainWindow, setting up the visual application for user interaction. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.__main__` + +- `deeplabcut.gui.launch_script` + + + + + +### Main GUI Window + +The central window of the DeepLabCut GUI. It orchestrates the display and management of various functional tabs and directly interacts with the core DeepLabCut engine. It utilizes the deeplabcut.core.engine.Engine for core workflow management through composition, rather than inheritance. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.gui.window.MainWindow` (83:740) + + + + + +### GUI Tabs + +This component comprises a collection of modules, each representing a distinct functional tab within the GUI. Examples include AnalyzeVideos, CreateTrainingDataset, TrainNetwork, and LabelFrames. These tabs encapsulate specific UI logic and interactions for each step of the DeepLabCut workflow. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.gui.tabs` + +- `deeplabcut.gui.tabs.analyze_videos.AnalyzeVideos` (32:372) + +- `deeplabcut.gui.tabs.create_training_dataset.CreateTrainingDataset` (43:544) + +- `deeplabcut.gui.tabs.train_network.TrainNetwork` (49:273) + +- `deeplabcut.gui.tabs.label_frames.LabelFrames` (102:146) + + + + + +### GUI Components + +This component provides reusable base classes and common UI building blocks, such as DefaultTab, BodypartListWidget, ShuffleSpinBox, and VideoSelectionWidget. These components promote consistency and reusability across different GUI tabs, ensuring a uniform user experience. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.gui.components` + + + + + +### GUI Widgets + +This component contains specialized, interactive UI controls that facilitate user interaction within the GUI. Examples include DictViewer, DragDropListView, ItemSelectionFrame, StreamReceiver, and StreamWriter. These widgets handle specific input and output functionalities. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.gui.widgets` + + + + + +### GUI Utilities + +This component offers helper functions and classes for GUI-specific tasks, including managing threads for long-running operations to maintain UI responsiveness (e.g., Worker, CaptureWorker). It ensures that the GUI remains interactive even during computationally intensive tasks. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.gui.utils` + + + + + +### Core Engine + +While not strictly part of the "User Interface" component, the Core Engine (deeplabcut.core.engine.Engine) is a critical dependency. It encapsulates the core logic and functionalities of DeepLabCut, including project management, training, analysis, and data handling. The GUI and CLI interact with this engine to perform all DeepLabCut operations. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.core.engine.Engine` (25:48) + + + + + +### Project Management + +This component handles the creation, loading, and management of DeepLabCut projects. It interacts with the file system to store and retrieve project configurations, video paths, and other project-related metadata. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.gui.tabs.manage_project.ManageProject` (24:75) + +- `deeplabcut.gui.tabs.create_project.ProjectCreator` (184:538) + + + + + +### Training Data Generation + +This component is responsible for the processes related to generating and managing training datasets, including extracting frames from videos, labeling frames, and creating the actual training dataset. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.gui.tabs.extract_frames.ExtractFrames` (83:315) + +- `deeplabcut.gui.tabs.label_frames.LabelFrames` (102:146) + +- `deeplabcut.gui.tabs.create_training_dataset.CreateTrainingDataset` (43:544) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/on_boarding.md b/.codeboarding/on_boarding.md new file mode 100644 index 0000000000..c7b54f7dd2 --- /dev/null +++ b/.codeboarding/on_boarding.md @@ -0,0 +1,177 @@ +```mermaid + +graph LR + + User_Interface["User Interface"] + + Project_Data_Workflow_Management["Project & Data Workflow Management"] + + Core_Deep_Learning_Engine["Core Deep Learning Engine"] + + Advanced_Analysis_Post_processing["Advanced Analysis & Post-processing"] + + System_Utilities_Benchmarking["System Utilities & Benchmarking"] + + User_Interface -- "initiates workflows and configurations in" --> Project_Data_Workflow_Management + + User_Interface -- "receives and displays final processed results from" --> Advanced_Analysis_Post_processing + + Project_Data_Workflow_Management -- "provides prepared data and model configurations to" --> Core_Deep_Learning_Engine + + Project_Data_Workflow_Management -- "utilizes general services from" --> System_Utilities_Benchmarking + + Core_Deep_Learning_Engine -- "sends raw pose predictions to" --> Advanced_Analysis_Post_processing + + Core_Deep_Learning_Engine -- "utilizes general services from" --> System_Utilities_Benchmarking + + Advanced_Analysis_Post_processing -- "receives raw predictions from" --> Core_Deep_Learning_Engine + + Advanced_Analysis_Post_processing -- "provides processed results to" --> User_Interface + + System_Utilities_Benchmarking -- "provides foundational services to" --> Project_Data_Workflow_Management + + System_Utilities_Benchmarking -- "provides foundational services to" --> Core_Deep_Learning_Engine + + click User_Interface href "https://github.com/DeepLabCut/DeepLabCut/blob/main/.codeboarding//User_Interface.md" "Details" + + click Project_Data_Workflow_Management href "https://github.com/DeepLabCut/DeepLabCut/blob/main/.codeboarding//Project_Data_Workflow_Management.md" "Details" + + click Core_Deep_Learning_Engine href "https://github.com/DeepLabCut/DeepLabCut/blob/main/.codeboarding//Core_Deep_Learning_Engine.md" "Details" + + click Advanced_Analysis_Post_processing href "https://github.com/DeepLabCut/DeepLabCut/blob/main/.codeboarding//Advanced_Analysis_Post_processing.md" "Details" + + click System_Utilities_Benchmarking href "https://github.com/DeepLabCut/DeepLabCut/blob/main/.codeboarding//System_Utilities_Benchmarking.md" "Details" + +``` + + + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Details + + + +The DeepLabCut architecture is designed as a modular, pipeline-driven, and data-centric system, emphasizing a clear separation of concerns. The analysis consolidates the project's functionalities into five core components, facilitating maintainability, scalability, and user-friendliness for deep learning-based computer vision tasks. + + + +### User Interface [[Expand]](./User_Interface.md) + +The primary interaction layer for users, providing both a comprehensive graphical interface (GUI) and a command-line interface (CLI) to initiate, manage, and monitor all DeepLabCut workflows. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.cli` (1:1) + +- `deeplabcut.__main__` (1:1) + +- `deeplabcut.gui` (1:1) + + + + + +### Project & Data Workflow Management [[Expand]](./Project_Data_Workflow_Management.md) + +Manages the entire project lifecycle, including creating new DeepLabCut projects, handling video files, extracting frames for labeling, organizing datasets, and managing project-specific configurations. It also integrates model loading and configuration. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.create_project` (1:1) + +- `deeplabcut.generate_training_dataset` (1:1) + +- `deeplabcut.modelzoo` (1:1) + + + + + +### Core Deep Learning Engine [[Expand]](./Core_Deep_Learning_Engine.md) + +The central computational engine responsible for neural network model definition, training, inference (pose prediction), and internal evaluation, abstracting underlying deep learning frameworks (TensorFlow/PyTorch) via a compatibility layer. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.compat` (1:1) + +- `deeplabcut.pose_estimation_tensorflow` (1:1) + +- `deeplabcut.pose_estimation_pytorch` (1:1) + + + + + +### Advanced Analysis & Post-processing [[Expand]](./Advanced_Analysis_Post_processing.md) + +Refines raw pose estimation outputs by applying filtering, correcting outliers, performing 3D pose reconstruction from 2D estimations, and handling multi-animal tracking functionalities. It also prepares data for final display. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.post_processing` (1:1) + +- `deeplabcut.refine_training_dataset` (1:1) + +- `deeplabcut.pose_estimation_3d` (1:1) + +- `deeplabcut.pose_tracking_pytorch` (1:1) + + + + + +### System Utilities & Benchmarking [[Expand]](./System_Utilities_Benchmarking.md) + +A foundational component providing a comprehensive set of reusable helper functions, common data structures, video I/O, file system interactions, configuration parsing, plotting, and tools for quantitatively assessing model performance. + + + + + +**Related Classes/Methods**: + + + +- `deeplabcut.utils` (1:1) + +- `deeplabcut.core` (1:1) + +- `deeplabcut.benchmark` (1:1) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file