Skip to content

Commit 99ef25a

Browse files
alex-luxonisalex-luxonis
committed
Merge 'origin/report_cam_settings' into multi_cam_support
2 parents b29822e + 57c3cc2 commit 99ef25a

39 files changed

+1403
-321
lines changed

‎CMakeLists.txt‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ endif()
3535

3636
# Pybindings project
3737
set(TARGET_NAME depthai)
38-
project(depthai VERSION "1") # revision of bindings [depthai-core].[rev]
38+
project(depthai VERSION "0") # revision of bindings [depthai-core].[rev]
3939

4040
# Set default build type depending on context
4141
set(default_build_type "Release")
@@ -100,6 +100,7 @@ pybind11_add_module(${TARGET_NAME}
100100
src/pipeline/AssetManagerBindings.cpp
101101
src/openvino/OpenVINOBindings.cpp
102102
src/log/LogBindings.cpp
103+
src/VersionBindings.cpp
103104

104105
src/pipeline/node/NodeBindings.cpp
105106

@@ -124,6 +125,7 @@ pybind11_add_module(${TARGET_NAME}
124125
src/pipeline/node/FeatureTrackerBindings.cpp
125126
src/pipeline/node/AprilTagBindings.cpp
126127
src/pipeline/node/DetectionParserBindings.cpp
128+
src/pipeline/node/WarpBindings.cpp
127129

128130
src/pipeline/datatype/ADatatypeBindings.cpp
129131
src/pipeline/datatype/AprilTagConfigBindings.cpp

‎depthai-core‎

Submodule depthai-core updated 39 files

‎docs/source/_static/install_dependencies.sh‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#!/bin/bash
22

3-
set -e
3+
trap 'RET=$? ; echo -e >&2 "\n\x1b[31mFailed installing dependencies. Could be a bug in the installer or unsupported platform. Open a bug report over at https://github.com/luxonis/depthai - exited with status $RET at line $LINENO \x1b[0m\n" ; exit $RET' ERR
44

55
readonly linux_pkgs=(
66
python3
@@ -124,9 +124,11 @@ elif [ -f /etc/os-release ]; then
124124
python3 -m pip install --upgrade pip
125125
fi
126126

127-
dpkg -s uvcdynctrl > /dev/null 2>&1
127+
# As set -e is set, retrieve the return value without exiting
128+
RET=0
129+
dpkg -s uvcdynctrl > /dev/null 2>&1 || RET=$? || true
128130
# is uvcdynctrl installed
129-
if [ $? -eq 0 ]; then
131+
if [[ "$RET" == "0" ]]; then
130132
echo -e "\033[33mWe detected \"uvcdynctrl\" installed on your system. \033[0m"
131133
echo -e "\033[33mWe recommend removing this package, as it creates a huge log files if a camera is used in UVC mode (webcam)\033[0m"
132134
echo -e "\033[33mYou can do so by running the following commands:\033[0m"
@@ -154,7 +156,8 @@ elif [ -f /etc/os-release ]; then
154156
fi
155157

156158
# Allow all users to read and write to Myriad X devices
157-
echo 'SUBSYSTEM=="usb", ATTRS{idVendor}=="03e7", MODE="0666"' | sudo tee /etc/udev/rules.d/80-movidius.rules
159+
echo "Installing udev rules..."
160+
echo 'SUBSYSTEM=="usb", ATTRS{idVendor}=="03e7", MODE="0666"' | sudo tee /etc/udev/rules.d/80-movidius.rules > /dev/null
158161
sudo udevadm control --reload-rules && sudo udevadm trigger
159162
else
160163
echo "ERROR: Host not supported"

‎docs/source/components/nodes/image_manip.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@ ImageManip
33

44
ImageManip node can be used to crop, rotate rectangle area or perform various image transforms: rotate, mirror, flip, perspective transform.
55

6+
For downscaling, ImageManip uses the bilinear/bicubic interpolation.
7+
68
How to place it
79
###############
810

‎docs/source/components/nodes/script.rst‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,24 @@ In the script node you can interface with GPIOs of the VPU using module GPIO. Cu
120120
GPIO.PullDownUp: GPIO.PULL_NONE, GPIO.PULL_DOWN, GPIO.PULL_UP
121121
GPIO.Edge: GPIO.RISING, GPIO.FALLING, GPIO.LEVEL_HIGH, GPIO.LEVEL_LOW
122122
123+
Here's **an example** of toggling GPIO pin 40 inside Script node from the host (via :ref:`XLinkIn`). On `OAK-SoM-Pro <https://docs.luxonis.com/projects/hardware/en/latest/pages/BW2099.html>`__,
124+
GPIO 40 drives FSYNC signal for both 4-lane cameras, and we have used the code below for this exact reason.
125+
126+
.. code-block:: python
127+
128+
import GPIO
129+
MX_PIN = 40
130+
131+
ret = GPIO.setup(MX_PIN, GPIO.OUT, GPIO.PULL_DOWN)
132+
toggleVal = True
133+
134+
while True:
135+
data = node.io['in'].get() # Wait for a message from the host computer
136+
137+
node.warn('GPIO toggle: ' + str(toggleVal))
138+
toggleVal = not toggleVal
139+
ret = GPIO.write(MX_PIN, toggleVal) # Toggle the GPIO
140+
123141
Using DepthAI :ref:`Messages <components_messages>`
124142
###################################################
125143

‎docs/source/components/nodes/stereo_depth.rst‎

Lines changed: 51 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
StereoDepth
2-
===========
2+
###########
33

4-
Stereo depth node calculates the disparity/depth from two :ref:`mono cameras <MonoCamera>`.
4+
StereoDepth node calculates the disparity/depth from the stereo camera pair (2x :ref:`MonoCamera <MonoCamera>`).
55

66
How to place it
7-
###############
7+
===============
88

99
.. tabs::
1010

@@ -20,7 +20,7 @@ How to place it
2020

2121

2222
Inputs and Outputs
23-
##################
23+
==================
2424

2525
.. code-block::
2626
@@ -71,7 +71,7 @@ Inputs and Outputs
7171
- :code:`debugDispCostDump` - :ref:`ImgFrame`
7272

7373
Internal block diagram of StereoDepth node
74-
##########################################
74+
==========================================
7575

7676
.. image:: /_static/images/components/depth_diagram.png
7777
:target: https://whimsical.com/stereo-node-EKcfcXGjGpNL6cwRPV6NPv
@@ -83,7 +83,7 @@ configurable sooner.
8383
If you click on the image, you will be redirected to the webapp. Some blocks have notes that provide additional technical information.
8484

8585
Currently configurable blocks
86-
*****************************
86+
-----------------------------
8787

8888
.. tabs::
8989

@@ -165,12 +165,12 @@ Currently configurable blocks
165165
:no-link:
166166

167167
Limitations
168-
###########
168+
===========
169169

170170
- Median filtering is disabled when subpixel mode is set to 4 or 5 bits.
171171

172172
Stereo depth FPS
173-
################
173+
================
174174

175175
.. list-table::
176176
:header-rows: 1
@@ -201,7 +201,7 @@ All stereo modes were measured for :code:`depth` output with **5x5 median filter
201201
to **60 FPS** and for 400P mono cameras were set to **110 FPS**.
202202

203203
Usage
204-
#####
204+
=====
205205

206206
.. tabs::
207207

@@ -238,15 +238,15 @@ Usage
238238
right->out.link(stereo->right);
239239

240240
Examples of functionality
241-
#########################
241+
=========================
242242

243243
- :ref:`Depth Preview`
244244
- :ref:`RGB Depth alignment`
245245
- :ref:`Mono & MobilenetSSD & Depth`
246246
- :ref:`RGB & MobilenetSSD with spatial data`
247247

248248
Reference
249-
#########
249+
=========
250250

251251
.. tabs::
252252

@@ -266,7 +266,7 @@ Reference
266266
:undoc-members:
267267

268268
Disparity
269-
#########
269+
=========
270270

271271
Disparity refers to the distance between two corresponding points in the left and right image of a stereo pair.
272272
By looking at the image below, it can be seen that point :code:`X` gets projected to :code:`XL = (u, v)` in the :code:`Left view` and :code:`XR = (p, q)` in the :code:`Right view`.
@@ -290,7 +290,7 @@ For the final disparity map, a filtering is applied based on the confidence thre
290290
the threshold get invalidated, i.e. their disparity value is set to zero. You can set the confidence threshold with :code:`stereo.initialConfig.setConfidenceThreshold()`.
291291

292292
Calculate depth using disparity map
293-
###################################
293+
===================================
294294

295295
Disparity and depth are inversely related. As disparity decreases, depth increases exponentially depending on baseline and focal length. Meaning, if the disparity value is close to zero, then a small change in disparity generates a large change in depth. Similarly, if the disparity value is big, then large changes in disparity do not lead to a large change in depth.
296296

@@ -339,7 +339,7 @@ Examples for calculating the depth value, using the OAK-D (7.5cm baseline):
339339
Note the value of disparity depth data is stored in :code:`uint16`, where 0 is a special value, meaning that distance is unknown.
340340

341341
Min stereo depth distance
342-
#########################
342+
=========================
343343

344344
If the depth results for close-in objects look weird, this is likely because they are below the minimum depth-perception distance of the device.
345345

@@ -375,8 +375,41 @@ or roughly 35cm.
375375

376376
See `these examples <https://github.com/luxonis/depthai-experiments/tree/master/gen2-camera-demo#real-time-depth-from-depthai-stereo-pair>`__ for how to enable Extended Disparity.
377377

378+
Disparity shift to lower min depth perception
379+
---------------------------------------------
380+
381+
Another option to perceive closer depth range is to use disparity shift. Disparity shift will shift the starting point
382+
of the disparity search, which will significantly decrease max depth (MazZ) perception, but it will also decrease min depth (MinZ) perception.
383+
Disparity shift can be combined with extended/subpixel/LR-check modes.
384+
385+
.. image:: https://user-images.githubusercontent.com/18037362/189375017-2fa137d2-ad6b-46de-8899-6304bbc6c9d7.png
386+
387+
**Left graph** shows min and max disparity and depth for OAK-D (7.5cm baseline, 800P resolution, ~70° HFOV) by default (disparity shift=0). See :ref:`Calculate depth using disparity map`.
388+
Since hardware (stereo block) has a fixed 95 pixel disparity search, DepthAI will search from 0 pixels (depth=INF) to 95 pixels (depth=71cm).
389+
390+
**Right graph** shows the same, but at disparity shift set to 30 pixels. This means that disparity search will be from 30 pixels (depth=2.2m) to 125 pixels (depth=50cm).
391+
This also means that depth will be very accurate at the short range (**theoretically** below 5mm depth error).
392+
393+
**Limitations**:
394+
395+
- Because of the inverse relationship between disparity and depth, MaxZ will decrease much faster than MinZ as the disparity shift is increased. Therefore, it is **advised not to use a larger than necessary disparity shift**.
396+
- Tradeoff in reducing the MinZ this way is that objects at **distances farther away than MaxZ will not be seen**.
397+
- Because of the point above, **we only recommend using disparity shift when MaxZ is known**, such as having a depth camera mounted above a table pointing down at the table surface.
398+
- Output disparity map is not expanded, only the depth map. So if disparity shift is set to 50, and disparity value obtained is 90, the real disparity is 140.
399+
400+
**Compared to Extended disparity**, disparity shift:
401+
402+
- **(+)** Is faster, as it doesn't require an extra computation, which means there's also no extra latency
403+
- **(-)** Reduces the MaxZ (significantly), while extended disparity only reduces MinZ.
404+
405+
Disparity shift can be combined with extended disparity.
406+
407+
.. doxygenfunction:: dai::StereoDepthConfig::setDisparityShift
408+
:project: depthai-core
409+
:no-link:
410+
378411
Max stereo depth distance
379-
#########################
412+
=========================
380413

381414
The maximum depth perception distance depends on the :ref:`accuracy of the depth perception <Depth perception accuracy>`. The formula used to calculate this distance is an approximation, but is as follows:
382415

@@ -397,7 +430,7 @@ So using this formula for existing models the *theoretical* max distance is:
397430
If greater precision for long range measurements is required, consider enabling Subpixel Disparity or using a larger baseline distance between mono cameras. For a custom baseline, you could consider using `OAK-FFC <https://docs.luxonis.com/projects/hardware/en/latest/pages/DM1090.html>`__ device or design your own baseboard PCB with required baseline. For more information see Subpixel Disparity under the Stereo Mode tab in :ref:`this table <Currently configurable blocks>`.
398431

399432
Depth perception accuracy
400-
#########################
433+
=========================
401434

402435
Disparity depth works by matching features from one image to the other and its accuracy is based on multiple parameters:
403436

@@ -415,7 +448,7 @@ Lower baseline enables us to detect the depth at a closer distance as long as th
415448
So the common norm is to adjust the baseline according to how far/close we want to be able to detect objects.
416449

417450
Limitation
418-
##########
451+
==========
419452

420453
Since depth is calculated from disparity, which requires the pixels to overlap, there is inherently a vertical
421454
band on the left side of the left mono camera and on the right side of the right mono camera, where depth
@@ -472,7 +505,7 @@ forum post.
472505
`here <https://github.com/luxonis/depthai-hardware/issues/114>`__.
473506

474507
Measuring real-world object dimensions
475-
######################################
508+
======================================
476509

477510
Because the depth map contains the Z distance, objects in parallel with the camera are measured accurately standard. For objects not in parallel, the Euclidean distance calculation can be used. Please refer to the below:
478511

‎docs/source/includes/container-encoding.rst‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,14 @@
11
Encoded bitstream (either MJPEG, H264, or H265) from the device can also be **saved directly into .mp4 container**
22
with no computational overhead on the host computer. See `demo here <https://github.com/luxonis/depthai-experiments/tree/master/gen2-container-encoding>`__
3-
for more information.
3+
for more information.
4+
5+
**Matroska**
6+
7+
Besides ``ffmpeg`` and ``.mp4`` video container (which is patent encumbered), you could also use the ``mkvmerge``
8+
(see `MKVToolNix <https://mkvtoolnix.download/doc/mkvmerge.html>`__ for GUI usage) and ``.mkv`` video container
9+
to mux encoded stream into video file that is supported by all major video players
10+
(eg. `VLC <https://www.videolan.org/vlc/>`__)
11+
12+
.. code-block::
13+
14+
mkvmerge -o vid.mkv video.h265

‎docs/source/samples/Script/script_change_pipeline_flow.rst‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
Script change pipeline flow
2+
===========================
3+
4+
This example shows how you can change the flow of data inside your pipeline in runtime using the :ref:`Script` node. In this example, we send a message from
5+
the host to choose whether we want Script node to forwards color frame to the :ref:`MobileNetDetectionNetwork`.
6+
7+
Demo
8+
####
9+
10+
.. image:: https://user-images.githubusercontent.com/18037362/187734814-df3b46c9-5e04-4a9d-bf6f-d738b40b4421.gif
11+
12+
Pipeline Graph
13+
##############
14+
15+
.. image:: https://user-images.githubusercontent.com/18037362/187736249-db7ff175-fcea-4d4e-b567-f99087bd82ee.png
16+
17+
Setup
18+
#####
19+
20+
.. include:: /includes/install_from_pypi.rst
21+
22+
Source code
23+
###########
24+
25+
.. tabs::
26+
27+
.. tab:: Python
28+
29+
Also `available on GitHub <https://github.com/luxonis/depthai-python/blob/main/examples/Script/script_change_pipeline_flow.py>`__
30+
31+
.. literalinclude:: ../../../../examples/Script/script_change_pipeline_flow.py
32+
:language: python
33+
:linenos:
34+
35+
.. tab:: C++
36+
37+
Also `available on GitHub <https://github.com/luxonis/depthai-core/blob/main/examples/Script/script_change_pipeline_flow.cpp>`__
38+
39+
.. literalinclude:: ../../../../depthai-core/examples/Script/script_change_pipeline_flow.cpp
40+
:language: cpp
41+
:linenos:
42+
43+
.. include:: /includes/footer-short.rst

‎docs/source/samples/SpatialDetection/spatial_calculator_multi_roi.rst‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
Spatial Calculator Multi-ROI
2+
============================
3+
4+
This example shows how one can use multiple ROIs with a single :ref:`SpatailLocationCalculator` node. A similar logic could be used as a simple depth line
5+
scanning camera for mobile robots.
6+
7+
.. rubric:: Similar samples:
8+
9+
- :ref:`Spatial location calculator`
10+
11+
Demo
12+
####
13+
14+
.. image:: https://user-images.githubusercontent.com/18037362/190861621-b57fd1e3-5a3d-4d79-b1a7-d17a0b78c63e.gif
15+
16+
Setup
17+
#####
18+
19+
.. include:: /includes/install_from_pypi.rst
20+
21+
Source code
22+
###########
23+
24+
.. tabs::
25+
26+
.. tab:: Python
27+
28+
Also `available on GitHub <https://github.com/luxonis/depthai-python/blob/main/examples/SpatialDetection/spatial_calculator_multi_roi.py>`__
29+
30+
.. literalinclude:: ../../../../examples/SpatialDetection/spatial_calculator_multi_roi.py
31+
:language: python
32+
:linenos:
33+
34+
.. tab:: C++
35+
36+
Also `available on GitHub <https://github.com/luxonis/depthai-core/blob/main/examples/SpatialDetection/spatial_calculator_multi_roi.cpp>`__
37+
38+
.. literalinclude:: ../../../../depthai-core/examples/SpatialDetection/spatial_calculator_multi_roi.cpp
39+
:language: cpp
40+
:linenos:
41+
42+
.. include:: /includes/footer-short.rst

0 commit comments

Comments
 (0)