Awesome
DeepStream-Yolo
NVIDIA DeepStream SDK 7.0 / 6.4 / 6.3 / 6.2 / 6.1.1 / 6.1 / 6.0.1 / 6.0 / 5.1 configuration for YOLO models
For now, I am limited for some updates. Thank you for understanding.
YOLO-Pose: https://github.com/marcoslucianops/DeepStream-Yolo-Pose
YOLO-Seg: https://github.com/marcoslucianops/DeepStream-Yolo-Seg
YOLO-Face: https://github.com/marcoslucianops/DeepStream-Yolo-Face
Important: please export the ONNX model with the new export file, generate the TensorRT engine again with the updated files, and use the new config_infer_primary file according to your model
Future updates
- DeepStream tutorials
- Updated INT8 calibration
- Support for classification models
Improvements on this repository
- Support for INT8 calibration
- Support for non square models
- Models benchmarks
- Support for Darknet models (YOLOv4, etc) using cfg and weights conversion with GPU post-processing
- Support for RT-DETR, YOLO-NAS, PPYOLOE+, PPYOLOE, DAMO-YOLO, YOLOX, YOLOR, YOLOv8, YOLOv7, YOLOv6 and YOLOv5 using ONNX conversion with GPU post-processing
- GPU bbox parser
- Custom ONNX model parser
- Dynamic batch-size
- INT8 calibration (PTQ) for Darknet and ONNX exported models
Getting started
- Requirements
- Supported models
- Benchmarks
- dGPU installation
- Basic usage
- Docker usage
- NMS configuration
- Notes
- INT8 calibration
- YOLOv5 usage
- YOLOv6 usage
- YOLOv7 usage
- YOLOv8 usage
- YOLOR usage
- YOLOX usage
- DAMO-YOLO usage
- PP-YOLOE / PP-YOLOE+ usage
- YOLO-NAS usage
- RT-DETR PyTorch usage
- RT-DETR Paddle usage
- RT-DETR Ultralytics usage
- Using your custom model
- Multiple YOLO GIEs
Requirements
DeepStream 7.0 on x86 platform
- Ubuntu 22.04
- CUDA 12.2 Update 2
- TensorRT 8.6 GA (8.6.1.6)
- NVIDIA Driver 535 (>= 535.161.08)
- NVIDIA DeepStream SDK 7.0
- GStreamer 1.20.3
- DeepStream-Yolo
DeepStream 6.4 on x86 platform
- Ubuntu 22.04
- CUDA 12.2 Update 2
- TensorRT 8.6 GA (8.6.1.6)
- NVIDIA Driver 535 (>= 535.104.12)
- NVIDIA DeepStream SDK 6.4
- GStreamer 1.20.3
- DeepStream-Yolo
DeepStream 6.3 on x86 platform
- Ubuntu 20.04
- CUDA 12.1 Update 1
- TensorRT 8.5 GA Update 2 (8.5.3.1)
- NVIDIA Driver 525 (>= 525.125.06)
- NVIDIA DeepStream SDK 6.3
- GStreamer 1.16.3
- DeepStream-Yolo
DeepStream 6.2 on x86 platform
- Ubuntu 20.04
- CUDA 11.8
- TensorRT 8.5 GA Update 1 (8.5.2.2)
- NVIDIA Driver 525 (>= 525.85.12)
- NVIDIA DeepStream SDK 6.2
- GStreamer 1.16.3
- DeepStream-Yolo
DeepStream 6.1.1 on x86 platform
- Ubuntu 20.04
- CUDA 11.7 Update 1
- TensorRT 8.4 GA (8.4.1.5)
- NVIDIA Driver 515.65.01
- NVIDIA DeepStream SDK 6.1.1
- GStreamer 1.16.2
- DeepStream-Yolo
DeepStream 6.1 on x86 platform
- Ubuntu 20.04
- CUDA 11.6 Update 1
- TensorRT 8.2 GA Update 4 (8.2.5.1)
- NVIDIA Driver 510.47.03
- NVIDIA DeepStream SDK 6.1
- GStreamer 1.16.2
- DeepStream-Yolo
DeepStream 6.0.1 / 6.0 on x86 platform
- Ubuntu 18.04
- CUDA 11.4 Update 1
- TensorRT 8.0 GA (8.0.1)
- NVIDIA Driver 470.63.01
- NVIDIA DeepStream SDK 6.0.1 / 6.0
- GStreamer 1.14.5
- DeepStream-Yolo
DeepStream 5.1 on x86 platform
- Ubuntu 18.04
- CUDA 11.1
- TensorRT 7.2.2
- NVIDIA Driver 460.32.03
- NVIDIA DeepStream SDK 5.1
- GStreamer 1.14.5
- DeepStream-Yolo
DeepStream 7.0 on Jetson platform
DeepStream 6.4 on Jetson platform
DeepStream 6.3 on Jetson platform
- JetPack 5.1.3 / 5.1.2
- NVIDIA DeepStream SDK 6.3
- DeepStream-Yolo
DeepStream 6.2 on Jetson platform
- JetPack 5.1.3 / 5.1.2 / 5.1.1 / 5.1
- NVIDIA DeepStream SDK 6.2
- DeepStream-Yolo
DeepStream 6.1.1 on Jetson platform
DeepStream 6.1 on Jetson platform
DeepStream 6.0.1 / 6.0 on Jetson platform
DeepStream 5.1 on Jetson platform
Supported models
- Darknet
- MobileNet-YOLO
- YOLO-Fastest
- YOLOv5
- YOLOv6
- YOLOv7
- YOLOv8
- YOLOR
- YOLOX
- DAMO-YOLO
- PP-YOLOE / PP-YOLOE+
- YOLO-NAS
Basic usage
1. Download the repo
git clone https://github.com/marcoslucianops/DeepStream-Yolo.git
cd DeepStream-Yolo
2. Download the cfg
and weights
files from Darknet repo to the DeepStream-Yolo folder
3. Compile the lib
3.1. Set the CUDA_VER
according to your DeepStream version
export CUDA_VER=XY.Z
-
x86 platform
DeepStream 7.0 / 6.4 = 12.2 DeepStream 6.3 = 12.1 DeepStream 6.2 = 11.8 DeepStream 6.1.1 = 11.7 DeepStream 6.1 = 11.6 DeepStream 6.0.1 / 6.0 = 11.4 DeepStream 5.1 = 11.1
-
Jetson platform
DeepStream 7.0 / 6.4 = 12.2 DeepStream 6.3 / 6.2 / 6.1.1 / 6.1 = 11.4 DeepStream 6.0.1 / 6.0 / 5.1 = 10.2
3.2. Make the lib
make -C nvdsinfer_custom_impl_Yolo clean && make -C nvdsinfer_custom_impl_Yolo
4. Edit the config_infer_primary.txt
file according to your model (example for YOLOv4)
[property]
...
custom-network-config=yolov4.cfg
model-file=yolov4.weights
...
NOTE: For Darknet models, by default, the dynamic batch-size is set. To use static batch-size, uncomment the line
...
force-implicit-batch-dim=1
...
5. Run
deepstream-app -c deepstream_app_config.txt
NOTE: The TensorRT engine file may take a very long time to generate (sometimes more than 10 minutes).
NOTE: If you want to use YOLOv2 or YOLOv2-Tiny models, change the deepstream_app_config.txt
file before run it
...
[primary-gie]
...
config-file=config_infer_primary_yoloV2.txt
...
Docker usage
-
x86 platform
nvcr.io/nvidia/deepstream:7.0-gc-triton-devel nvcr.io/nvidia/deepstream:7.0-triton-multiarch
-
Jetson platform
nvcr.io/nvidia/deepstream:7.0-triton-multiarch
NOTE: To compile the nvdsinfer_custom_impl_Yolo
, you need to install the g++ inside the container
apt-get install build-essential
NOTE: With DeepStream 7.0, the docker containers do not package libraries necessary for certain multimedia operations like audio data parsing, CPU decode, and CPU encode. This change could affect processing certain video streams/files like mp4 that include audio track. Please run the below script inside the docker images to install additional packages that might be necessary to use all of the DeepStreamSDK features:
/opt/nvidia/deepstream/deepstream/user_additional_install.sh
NMS Configuration
To change the nms-iou-threshold
, pre-cluster-threshold
and topk
values, modify the config_infer file
[class-attrs-all]
nms-iou-threshold=0.45
pre-cluster-threshold=0.25
topk=300
NOTE: Make sure to set cluster-mode=2
in the config_infer file.
Notes
-
Sometimes while running gstreamer pipeline or sample apps, user can encounter error:
GLib (gthread-posix.c): Unexpected error from C library during 'pthread_setspecific': Invalid argument. Aborting.
. The issue is caused because of a bug inglib 2.0-2.72
version which comes with Ubuntu 22.04 by default. The issue is addressed inglib 2.76
and its installation is required to fix the issue (https://github.com/GNOME/glib/tree/2.76.6).-
Migrate
glib
to newer versionpip3 install meson pip3 install ninja
NOTE: It is recommended to use Python virtualenv.
git clone https://github.com/GNOME/glib.git cd glib git checkout 2.76.6 meson build --prefix=/usr ninja -C build/ cd build/ ninja install
-
Check and confirm the newly installed glib version:
pkg-config --modversion glib-2.0
-
-
Sometimes with RTSP streams the application gets stuck on reaching EOS. This is because of an issue in rtpjitterbuffer component. To fix this issue, a script has been provided with required details to update gstrtpmanager library.
/opt/nvidia/deepstream/deepstream/update_rtpmanager.sh
Extract metadata
You can get metadata from DeepStream using Python and C/C++. For C/C++, you can edit the deepstream-app
or deepstream-test
codes. For Python, your can install and edit deepstream_python_apps.
Basically, you need manipulate the NvDsObjectMeta
(Python / C/C++) and NvDsFrameMeta
(Python / C/C++) to get the label, position, etc. of bboxes.
My projects: https://www.youtube.com/MarcosLucianoTV