Awesome

ONNX-TensorRT

Yolov5(4.0)/Yolov5(5.0)/YoloR/YoloX/Yolov4/Yolov3/CenterNet/CenterFace/RetinaFace/Classify/Unet Implementation

Yolov4/Yolov3/Yolov5/yolor/YoloX

centernet

Unet

<img src="./model/result/unet.jpg" style="zoom:70%;" /> <img src="./model/result/unet_result.png" style="zoom:70%;" />

CenterFace

retinaface

INTRODUCTION

you have the trained model file from the darknet/libtorch/pytorch/mxnet

• yolov5-4.0(5s/5m/5s/5x)
• yolov5-5.0(5s/5m/5s/5x)
• yolov4 , yolov4-tiny
• yolov3 , yolov3-tiny
• yolor
• YoloX
• centernet
• Unet
• CenterFace
• RetinaFace
• classify(mnist\alexnet\resnet18\resnet34\resnet50\shufflenet_v2\mobilenet_v2)

Features

• inequal net width and height

• batch inference

  ---

  onnx-tensorrt batch inference : onnx re-export(batch:2)

• support FP32(m_config.mode = 0),FP16(m_config.mode = 1),INT8(m_config.mode = 2)

• dynamic input size(tiny_tensorrt_dyn_onnx)

window x64 (detect time)

x64(inference / detect time)

windows10

• dependency : spdlog，onnx，onnx-tensorrt，protobuf-3.11.4，TensorRT 7.2.2.3 , cuda 11.1 , cudnn 8.0 , opencv3.4, vs2019

• build:

  open MSVC tiny_tensorrt_onnx.sln file

  tiny_tensorrt_dyn_onnx:dynamic shape

  tiny_tensorrt_onnx: normal

• build onnx-tensorrt

  step1: https://github.com/onnx/onnx-tensorrt.git

  step2: https://drive.google.com/drive/folders/1DndiqyCZ796p3-xXI3O4AMCIGcUWQ1q2?usp=sharing or https://share.weiyun.com/CJCwngAM

  step3: builtin_op_importers.cpp replace onnx-tensorrt\builtin_op_importers.cpp

  step4: tortoiseGit->apply patch serial and choose 0001-Compile-onnx-tensorrt-by-MSVC-on-Windows.patch

  ​ look https://github.com/ttanzhiqiang/onnx-tensorrt_7.2.1

  step5:build onnx.lib\onnx_proto.lib\nvonnxparser.dll\nvonnxparser_static.lib

Model and 3rdparty

model : https://drive.google.com/drive/folders/1KzBjmCOG9ghcq9L6-iqfz6QwBQq6Hl4_?usp=sharing or https://share.weiyun.com/td9CRDhW

3rdparty:https://drive.google.com/drive/folders/1SddUgQ5kGlv6dDGPqnVWZxgCoBY85rM2?usp=sharing or https://share.weiyun.com/WEZ3TGtb

API

struct Config
{
    std::string cfgFile = "configs/yolov3.cfg";

    std::string onnxModelpath = "configs/yolov3.onnx";

    std::string engineFile = "configs/yolov3.engine";

    std::string calibration_image_list_file = "configs/images/";

    std::vector<std::string> customOutput;

    int calibration_width = 0;

    int calibration_height = 0;
    
    int maxBatchSize = 1;

    int mode; //0，1，2

    //std::string calibration_image_list_file_txt = "configs/calibration_images.txt";
};

class YoloDectector
{
void init(Config config);
void detect(const std::vector<cv::Mat>& vec_image,
	std::vector<BatchResult>& vec_batch_result);
}

REFERENCE

https://github.com/onnx/onnx-tensorrt.git

https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleDynamicReshape

https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps

https://github.com/enazoe/yolo-tensorrt.git

https://github.com/zerollzeng/tiny-tensorrt.git

Contact