Home

Awesome

PaddleSharp ๐ŸŒŸ main QQ

English | ็ฎ€ไฝ“ไธญๆ–‡

๐Ÿ’— .NET Wrapper for PaddleInference C API, support Windows(x64) ๐Ÿ’ป, NVIDIA Cuda 10.2+ based GPU ๐ŸŽฎ and Linux(Ubuntu-22.04 x64) ๐Ÿง, currently contained following main components:

NuGet Packages/Docker Images ๐Ÿ“ฆ

Release notes ๐Ÿ“

Please checkout this page ๐Ÿ“„.

Infrastructure packages ๐Ÿ—๏ธ

NuGet Package ๐Ÿ’ผVersion ๐Ÿ“ŒDescription ๐Ÿ“š
Sdcb.PaddleInferenceNuGetPaddle Inference C API .NET binding โš™๏ธ

Native packages ๐Ÿ—๏ธ

PackageVersion ๐Ÿ“ŒDescription
Sdcb.PaddleInference.runtime.win64.mklNuGetwin64+mkldnn
Sdcb.PaddleInference.runtime.win64.openblasNuGetwin64+openblas
Sdcb.PaddleInference.runtime.win64.openblas-noavxNuGetno AVX, for old CPUs
Sdcb.PaddleInference.runtime.win64.cu120-sm86-89NuGetfor NVIDIA 30/40 series
Sdcb.PaddleInference.runtime.win64.cu120-sm80NuGetfor NVIDIA A100/A10
Sdcb.PaddleInference.runtime.win64.cu120-sm61-75NuGetfor NVIDIA 10/20 series

Note: cu120 means CUDA 12.0, it's compiled in CUDA 12.0.1/cuDNN 8.9.7.29/Tensor RT 8.6.1.6 version.

Linux OS packages(preview):

PackageVersion ๐Ÿ“ŒDescription
Sdcb.PaddleInference.runtime.linux-loongarch64NuGetLoongnix GCC 8.2 Loongarch64
Sdcb.PaddleInference.runtime.linux64.mkl.gcc82NuGetLinux-x64 GCC 8.2(tested in Ubuntu 22.04)

Be aware, as the Linux operating system cannot modify the value of LD_LIBRARY_PATH at runtime. If dependent dynamic libraries (such as libcommon.so) are loaded before the main dynamic library (such as libpaddle_inference_c.so), and also due to protobuf errors reported: https://github.com/PaddlePaddle/Paddle/issues/62670

Therefore, all NuGet packages for Linux operating systems are in a preview state, and I'm unable to resolve this issue. Currently, if you are using the NuGet package on Linux, you need to manually specify the LD_LIBRARY_PATH environment variable before running the program, using the following commands:

Some of packages already deprecated(Version <= 2.5.0):

PackageVersion ๐Ÿ“ŒDescription
Sdcb.PaddleInference.runtime.win64.cuda102_cudnn76_tr72_sm61_75NuGetwin64/CUDA 10.2/cuDNN 7.6/TensorRT 7.2/sm61+sm75
Sdcb.PaddleInference.runtime.win64.cuda118_cudnn86_tr85_sm86_89NuGetwin64/CUDA 11.8/cuDNN 8.6/TensorRT 8.5/sm86+sm89
Sdcb.PaddleInference.runtime.win64.cuda117_cudnn84_tr84_sm86NuGetwin64/CUDA 11.7/cuDNN 8.4/TensorRT 8.4/sm86
Sdcb.PaddleInference.runtime.win64.cuda102_cudnn76_sm61_75NuGetwin64/CUDA 10.2/cuDNN 7.6/sm61+sm75
Sdcb.PaddleInference.runtime.win64.cuda116_cudnn84_sm86_onnxNuGetwin64/CUDA 11.6/cuDNN 8.4/sm86/onnx

Any other packages that starts with Sdcb.PaddleInference.runtime might deprecated.

Baidu packages were downloaded from here: https://www.paddlepaddle.org.cn/inference/master/guides/install/download_lib.html#windows

All Windows packages were compiled manually by me.

Baidu official GPU packages are too large(>1.5GB) to publish to nuget.org, there is a limitation of 250MB when upload to Github, there is some related issues to this:

But You're good to build your own GPU nuget package using 01-build-native.linq ๐Ÿ› ๏ธ.

Paddle Devices

FAQ โ“

Why my code runs good in my windows machine, but DllNotFoundException in other machine: ๐Ÿ’ป

  1. Please ensure the latest Visual C++ Redistributable was installed in Windows (typically it should automatically installed if you have Visual Studio installed) ๐Ÿ› ๏ธ Otherwise, it will fail with the following error (Windows only):

    DllNotFoundException: Unable to load DLL 'paddle_inference_c' or one of its dependencies (0x8007007E)
    

    If it's Unable to load DLL OpenCvSharpExtern.dll or one of its dependencies, then most likely the Media Foundation is not installed in the Windows Server 2012 R2 machine: <img width="830" alt="image" src="https://user-images.githubusercontent.com/1317141/193706883-6a71ea83-65d9-448b-afee-2d25660430a1.png">

  2. Many old CPUs do not support AVX instructions, please ensure your CPU supports AVX, or download the x64-noavx-openblas DLLs and disable Mkldnn: PaddleDevice.Openblas() ๐Ÿš€

  3. If you're using Win7-x64, and your CPU does support AVX2, then you might also need to extract the following 3 DLLs into C:\Windows\System32 folder to make it run: ๐Ÿ’พ

    • api-ms-win-core-libraryloader-l1-2-0.dll
    • api-ms-win-core-processtopology-obsolete-l1-1-0.dll
    • API-MS-Win-Eventing-Provider-L1-1-0.dll

    You can download these 3 DLLs here: win7-x64-onnxruntime-missing-dlls.zip โฌ‡๏ธ

How to enable GPU? ๐ŸŽฎ

Enable GPU support can significantly improve the throughput and lower the CPU usage. ๐Ÿš€

Steps to use GPU in Windows:

  1. (for Windows) Install the package: Sdcb.PaddleInference.runtime.win64.cu120* instead of Sdcb.PaddleInference.runtime.win64.mkl, do not install both. ๐Ÿ“ฆ
  2. Install CUDA from NVIDIA, and configure environment variables to PATH or LD_LIBRARY_PATH (Linux) ๐Ÿ”ง
  3. Install cuDNN from NVIDIA, and configure environment variables to PATH or LD_LIBRARY_PATH (Linux) ๐Ÿ› ๏ธ
  4. Install TensorRT from NVIDIA, and configure environment variables to PATH or LD_LIBRARY_PATH (Linux) โš™๏ธ

You can refer to this blog page for GPU in Windows: ๅ…ณไบŽPaddleSharp GPUไฝฟ็”จ ๅธธ่ง้—ฎ้ข˜่ฎฐๅฝ• ๐Ÿ“

If you're using Linux, you need to compile your own OpenCvSharp4 environment following the docker build scripts and the CUDA/cuDNN/TensorRT configuration tasks. ๐Ÿง

After these steps are completed, you can try specifying PaddleDevice.Gpu() in the paddle device configuration parameter, then enjoy the performance boost! ๐ŸŽ‰

Thanks & Sponsors ๐Ÿ™

Contact ๐Ÿ“ž

QQ group of C#/.NET computer vision technical communication (C#/.NET่ฎก็ฎ—ๆœบ่ง†่ง‰ๆŠ€ๆœฏไบคๆต็พค): 579060605