Awesome
PaddleSharp ๐
English | ็ฎไฝไธญๆ
๐ .NET Wrapper for PaddleInference
C API, support Windows(x64) ๐ป, NVIDIA Cuda 10.2+ based GPU ๐ฎ and Linux(Ubuntu-22.04 x64) ๐ง, currently contained following main components:
- PaddleOCR ๐ support 14 OCR languages model download on-demand, allow rotated text angle detection, 180 degree text detection, also support table recognition ๐.
- PaddleDetection ๐ฏ support PPYolo detection model and PicoDet model ๐น.
- RotationDetection ๐ use Baidu's official
text_image_orientation_infer
model to detect text picture's rotation angle(0, 90, 180, 270
). - PaddleNLP ChineseSegmenter ๐ support
PaddleNLP
Lac Chinese segmenter model, supports tagging/customized words. - Paddle2Onnx ๐ Allow user export
ONNX
model usingC#
.
NuGet Packages/Docker Images ๐ฆ
Release notes ๐
Please checkout this page ๐.
Infrastructure packages ๐๏ธ
NuGet Package ๐ผ | Version ๐ | Description ๐ |
---|---|---|
Sdcb.PaddleInference | Paddle Inference C API .NET binding โ๏ธ |
Native packages ๐๏ธ
Note: cu120
means CUDA 12.0, it's compiled in CUDA 12.0.1/cuDNN 8.9.7.29/Tensor RT 8.6.1.6 version.
Linux OS packages(preview):
Package | Version ๐ | Description |
---|---|---|
Sdcb.PaddleInference.runtime.linux-loongarch64 | Loongnix GCC 8.2 Loongarch64 | |
Sdcb.PaddleInference.runtime.linux64.mkl.gcc82 | Linux-x64 GCC 8.2(tested in Ubuntu 22.04) |
Be aware, as the Linux operating system cannot modify the value of LD_LIBRARY_PATH
at runtime. If dependent dynamic libraries (such as libcommon.so) are loaded before the main dynamic library (such as libpaddle_inference_c.so), and also due to protobuf errors reported: https://github.com/PaddlePaddle/Paddle/issues/62670
Therefore, all NuGet packages for Linux operating systems are in a preview state, and I'm unable to resolve this issue. Currently, if you are using the NuGet package on Linux, you need to manually specify the LD_LIBRARY_PATH
environment variable before running the program, using the following commands:
-
For x64 CPUs:
export LD_LIBRARY_PATH=/<program directory>/bin/Debug/net8.0/runtimes/linux-x64/native:$LD_LIBRARY_PATH
-
For Loongson 5000 or above CPUs (linux-loongarch64):
export LD_LIBRARY_PATH=/<program directory>/bin/Debug/net8.0/runtimes/linux-loongarch64/native:$LD_LIBRARY_PATH
Some of packages already deprecated(Version <= 2.5.0):
Any other packages that starts with Sdcb.PaddleInference.runtime
might deprecated.
Baidu packages were downloaded from here: https://www.paddlepaddle.org.cn/inference/master/guides/install/download_lib.html#windows
All Windows packages were compiled manually by me.
Baidu official GPU packages are too large(>1.5GB) to publish to nuget.org, there is a limitation of 250MB when upload to Github, there is some related issues to this:
- https://github.com/PaddlePaddle/Paddle/issues/43874 โ
- https://github.com/NuGet/Home/issues/11706#issuecomment-1167305006 โ
But You're good to build your own GPU nuget package using 01-build-native.linq
๐ ๏ธ.
Paddle Devices
-
Mkldnn -
PaddleDevice.Mkldnn()
Based on Mkldnn, generally fast
-
Openblas -
PaddleDevice.Openblas()
Based on openblas, slower, but dependencies file smaller and consume lesser memory
-
Onnx -
PaddleDevice.Onnx()
Based on onnxruntime, is also pretty fast and consume less memory
-
Gpu -
PaddleDevice.Gpu()
Much faster but relies on NVIDIA GPU and CUDA
If you wants to use GPU, you should refer to FAQ
How to enable GPU?
section, CUDA/cuDNN/TensorRT need to be installed manually.
FAQ โ
Why my code runs good in my windows machine, but DllNotFoundException in other machine: ๐ป
-
Please ensure the latest Visual C++ Redistributable was installed in
Windows
(typically it should automatically installed if you haveVisual Studio
installed) ๐ ๏ธ Otherwise, it will fail with the following error (Windows only):DllNotFoundException: Unable to load DLL 'paddle_inference_c' or one of its dependencies (0x8007007E)
If it's Unable to load DLL OpenCvSharpExtern.dll or one of its dependencies, then most likely the Media Foundation is not installed in the Windows Server 2012 R2 machine: <img width="830" alt="image" src="https://user-images.githubusercontent.com/1317141/193706883-6a71ea83-65d9-448b-afee-2d25660430a1.png">
-
Many old CPUs do not support AVX instructions, please ensure your CPU supports AVX, or download the x64-noavx-openblas DLLs and disable Mkldnn:
PaddleDevice.Openblas()
๐ -
If you're using Win7-x64, and your CPU does support AVX2, then you might also need to extract the following 3 DLLs into
C:\Windows\System32
folder to make it run: ๐พ- api-ms-win-core-libraryloader-l1-2-0.dll
- api-ms-win-core-processtopology-obsolete-l1-1-0.dll
- API-MS-Win-Eventing-Provider-L1-1-0.dll
You can download these 3 DLLs here: win7-x64-onnxruntime-missing-dlls.zip โฌ๏ธ
How to enable GPU? ๐ฎ
Enable GPU support can significantly improve the throughput and lower the CPU usage. ๐
Steps to use GPU in Windows:
- (for Windows) Install the package:
Sdcb.PaddleInference.runtime.win64.cu120*
instead ofSdcb.PaddleInference.runtime.win64.mkl
, do not install both. ๐ฆ - Install CUDA from NVIDIA, and configure environment variables to
PATH
orLD_LIBRARY_PATH
(Linux) ๐ง - Install cuDNN from NVIDIA, and configure environment variables to
PATH
orLD_LIBRARY_PATH
(Linux) ๐ ๏ธ - Install TensorRT from NVIDIA, and configure environment variables to
PATH
orLD_LIBRARY_PATH
(Linux) โ๏ธ
You can refer to this blog page for GPU in Windows: ๅ ณไบPaddleSharp GPUไฝฟ็จ ๅธธ่ง้ฎ้ข่ฎฐๅฝ ๐
If you're using Linux, you need to compile your own OpenCvSharp4 environment following the docker build scripts and the CUDA/cuDNN/TensorRT configuration tasks. ๐ง
After these steps are completed, you can try specifying PaddleDevice.Gpu()
in the paddle device configuration parameter, then enjoy the performance boost! ๐
Thanks & Sponsors ๐
- ๅดไบฎ https://github.com/cuiliang
- ๆข็ฐไผ
- ๆทฑๅณ-้ฑๆๆพ
- iNeuOSๅทฅไธไบ่็ฝๆไฝ็ณป็ป๏ผhttp://www.ineuos.net
Contact ๐
QQ group of C#/.NET computer vision technical communication (C#/.NET่ฎก็ฎๆบ่ง่งๆๆฏไบคๆต็พค): 579060605