Awesome
onnx-test
Setup steps
1.setup
pip install optimum[exporters,onnxruntime]
2.convert onnx
optimum-cli export onnx --model sentence-transformers/all-MiniLM-L6-v2 all-MiniLM-L6-v2-onnx
Better example
optimum-cli export onnx -m Helsinki-NLP/opus-mt-zh-en --optimize O2 optus-mt-zh-en-onnx
OnnxRuntime
optimum-cli onnxruntime quantize \
--avx512 \
--onnx_model bert-tiny-onnx \
-o quantized_bert-tiny-onnx