Awesome

ONNX Model Zoo

Introduction

Welcome to the ONNX Model Zoo! The Open Neural Network Exchange (ONNX) is an open standard format created to represent machine learning models. Supported by a robust community of partners, ONNX defines a common set of operators and a common file format to enable AI developers to use models with a variety of frameworks, tools, runtimes, and compilers.

This repository is a curated collection of pre-trained, state-of-the-art models in the ONNX format. These models are sourced from prominent open-source repositories and have been contributed by a diverse group of community members. Our aim is to facilitate the spread and usage of machine learning models among a wider audience of developers, researchers, and enthusiasts.

To handle ONNX model files, which can be large, we use Git LFS (Large File Storage).

Models

Currently, we are expanding the ONNX Model Zoo by incorporating additional models from the following categories. As we are rigorously validating the new models for accuracy, refer to the validated models below that have been successfully validated for accuracy:

• Computer Vision
• Natural Language Processing (NLP)
• Generative AI
• Graph Machine Learning

These models are sourced from prominent open-source repositories such as timm, torchvision, torch_hub, and transformers, and exported into the ONNX format using the open-source TurnkeyML toolchain.

Validated Models

Vision

• Image Classification
• Object Detection & Image Segmentation
• Body, Face & Gesture Analysis
• Image Manipulation

Language

• Machine Comprehension
• Machine Translation
• Language Modelling

Other

• Visual Question Answering & Dialog
• Speech & Audio Processing
• Other interesting models

Read the Usage section below for more details on the file formats in the ONNX Model Zoo (.onnx, .pb, .npz), downloading multiple ONNX models through Git LFS command line, and starter Python code for validating your ONNX model using test data.

INT8 models are generated by Intel® Neural Compressor. Intel® Neural Compressor is an open-source Python library which supports automatic accuracy-driven tuning strategies to help user quickly find out the best quantized model. It implements dynamic and static quantization for ONNX models and can represent quantized ONNX models with operator oriented as well as tensor oriented (QDQ) ways. Users can use web-based UI service or python code to do quantization. Read the Introduction for more details.

Image Classification <a name="image_classification"/>

This collection of models take images as input, then classifies the major objects in the images into 1000 object categories such as keyboard, mouse, pencil, and many animals.

Model Class	Reference	Description	Huggingface Spaces
<b>MobileNet</b>	Sandler et al.	Light-weight deep neural network best suited for mobile and embedded vision applications. <br>Top-5 error from paper - ~10%
<b>ResNet</b>	He et al.	A CNN model (up to 152 layers). Uses shortcut connections to achieve higher accuracy when classifying images. <br> Top-5 error from paper - ~3.6%
<b>SqueezeNet</b>	Iandola et al.	A light-weight CNN model providing AlexNet level accuracy with 50x fewer parameters. <br>Top-5 error from paper - ~20%
<b>VGG</b>	Simonyan et al.	Deep CNN model(up to 19 layers). Similar to AlexNet but uses multiple smaller kernel-sized filters that provides more accuracy when classifying images. <br>Top-5 error from paper - ~8%
<b>AlexNet</b>	Krizhevsky et al.	A Deep CNN model (up to 8 layers) where the input is an image and the output is a vector of 1000 numbers. <br> Top-5 error from paper - ~15%
<b>GoogleNet</b>	Szegedy et al.	Deep CNN model(up to 22 layers). Comparatively smaller and faster than VGG and more accurate in detailing than AlexNet. <br> Top-5 error from paper - ~6.7%
<b>CaffeNet</b>	Krizhevsky et al.	Deep CNN variation of AlexNet for Image Classification in Caffe where the max pooling precedes the local response normalization (LRN) so that the LRN takes less compute and memory.
<b>RCNN_ILSVRC13</b>	Girshick et al.	Pure Caffe implementation of R-CNN for image classification. This model uses localization of regions to classify and extract features from images.
<b>DenseNet-121</b>	Huang et al.	Model that has every layer connected to every other layer and passes on its own feature providing strong gradient flow and more diversified features.
<b>Inception_V1</b>	Szegedy et al.	This model is same as GoogLeNet, implemented through Caffe2 that has improved utilization of the computing resources inside the network and helps with the vanishing gradient problem. <br> Top-5 error from paper - ~6.7%
<b>Inception_V2</b>	Szegedy et al.	Deep CNN model for Image Classification as an adaptation to Inception v1 with batch normalization. This model has reduced computational cost and improved image resolution compared to Inception v1. <br> Top-5 error from paper ~4.82%
<b>ShuffleNet_V1</b>	Zhang et al.	Extremely computation efficient CNN model that is designed specifically for mobile devices. This model greatly reduces the computational cost and provides a ~13x speedup over AlexNet on ARM-based mobile devices. Compared to MobileNet, ShuffleNet achieves superior performance by a significant margin due to it's efficient structure. <br> Top-1 error from paper - ~32.6%
<b>ShuffleNet_V2</b>	Zhang et al.	Extremely computation efficient CNN model that is designed specifically for mobile devices. This network architecture design considers direct metric such as speed, instead of indirect metric like FLOP. <br> Top-1 error from paper - ~30.6%
<b>ZFNet-512</b>	Zeiler et al.	Deep CNN model (up to 8 layers) that increased the number of features that the network is capable of detecting that helps to pick image features at a finer level of resolution. <br> Top-5 error from paper - ~14.3%
<b>EfficientNet-Lite4</b>	Tan et al.	CNN model with an order of magnitude of few computations and parameters, while still acheiving state-of-the-art accuracy and better efficiency than previous ConvNets. <br> Top-5 error from paper - ~2.9%

Domain-based Image Classification <a name="domain_based_image"/>

This subset of models classify images for specific domains and datasets.

Object Detection & Image Segmentation <a name="object_detection"/>

Object detection models detect the presence of multiple objects in an image and segment out areas of the image where the objects are detected. Semantic segmentation models partition an input image by labeling each pixel into a set of pre-defined categories.

Model Class	Reference	Description	Hugging Face Spaces
<b>Tiny YOLOv2</b>	Redmon et al.	A real-time CNN for object detection that detects 20 different classes. A smaller version of the more complex full YOLOv2 network.
<b>SSD</b>	Liu et al.	Single Stage Detector: real-time CNN for object detection that detects 80 different classes.
<b>SSD-MobileNetV1</b>	Howard et al.	A variant of MobileNet that uses the Single Shot Detector (SSD) model framework. The model detects 80 different object classes and locates up to 10 objects in an image.
<b>Faster-RCNN</b>	Ren et al.	Increases efficiency from R-CNN by connecting a RPN with a CNN to create a single, unified network for object detection that detects 80 different classes.
<b>Mask-RCNN</b>	He et al.	A real-time neural network for object instance segmentation that detects 80 different classes. Extends Faster R-CNN as each of the 300 elected ROIs go through 3 parallel branches of the network: label prediction, bounding box prediction and mask prediction.
<b>RetinaNet</b>	Lin et al.	A real-time dense detector network for object detection that addresses class imbalance through Focal Loss. RetinaNet is able to match the speed of previous one-stage detectors and defines the state-of-the-art in two-stage detectors (surpassing R-CNN).
<b>YOLO v2-coco</b>	Redmon et al.	A CNN model for real-time object detection system that can detect over 9000 object categories. It uses a single network evaluation, enabling it to be more than 1000x faster than R-CNN and 100x faster than Faster R-CNN. This model is trained with COCO dataset and contains 80 classes.
<b>YOLO v3</b>	Redmon et al.	A deep CNN model for real-time object detection that detects 80 different classes. A little bigger than YOLOv2 but still very fast. As accurate as SSD but 3 times faster.
<b>Tiny YOLOv3</b>	Redmon et al.	A smaller version of YOLOv3 model.
<b>YOLOv4</b>	Bochkovskiy et al.	Optimizes the speed and accuracy of object detection. Two times faster than EfficientDet. It improves YOLOv3's AP and FPS by 10% and 12%, respectively, with mAP50 of 52.32 on the COCO 2017 dataset and FPS of 41.7 on a Tesla V100.
<b>DUC</b>	Wang et al.	Deep CNN based pixel-wise semantic segmentation model with >80% mIOU (mean Intersection Over Union). Trained on cityscapes dataset, which can be effectively implemented in self driving vehicle systems.
<b>FCN	Long et al.	Deep CNN based segmentation model trained end-to-end, pixel-to-pixel that produces efficient inference and learning. Built off of AlexNet, VGG net, GoogLeNet classification methods. <br>contribute

Body, Face & Gesture Analysis <a name="body_analysis"/>

Face detection models identify and/or recognize human faces and emotions in given images. Body and Gesture Analysis models identify gender and age in given image.

Model Class	Reference	Description	Hugging Face Spaces
<b>ArcFace</b>	Deng et al.	A CNN based model for face recognition which learns discriminative features of faces and produces embeddings for input face images.
<b>UltraFace</b>	Ultra-lightweight face detection model	This model is a lightweight facedetection model designed for edge computing devices.
<b>Emotion FerPlus</b>	Barsoum et al.	Deep CNN for emotion recognition trained on images of faces.
<b>Age and Gender Classification using Convolutional Neural Networks</b>	Rothe et al.	This model accurately classifies gender and age even the amount of learning data is limited.

Image Manipulation <a name="image_manipulation"/>

Image manipulation models use neural networks to transform input images to modified output images. Some popular models in this category involve style transfer or enhancing images by increasing resolution.

Model Class	Reference	Description	Hugging Face Spaces
Unpaired Image to Image Translation using Cycle consistent Adversarial Network	Zhu et al.	The model uses learning to translate an image from a source domain X to a target domain Y in the absence of paired examples. <br>contribute
<b>Super Resolution with sub-pixel CNN</b>	Shi et al.	A deep CNN that uses sub-pixel convolution layers to upscale the input image.
<b>Fast Neural Style Transfer</b>	Johnson et al.	This method uses a loss network pretrained for image classification to define perceptual loss functions that measure perceptual differences in content and style between images. The loss network remains fixed during the training process.

Speech & Audio Processing <a name="speech"/>

This class of models uses audio data to train models that can identify voice, generate music, or even read text out loud.

Machine Comprehension <a name="machine_comprehension"/>

This subset of natural language processing models that answer questions about a given context paragraph.

Model Class	Reference	Description	Hugging Face Spaces
<b>Bidirectional Attention Flow</b>	Seo et al.	A model that answers a query about a given context paragraph.
<b>BERT-Squad</b>	Devlin et al.	This model answers questions based on the context of the given input paragraph.
<b>RoBERTa</b>	Liu et al.	A large transformer-based model that predicts sentiment based on given input text.
<b>GPT-2</b>	Radford et al.	A large transformer-based language model that given a sequence of words within some text, predicts the next word.
<b>T5</b>	Raffel et al.	A large transformer-based language model trained on multiple tasks at once to achieve better semantic understanding of the prompt, capable of sentiment-analysis, question-answering, similarity-detection, translation, summarization, etc.

Machine Translation <a name="machine_translation"/>

This class of natural language processing models learns how to translate input text to another language.

Language Modelling <a name="language_modelling"/>

This subset of natural language processing models learns representations of language from large corpuses of text.

Visual Question Answering & Dialog <a name="visual_qna"/>

This subset of natural language processing models uses input images to answer questions about those images.

Other interesting models <a name="others"/>

There are many interesting deep learning models that do not fit into the categories described above. The ONNX team would like to highly encourage users and researchers to contribute their models to the growing model zoo.

Usage <a name="usage-"/>

Every ONNX backend should support running the models out of the box. After downloading and extracting the tarball of each model, you will find:

• A protobuf file model.onnx that represents the serialized ONNX model.
• Test data (in the form of serialized protobuf TensorProto files or serialized NumPy archives).

Usage - Test data starter code

The test data files can be used to validate ONNX models from the Model Zoo. We have provided the following interface examples for you to get started. Please replace onnx_backend in your code with the appropriate framework of your choice that provides ONNX inferencing support, and likewise replace backend.run_model with the framework's model evaluation logic.

There are two different formats for the test data files:

• Serialized protobuf TensorProtos (.pb), stored in folders with the naming convention test_data_set_*.

import numpy as np
import onnx
import os
import glob
import onnx_backend as backend

from onnx import numpy_helper

model = onnx.load('model.onnx')
test_data_dir = 'test_data_set_0'

# Load inputs
inputs = []
inputs_num = len(glob.glob(os.path.join(test_data_dir, 'input_*.pb')))
for i in range(inputs_num):
    input_file = os.path.join(test_data_dir, 'input_{}.pb'.format(i))
    tensor = onnx.TensorProto()
    with open(input_file, 'rb') as f:
        tensor.ParseFromString(f.read())
    inputs.append(numpy_helper.to_array(tensor))

# Load reference outputs
ref_outputs = []
ref_outputs_num = len(glob.glob(os.path.join(test_data_dir, 'output_*.pb')))
for i in range(ref_outputs_num):
    output_file = os.path.join(test_data_dir, 'output_{}.pb'.format(i))
    tensor = onnx.TensorProto()
    with open(output_file, 'rb') as f:
        tensor.ParseFromString(f.read())
    ref_outputs.append(numpy_helper.to_array(tensor))

# Run the model on the backend
outputs = list(backend.run_model(model, inputs))

# Compare the results with reference outputs.
for ref_o, o in zip(ref_outputs, outputs):
    np.testing.assert_almost_equal(ref_o, o)

• Serialized Numpy archives, stored in files with the naming convention test_data_*.npz. Each file contains one set of test inputs and outputs.

import numpy as np
import onnx
import onnx_backend as backend

# Load the model and sample inputs and outputs
model = onnx.load(model_pb_path)
sample = np.load(npz_path, encoding='bytes')
inputs = list(sample['inputs'])
outputs = list(sample['outputs'])

# Run the model with an onnx backend and verify the results
np.testing.assert_almost_equal(outputs, backend.run_model(model, inputs))

Usage - Model quantization

You can get quantized ONNX models by using Intel® Neural Compressor. It provides web-based UI service to make quantization easier and supports code-based usage for more abundant quantization settings. Refer to bench document for how to use web-based UI service and example document for a simple code-based demo.

Usage

There are multiple ways to access the ONNX Model Zoo:

Git Clone (Not Recommended)

Cloning the repository using git won't automatically download the ONNX models due to their size. To manage these files, first, install Git LFS by running:

pip install git-lfs

To download a specific model:

git lfs pull --include="[path to model].onnx" --exclude=""

To download all models:

git lfs pull --include="*" --exclude=""

GitHub UI

Alternatively, you can download models directly from GitHub. Navigate to the model's page and click the "Download" button on the top right corner.

Model Visualization

For a graphical representation of each model's architecture, we recommend using Netron.

Contributions

Contributions to the ONNX Model Zoo are welcome! Please check our contribution guidelines for more information on how you can contribute to the growth and improvement of this resource.

Thank you for your interest in the ONNX Model Zoo, and we look forward to your participation in our community!

License

Apache License v2.0