Home

Awesome

<a href="https://scan.coverity.com/projects/intel-models"> <img alt="Coverity Scan Build Status" src="https://scan.coverity.com/projects/30278/badge.svg"/> </a> </a> <a href="https://www.bestpractices.dev/projects/8925"><img src="https://www.bestpractices.dev/projects/8925/badge"></a>

Intel® AI Reference Models

This repository contains links to pre-trained models, sample scripts, best practices, and step-by-step tutorials for many popular open-source machine learning models optimized by Intel to run on Intel® Xeon® Scalable processors and Intel® Data Center GPUs.

Containers for running the workloads can be found at Intel® AI Containers.

Intel® AI Reference Models in a Jupyter Notebook is also available for the listed workloads

Purpose of Intel® AI Reference Models

Intel optimizes popular deep learning frameworks such as TensorFlow* and PyTorch* by contributing to the upstream projects. Additional optimizations are built into plugins/extensions such as the Intel Extension for Pytorch* and the Intel Extension for TensorFlow*. Popular neural network models running against common datasets are the target workloads that drive these optimizations.

The purpose of the Intel® AI Reference Models repository (and associated containers) is to quickly replicate the complete software environment that demonstrates the best-known performance of each of these target model/dataset combinations. When executed in optimally-configured hardware environments, these software environments showcase the AI capabilities of Intel platforms.

DISCLAIMER: These scripts are not intended for benchmarking Intel platforms. For any performance and/or benchmarking information on specific Intel platforms, visit https://www.intel.ai/blog.

Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

License

The Intel® AI Reference Models is licensed under Apache License Version 2.0.

Datasets

To the extent that any public datasets are referenced by Intel or accessed using tools or code on this site those datasets are provided by the third party indicated as the data source. Intel does not create the data, or datasets, and does not warrant their accuracy or quality. By accessing the public dataset(s) you agree to the terms associated with those datasets and that your use complies with the applicable license.

Please check the list of datasets used in Intel® AI Reference Models in datasets directory.

Intel expressly disclaims the accuracy, adequacy, or completeness of any public datasets, and is not liable for any errors, omissions, or defects in the data, or for any reliance on the data. Intel is not liable for any liability or damages relating to your use of public datasets.

Use cases

The model documentation in the tables below have information on the prerequisites to run each model. The model scripts run on Linux. Certain models are also able to run using bare metal on Windows. For more information and a list of models that are supported on Windows, see the documentation here.

Instructions available to run on Sapphire Rapids.

For best performance on Intel® Data Center GPU Flex and Max Series, please check the list of supported workloads. It provides instructions to run inference and training using Intel(R) Extension for PyTorch or Intel(R) Extension for TensorFlow.

Image Recognition

ModelFrameworkModeModel DocumentationBenchmark/Test Dataset
ResNet 50v1.5 Sapphire RapidsTensorFlowInferenceInt8 FP32 BFloat16 BFloat32ImageNet 2012
ResNet 50v1.5 Sapphire RapidsTensorFlowTrainingFP32 BFloat16 BFloat32ImageNet 2012
ResNet 50PyTorchInferenceInt8 FP32 BFloat16 BFloat32[ImageNet 2012]
ResNet 50PyTorchTrainingFP32 BFloat16 BFloat32[ImageNet 2012]
Vision TransformerPyTorchInferenceFP32 BFloat16 BFloat32 FP16 INT8[ImageNet 2012]

Image Segmentation

ModelFrameworkModeModel DocumentationBenchmark/Test Dataset
3D U-NetTensorFlowInferenceFP32 BFloat16 Int8BRATS 2018

Language Modeling

ModelFrameworkModeModel DocumentationBenchmark/Test Dataset
BERT large Sapphire RapidsTensorflowInferenceFP32 BFloat16 Int8 BFloat32SQuAD
BERT large Sapphire RapidsTensorflowTrainingFP32 BFloat16 BFloat32SQuAD
BERT large (Hugging Face)TensorFlowInferenceFP32 FP16 BFloat16 BFloat32SQuAD
BERT largePyTorchInferenceFP32 Int8 BFloat16 BFloat32BERT Large SQuAD1.1
BERT largePyTorchTrainingFP32 BFloat16 BFloat32preprocessed text dataset
DistilBERT basePyTorchInferenceFP32 BF32 BF16Int8-FP32 Int8-BFloat16 BFloat32 DistilBERT Base SQuAD1.1
RNN-TPyTorchInferenceFP32 BFloat16 BFloat32RNN-T dataset
RNN-TPyTorchTrainingFP32 BFloat16 BFloat32RNN-T dataset
GPTJ 6BPyTorchInferenceFP32 FP16 BFloat16 BF32 INT8
GPTJ 6B MLPerfPyTorchInferenceINT4CNN-Daily Mail dataset
LLAMA2 7BPyTorchInferenceFP32 FP16 BFloat16 BF32 INT8
LLAMA2 7BPyTorchTrainingFP32 FP16 BFloat16 BF32
LLAMA2 13BPyTorchInferenceFP32 FP16 BFloat16 BF32 INT8
ChatGLMv3 6BPyTorchInferenceFP32 FP16 BFloat16 BF32 INT8

Language Translation

ModelFrameworkModeModel DocumentationBenchmark/Test Dataset
BERTTensorFlowInferenceFP32MRPC

Object Detection

ModelFrameworkModeModel DocumentationBenchmark/Test Dataset
Mask R-CNNPyTorchInferenceFP32 BFloat16 BFloat32COCO 2017
Mask R-CNNPyTorchTrainingFP32 BFloat16 BFloat32COCO 2017
SSD-ResNet34PyTorchInferenceFP32 Int8 BFloat16 BFloat32COCO 2017
SSD-ResNet34PyTorchTrainingFP32 BFloat16 BFloat32COCO 2017
Yolo V7PyTorchInferenceInt8 FP32 FP16 BFloat16 BFloat32[COCO 2017](/models_v2/pytorch/yolov7/inference/cpu/README.md## Prepare Dataset)

Recommendation

ModelFrameworkModeModel DocumentationBenchmark/Test Dataset
Wide & DeepTensorFlowInferenceFP32Census Income dataset
DLRMPyTorchInferenceFP32 Int8 BFloat16 BFloat32Criteo Terabyte
DLRMPyTorchTrainingFP32 BFloat16 BFloat32Criteo Terabyte
DLRM v2PyTorchInferenceFP32 FP16 BFloat16 BFloat32 Int8Criteo 1TB Click Logs dataset

Diffusion

ModelFrameworkModeModel DocumentationBenchmark/Test Dataset
Stable DiffusionTensorFlowInferenceFP32 BFloat16 FP16COCO 2017 validation dataset
Stable DiffusionPyTorchInferenceFP32 BFloat16 FP16 BFloat32 Int8-FP32 Int8-BFloat16COCO 2017 validation dataset
Stable DiffusionPyTorchTrainingFP32 BFloat16 FP16 BFloat32cat images
Latent Consistency Models(LCM)PyTorchInferenceFP32 BFloat16 FP16 BFloat32 Int8-FP32 Int8-BFloat16COCO 2017 validation dataset

Graph Networks

ModelFrameworkModeModel DocumentationBenchmark/Test Dataset
GraphSAGETensorFlowInferenceFP32 BFloat16 FP16 Int8 BFloat32Protein Protein Interaction

*Means the model belongs to MLPerf models and will be supported long-term.

Intel® Data Center GPU Workloads

ModelFrameworkModeGPU TypeModel Documentation
ResNet 50v1.5TensorFlowInferenceFlex SeriesFloat32 TF32 Float16 BFloat16 Int8
ResNet 50 v1.5TensorFlowTrainingMax SeriesBFloat16 FP32
ResNet 50 v1.5PyTorchInferenceFlex Series, Max Series, Arc SeriesInt8 FP32 FP16 TF32
ResNet 50 v1.5PyTorchTrainingMax Series, Arc SeriesBFloat16 TF32 FP32
DistilBERTPyTorchInferenceFlex Series, Max SeriesFP32 FP16 BF16 TF32
DLRM v1PyTorchInferenceFlex SeriesFP16 FP32
SSD-MobileNet*PyTorchInferenceArc SeriesINT8 FP16 FP32
EfficientNetPyTorchInferenceFlex SeriesFP16 BF16 FP32
EfficientNetTensorFlowInferenceFlex SeriesFP16
FBNetPyTorchInferenceFlex SeriesFP16 BF16 FP32
Wide Deep Large DatasetTensorFlowInferenceFlex SeriesFP16
YOLO V5PyTorchInferenceFlex SeriesFP16
BERT largePyTorchInferenceMax Series, Arc SeriesBFloat16 FP32 FP16
BERT largePyTorchTrainingMax Series, Arc SeriesBFloat16 FP32 TF32
BERT largeTensorFlowTrainingMax SeriesBFloat16 TF32 FP32
DLRM v2PyTorchInferenceMax SeriesFP32 BF16
DLRM v2PyTorchTrainingMax SeriesFP32 TF32 BF16
3D-UnetPyTorchInferenceMax SeriesFP16 INT8 FP32
3D-UnetTensorFlowTrainingMax SeriesBFloat16 FP32
Stable DiffusionPyTorchInferenceFlex Series, Max Series, Arc SeriesFP16 FP32
Stable DiffusionTensorFlowInferenceFlex SeriesFP16 FP32
Mask R-CNNTensorFlowInferenceFlex SeriesFP32 Float16
Mask R-CNNTensorFlowTrainingMax SeriesFP32 BFloat16
Swin TransformerPyTorchInferenceFlex SeriesFP16
FastPitchPyTorchInferenceFlex SeriesFP16
UNet++PyTorchInferenceFlex SeriesFP16
RNN-TPyTorchInferenceMax SeriesFP16 BF16 FP32
RNN-TPyTorchTrainingMax SeriesFP32 BF16 TF32
IFRNetPyTorchInferenceFlex SeriesFP16
RIFEPyTorchInferenceFlex SeriesFP16

How to Contribute

If you would like to add a new benchmarking script, please use this guide.