Awesome

• <span style="color:red">ADer</span> is an open source visual Anomaly Detection toolbox based on PyTorch, which supports multiple popular AD datasets and approaches. </br>
• We reproduce popular AD methods under the Multi-class Unsupervised Anomaly Detection (<span style="color:red">MUAD</span>) by default. </br>
• We hope it can bring convenience to your research and application. </br>

🐉 News

• 🔥 We have released several reproduced models, configuration files, and training logs in our <span style="color:red">benchmark</span> paper 🐲 Paper | Results & CFGs
• 🔥 <span style="color:red">COCO-AD</span> and powerful <span style="color:red">InvAD</span> is released 🐲 Paper | Project | Code
• 🔥 Plain ViT based <span style="color:red">ViTAD</span> is released 🐲 Paper | Project | Code
• 🔥 <span style="color:red">Real-IAD</span> is released: a new large-scale challenging industrial AD dataset 🐲 Paper | Project | Code

💡 Property

• 🚀Support Visualization
• 🚀 Multi-/Single-class Training and Testing
• 🚀 Convenient and flexible way to implement a new approach, refer to here.
• Reproduced popular methods in ADer Benchmark:
  • 🚀Augmentation-based
    • DRAEM, ICCV'21
    • SimpleNet, CVPR'23
    • RealNet, CVPR'24
  • 🚀Embedding-based
    • CFA, Access'22
    • PatchCore, CVPR'22
    • CFLOW-AD, WACV'22
    • PyramidalFlow, CVPR'23
  • 🚀Reconstruction-based
    • ViTAD, arXiv'23
    • InvAD, arXiv'24
    • InvAD-lite, arXiv'24
    • DiAD, AAAI'24: See tripartite implementation in this website
    • MambaAD, arXiv'24
    • RD, CVPR'22
  • 🚀Hybrid
    • UniAD, NeurIPS'22: download in TIMM to model/pretrain
    • RD++, CVPR'23
    • DesTSeg, CVPR'23
• 🚀 (Opt.) DDP Training
• By default, the weights used by different methods are automatically downloaded (model.kwargs.pretrained=True, model.kwargs.checkpoint_path=''). If you prefer to specify offline weights, you can download the model weights to model/pretrain and modify the settings to model.kwargs.pretrained=False, model.kwargs.checkpoint_path='model/pretrain/xxx.pth'.

🛠️ Getting Started

Installation

• Clone this repo:

  git clone https://github.com/zhangzjn/ader.git && cd ader
  

• Prepare general experimental environment

  pip3 install timm==0.8.15dev0 mmselfsup pandas transformers openpyxl imgaug numba numpy tensorboard fvcore accimage Ninja
  pip3 install mmdet==2.25.3
  pip3 install --upgrade protobuf==3.20.1 scikit-image faiss-gpu
  pip3 install adeval
  pip3 install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu118
  pip3 install fastprogress geomloss FrEIA mamba_ssm adeval fvcore==0.1.5.post20221221
  (or) conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=11.8 -c pytorch -c nvidia
  

Dataset Preparation

Please refer to Datasets Description for preparing visual AD datasets as needed.

• Real-IAD: A new large-scale challenging industrial AD dataset, containing 30 classes with totally 151,050 images; 2,000 ∼ 5,000 resolution; 0.01% ~ 6.75% defect proportions; 1:1 ~ 1:10 defect ratio.
• COCO-AD: Large-scale and general-purpose challenging AD-adapted dataset.
• [MVTec AD](data/README.md/###MVTec AD): Most popular AD dataset.
• VisA: Popular AD dataset.
• Uni-Medical: Unified medical AD dataset.
• (Opt.) [MVTec 3D-AD](data/README.md/###MVTec 3D-AD): Improved 3D version of MVTec AD.
• (Opt.) Cifar10 & Cifar100: For one-class-train, one-class-test, and unified settings.
• (Opt.) Tiny-ImageNet: A larger one-class dataset.

Train (Multi-class Unsupervised AD setting by default, MUAD)

• Check data and model settings for the config file configs/METHOD/METHOD_CFG.py
• Train with single GPU example: CUDA_VISIBLE_DEVICES=0 python run.py -c configs/METHOD/METHOD_cfg.py -m train
• Train with multiple GPUs (DDP) in one node:
  • export nproc_per_node=8
  • export nnodes=1
  • export node_rank=0
  • export master_addr=YOUR_MACHINE_ADDRESS
  • export master_port=12315
  • python -m torch.distributed.launch --nproc_per_node=$nproc_per_node --nnodes=$nnodes --node_rank=$node_rank --master_addr=$master_addr --master_port=$master_port --use_env run.py -c configs/METHOD/METHOD_CFG.py -m train.
• Modify trainer.resume_dir to resume training.
• Single-class Unsupervised AD (SUAD, not recommend currently)
  1. one GPU-0 for training one grid class in mvtec dataset: CUDA_VISIBLE_DEVICES=0 python3 run.py -c configs/vitad/single_cls/vitad_mvtec_bs16.py -m train data.cls_names=grid trainer.checkpoint=runs/vitad/single_class/vitad_mvtec_bs16/grid
  2. one GPU-0 for training all classes serially in mvtec dataset: python3 runs_single_class.py -d mvtec -c configs/vitad/single_cls/vitad_mvtec_bs16.py -n 1 -m -1 -g 0
  3. $GPU_NUM GPUs for training all classes parallelly in mvtec dataset:: python3 runs_single_class.py -d mvtec -c configs/vitad/single_cls/vitad_mvtec_bs16.py -n $GPU_NUM -m 1
  4. results will be saved in default dir: runs/vitad/single_cls/vitad_mvtec_bs16

Test

• Modify trainer.resume_dir or model.kwargs['checkpoint_path']
• Test with single GPU example: CUDA_VISIBLE_DEVICES=0 python run.py -c configs/METHOD/METHOD_cfg.py -m test
• Test with multiple GPUs (DDP) in one node: python -m torch.distributed.launch --nproc_per_node=$nproc_per_node --nnodes=$nnodes --node_rank=$node_rank --master_addr=$master_addr --master_port=$master_port --use_env run.py -c configs/METHOD/METHOD_CFG.py -m test.

Visualization

• Modify trainer.resume_dir or model.kwargs['checkpoint_path']
• Visualize with single GPU example: CUDA_VISIBLE_DEVICES=0 python run.py -c configs/METHOD/METHOD_cfg.py -m test vis=True vis_dir=VISUALIZATION_DIR

How to Build a Custom Approach

1. Add a model config cfg_model_MODEL_NAME to configs/__base__
2. Add configs to configs/MODEL_NAME/CFG.py for training and testing.
3. Add a model implementation file model/MODEL_NAME.py
4. Add a trainer implementation file trainer/MODEL_NAME_trainer.py
5. (Optional) Add specific files to data, loss, optim, etc.

📜 MUAD Results on Popular AD Datasets

• <span style="color:red">Red metrics</span> are recommended for comprehensive evaluations.<br> Subscripts I, R, and P represent image-level, region-level, and pixel-level, respectively.
• The following results are derived from the original paper. Since the benchmark re-runs all experiments on the same L40S platform, slight differences in the results are reasonable.

MVTec AD

VisA

COCO-AD

MVTec 3D-AD (RGB)

Uni-Medical

Real-IAD

Citation

If you use this toolbox or benchmark in your research, please cite our related works.

@article{ader,
  title={ADer: A Comprehensive Benchmark for Multi-class Visual Anomaly Detection},
  author={Jiangning Zhang and Haoyang He and Zhenye Gan and Qingdong He and Yuxuan Cai and Zhucun Xue and Yabiao Wang and Chengjie Wang and Lei Xie and Yong Liu},
  journal={arXiv preprint arXiv:2406.03262},
  year={2024}
}

@inproceedings{realiad,
  title={Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection},
  author={Wang, Chengjie and Zhu, Wenbing and Gao, Bin-Bin and Gan, Zhenye and Zhang, Jianning and Gu, Zhihao and Qian, Shuguang and Chen, Mingang and Ma, Lizhuang},
  booktitle={CVPR},
  year={2024}
}

@article{vitad,
  title={Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detection},
  author={Zhang, Jiangning and Chen, Xuhai and Wang, Yabiao and Wang, Chengjie and Liu, Yong and Li, Xiangtai and Yang, Ming-Hsuan and Tao, Dacheng},
  journal={arXiv preprint arXiv:2312.07495},
  year={2023}
}

@article{invad,
  title={Learning Feature Inversion for Multi-class Anomaly Detection under General-purpose COCO-AD Benchmark},
  author={Jiangning Zhang and Chengjie Wang and Xiangtai Li and Guanzhong Tian and Zhucun Xue and Yong Liu and Guansong Pang and Dacheng Tao},
  journal={arXiv preprint arXiv:2404.10760},
  year={2024}
}

@article{mambaad,
  title={MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection},
  author={He, Haoyang and Bai, Yuhu and Zhang, Jiangning and He, Qingdong and Chen, Hongxu and Gan, Zhenye and Wang, Chengjie and Li, Xiangtai and Tian, Guanzhong and Xie, Lei},
  year={2024}
}