Home

Awesome

CycleMLP: A MLP-like Architecture for Dense Prediction (ICLR 2022 Oral)

License: MIT Python 3.8

<p align="middle"> <img src="figures/teaser.png" height="300" /> &nbsp;&nbsp;&nbsp;&nbsp; <img src="figures/flops.png" height="300" /> </p>

This is a PyTorch implementation of the paper CycleMLP: A MLP-like Architecture for Dense Prediction.

Updates

Model Zoo

We provide CycleMLP models pretrained on ImageNet 2012.

ModelParametersFLOPsTop 1 Acc.Download
CycleMLP-B115M2.1G78.9%model
CycleMLP-B227M3.9G81.6%model
CycleMLP-B338M6.9G82.4%model
CycleMLP-B452M10.1G83.0%model
CycleMLP-B576M12.3G83.2%model
CycleMLP-T28M4.4G81.3%model
CycleMLP-S50M8.5G82.9%model
CycleMLP-B88M15.2G83.4%model

Usage

Install

pip install 'git+https://github.com/rwightman/pytorch-image-models@c2ba229d995c33aaaf20e00a5686b4dc857044be'

or

git clone https://github.com/rwightman/pytorch-image-models
cd pytorch-image-models
git checkout c2ba229d995c33aaaf20e00a5686b4dc857044be
pip install -e .

Data preparation

Download and extract ImageNet train and val images from http://image-net.org/. The directory structure is:

│path/to/imagenet/
├──train/
│  ├── n01440764
│  │   ├── n01440764_10026.JPEG
│  │   ├── n01440764_10027.JPEG
│  │   ├── ......
│  ├── ......
├──val/
│  ├── n01440764
│  │   ├── ILSVRC2012_val_00000293.JPEG
│  │   ├── ILSVRC2012_val_00002138.JPEG
│  │   ├── ......
│  ├── ......

Evaluation

To evaluate a pre-trained CycleMLP-B5 on ImageNet val with a single GPU run:

python main.py --eval --model CycleMLP_B5 --resume path/to/CycleMLP_B5.pth --data-path /path/to/imagenet

Training

To train CycleMLP-B5 on ImageNet on a single node with 8 gpus for 300 epochs run:

python -m torch.distributed.launch --nproc_per_node=8 --use_env main.py --model CycleMLP_B5 --batch-size 128 --data-path /path/to/imagenet --output_dir /path/to/save

Acknowledgement

This code is based on DeiT and pytorch-image-models. Thanks for their wonderful works

Citing

@inproceedings{
chen2022cyclemlp,
title={Cycle{MLP}: A {MLP}-like Architecture for Dense Prediction},
author={Shoufa Chen and Enze Xie and Chongjian GE and Runjian Chen and Ding Liang and Ping Luo},
booktitle={International Conference on Learning Representations},
year={2022},
url={https://openreview.net/forum?id=NMEceG4v69Y}
}

License

CycleMLP is released under MIT License.