Home

Awesome

Awesome CIFAR Zoo

<img src="https://user-images.githubusercontent.com/7837172/44953557-0fb54e80-aec9-11e8-9d38-2388bc70c5c5.png" width=15% align="right" />

Status: Archive ( Final test with PyTorch 1.7 and no longer maintained, I would recommend you to use pycls powered by FAIR, which is a simple and flexible codebase for image classification )

This repository contains the pytorch code for multiple CNN architectures and improve methods based on the following papers, hope the implementation and results will helpful for your research!!

Requirements and Usage

Requirements

pip install -r requirements.txt

Usage

simply run the cmd for the training:

## 1 GPU for lenet
CUDA_VISIBLE_DEVICES=0 python -u train.py --work-path ./experiments/cifar10/lenet

## resume from ckpt
CUDA_VISIBLE_DEVICES=0 python -u train.py --work-path ./experiments/cifar10/lenet --resume

## 2 GPUs for resnet1202
CUDA_VISIBLE_DEVICES=0,1 python -u train.py --work-path ./experiments/cifar10/preresnet1202

## 4 GPUs for densenet190bc
CUDA_VISIBLE_DEVICES=0,1,2,3 python -u train.py --work-path ./experiments/cifar10/densenet190bc

## 1 GPU for vgg19 inference
CUDA_VISIBLE_DEVICES=0 python -u eval.py --work-path ./experiments/cifar10/vgg19

We use yaml file config.yaml to save the parameters, check any files in ./experimets for more details.
You can see the training curve via tensorboard, tensorboard --logdir path-to-event --port your-port.
The training log will be dumped via logging, check log.txt in your work path.

Results on CIFAR

Vanilla architectures

architectureparamsbatch sizeepochC10 test acc (%)C100 test acc (%)
Lecun62K12825067.4634.10
alexnet2.4M12825075.5638.67
vgg1920M12825093.0072.07
preresnet200.27M12825091.8867.03
preresnet1101.7M12825094.2472.96
preresnet120219.4M12825094.7475.28
densenet100bc0.76M6430095.0877.55
densenet190bc25.6M6430096.1182.59
resnext29_16x64d68.1M12830095.9483.18
se_resnext29_16x64d68.6M12830096.1583.65
cbam_resnext29_16x64d68.7M12830096.2783.62
ge_resnext29_16x64d70.0M12830096.2183.57

With additional regularization

PS: the default data augmentation methods are RandomCrop + RandomHorizontalFlip + Normalize,
and the means which additional method be used. :cake:

architectureepochcutoutmixupC10 test acc (%)
preresnet2025091.88
preresnet2025092.57
preresnet2025092.71
preresnet2025092.66
preresnet11025094.24
preresnet11025094.67
preresnet11025094.94
preresnet11025095.66
se_resnext29_16x64d30096.15
se_resnext29_16x64d30096.60
se_resnext29_16x64d30096.86
se_resnext29_16x64d30097.03
cbam_resnext29_16x64d30097.16
ge_resnext29_16x64d30097.19
----------
shake_resnet26_2x64d180096.94
shake_resnet26_2x64d180097.20
shake_resnet26_2x64d180097.42
shake_resnet26_2x64d180097.71

PS: shake_resnet26_2x64d achieved 97.71% test accuracy with cutout and mixup!!
It's cool, right?

With different LR scheduler

architectureepochstep decaycosinehtd(-6,3)cutoutmixupC10 test acc (%)
preresnet2025091.88
preresnet2025092.13
preresnet2025092.44
preresnet2025093.30
preresnet11025094.24
preresnet11025094.48
preresnet11025094.82
preresnet11025095.88

Acknowledgments

Provided codes were adapted from

Feel free to contact me if you have any suggestions or questions, issues are welcome,
create a PR if you find any bugs or you want to contribute. :blush:

Citation

@misc{bigballon2019cifarzoo,
  author = {Wei Li},
  title = {CIFAR-ZOO: PyTorch implementation of CNNs for CIFAR dataset},
  howpublished = {\url{https://github.com/BIGBALLON/CIFAR-ZOO}},
  year = {2019}
}