Awesome

Switchable Whitening (SW)

Paper

Xingang Pan, Xiaohang Zhan, Jianping Shi, Xiaoou Tang, Ping Luo. "Switchable Whitening for Deep Representation Learning", ICCV2019.

Introduction

Switchable Whitening unifies various whitening and standardization techniques in a general form, and adaptively learns their importance ratios for different tasks.
This repo is for ImageNet classification. We also provide the code for Syncronized SW at models/ops/sync_switchwhiten.py, which could be used for detection and segmentation.

Requirements

python>=3.6
pytorch>=1.0.1
others
```
pip install -r requirements.txt
```

Results

Top1/Top5 error on the ImageNet validation set are reported. The pretrained models with SW are available at google drive or BaiduYun(password: xkdi).

Model	BN	SN	BW	SW (BW+IW)
ResNet-50	23.58/7.00	23.10/6.55	23.31/6.72	22.07/6.04
ResNet-101	22.48/6.23	22.01/5.91	22.10/5.98	20.87/5.54
DenseNet-121	24.96/7.85	24.38/7.26	24.56/7.55	23.56/6.85
DenseNet-169	24.02/7.06	23.16/6.55	23.24/6.65	22.48/6.29

Before Start

Clone the repository

git clone https://github.com/XingangPan/Switchable-Whitening.git

Download ImageNet dataset. You may follow the instruction at fb.resnet.torch to process the validation set.

Training

Train with nn.DataParallel

sh experiments/resnet50_sw/run.sh  # remember to modify --data to your ImageNet path

Distributed training based on slurm

sh experiments/resnet50_sw/run_slurm.sh ${PARTITION}

Practical concerns

Inspired by IterativeNorm, SW is accelarated via Newton's iteration.
For SW, 4x64 (GPU number x batchsize) performs slightly better than 8x32.

Citing SW

@inproceedings{pan2018switchable,
  author = {Pan, Xingang and Zhan, Xiaohang and Shi, Jianping and Tang, Xiaoou and Luo, Ping},
  title = {Switchable Whitening for Deep Representation Learning},
  booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
  month = {October},
  year = {2019}
}