Home

Awesome

Affine Parameter Sharing for Channel Search

This repository contains the implementation of our paper "Revisiting Parameter Sharing for Automatic Neural Channel Number Search" in NeurIPS 2020. We propose affine parameter sharing as a general framework to quantitatively analyze and better utilize parameter sharing for channel number search problems. An overall illustration is presented below: model

Dependencies

Our implementation is built on Python 3.6 with Pytorch. Install packages:

pip install -r requirements.txt

Execution

1: Prepare datasets

Our experiments are based on CIFAR-10 and ImageNet-2012. Please first download the datasets yourself. The default path to dataset is set to --data_path TODO in ./main.py. Please change this in the follwowing execution script correspondingly.

2: Searching

The searching scripts are in ./scripts/search.

3: Training from scratch

The scripts are in ./scripts/train/.

More descriptions of script args can be found in ./main.py.

Model Checkpoints & Training Logs

We also provide model checkpoints and training logs on CIFAR-10 and ImageNet datasets here. Note that the repository is minorly refacted from the original implementation in the paper, and the above procedures may not gaurantee to reproduce the same results due to searching randomness. However, the searched models are roughly close in general with similar accuracies.

Citation

If you find this repo helpful for your research, please:

@inproceedings{wang2020revisiting,
  title={Revisiting Parameter Sharing for Automatic Neural Channel Number Search},
  author={Wang, Jiaxing and Bai, Haoli and Wu, Jiaxiang and Shi, Xupeng and Huang, Junzhou and King, Irwin and Lyu, Michael and Cheng, Jian},
  booktitle={Advances in Neural Information Processing Systems},
  volume={33},
  year={2020}
}