Home

Awesome

Discrimination-aware Channel Pruning for Deep Neural Networks （NeurIPS 2018)

License

Architecture of Discrimination-aware Channel Pruning (DCP)

Architecture of DCP

Training Algorithm

<img height="300" src="./imgs/algorithm_1.png"/> <img height="300" src="./imgs/algorithm_2.png"/>

Recent Update

2019.05.10: We release a new version of dcp.

2018.11.27: We release the source code of dcp.

Requirements

Testing

  1. Download the pre-trained pruned model from the model zoo.

  2. Add DCP into PYTHONPATH.

# This is my path of DCP. You need to change to your path of DCP.
export PYTHONPATH=/home/liujing/Codes/Discrimination-aware-Channel-Pruning-for-Deep-Neural-Networks:$PYTHONPATH
  1. Set configuration for testing. You need to set save_path, data_path, dataset, pruning_rate, net_type, depth and the pretrained in dcp/test/test.hocon.
cd dcp/test/
vim test.hocon
  1. Run testing.
python main.py test.hocon

Channel Pruning Examples

  1. Download pre-trained mdoel.
  1. Add DCP into PYTHONPATH.
# This is my path of DCP. You need to change to your path of DCP.
export PYTHONPATH=/home/liujing/Codes/Discrimination-aware-Channel-Pruning-for-Deep-Neural-Networks:$PYTHONPATH
  1. Before channel pruning, you need to add discrimination-aware loss and fine tune the whole network. You need to set save_path, data_path, experiment_id and the pretrained in dcp/auxnet/cifar10_resnet.hocon.
cd dcp/auxnet/
vim cifar10_resnet.hocon
  1. Add discrimination-aware loss and conduct fine-tuning.
python main.py cifar10_resnet.hocon
  1. Set configuration for channel selection. You need to set save_path, data_path, pruning_rate and experiment_id in dcp/channel_selection/cifar10_resnet.hocon. Additionally, you need to set pretrained to the path of best_model_with_aux_fc.pth in check_point folder.
cd dcp/channel_selection/
vim cifar10_resnet.hocon
  1. Conduct channel selection.
python main.py cifar10_resnet.hocon
  1. Set configuration for fine-tuning. You need to set save_path, data_path, and experiment_id in dcp/finetune/cifar10_resnet.hocon. Additionally, you need to set pretrained to the path of model_xxx_cs_000.pth in check_point folder.
cd dcp/finetune/
vim cifar10_resnet.hocon
  1. Fine-tune the pruned model.
python main cifar10_resnet.hocon

Citation

If you find DCP useful in your research, please consider to cite the following related papers:

@incollection{NIPS2018_7367,
title = {Discrimination-aware Channel Pruning for Deep Neural Networks},
author = {Zhuang, Zhuangwei and Tan, Mingkui and Zhuang, Bohan and Liu, Jing and Guo, Yong and Wu, Qingyao and Huang, Junzhou and Zhu, Jinhui},
booktitle = {Advances in Neural Information Processing Systems 31},
editor = {S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett},
pages = {881--892},
year = {2018},
publisher = {Curran Associates, Inc.},
url = {http://papers.nips.cc/paper/7367-discrimination-aware-channel-pruning-for-deep-neural-networks.pdf}
}