Home

Awesome

CondenseNets

This repository contains the code (in PyTorch) for "CondenseNet: An Efficient DenseNet using Learned Group Convolutions" paper by Gao Huang*, Shichen Liu*, Laurens van der Maaten and Kilian Weinberger (* Authors contributed equally).

Citation

If you find our project useful in your research, please consider citing:

@inproceedings{huang2018condensenet,
  title={Condensenet: An efficient densenet using learned group convolutions},
  author={Huang, Gao and Liu, Shichen and Van der Maaten, Laurens and Weinberger, Kilian Q},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={2752--2761},
  year={2018}
}

Contents

  1. Introduction
  2. Usage
  3. Results
  4. Discussions
  5. Contacts

Introduction

CondenseNet is a novel, computationally efficient convolutional network architecture. It combines dense connectivity between layers with a mechanism to remove unused connections. The dense connectivity facilitates feature re-use in the network, whereas learned group convolutions remove connections between layers for which this feature re-use is superfluous. At test time, our model can be implemented using standard grouped convolutions —- allowing for efficient computation in practice. Our experiments demonstrate that CondenseNets are much more efficient than other compact convolutional networks such as MobileNets and ShuffleNets.

<img src="https://user-images.githubusercontent.com/9162722/32978657-b10fae0e-cc81-11e7-888d-1f9e4c028a9b.png">

Figure 1: Learned Group Convolution with G=C=3.

<img src="https://user-images.githubusercontent.com/9162722/31302319-6ca3a49c-ab33-11e7-938c-70379feca5bc.jpg" width="480">

Figure 2: CondenseNets with Fully Dense Connectivity and Increasing Growth Rate.

Usage

Dependencies

Train

As an example, use the following command to train a CondenseNet on ImageNet

python main.py --model condensenet -b 256 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0,1,2,3,4,5,6,7 --resume

As another example, use the following command to train a CondenseNet on CIFAR-10

python main.py --model condensenet -b 64 -j 12 cifar10 \
--stages 14-14-14 --growth 8-16-32 --gpu 0 --resume

Evaluation

We take the ImageNet model trained above as an example.

To evaluate the trained model, use evaluate to evaluate from the default checkpoint directory:

python main.py --model condensenet -b 64 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0 --resume \
--evaluate

or use evaluate-from to evaluate from an arbitrary path:

python main.py --model condensenet -b 64 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0 --resume \
--evaluate-from /PATH/TO/BEST/MODEL

Note that these models are still the large models. To convert the model to group-convolution version as described in the paper, use the convert-from function:

python main.py --model condensenet -b 64 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0 --resume \
--convert-from /PATH/TO/BEST/MODEL

Finally, to directly load from a converted model (that is, a CondenseNet), use a converted model file in combination with the evaluate-from option:

python main.py --model condensenet_converted -b 64 -j 20 /PATH/TO/IMAGENET \
--stages 4-6-8-10-8 --growth 8-16-32-64-128 --gpu 0 --resume \
--evaluate-from /PATH/TO/CONVERTED/MODEL

Other Options

We also include DenseNet implementation in this repository.
For more examples of usage, please refer to script.sh
For detailed options, please python main.py --help

Results

Results on ImageNet

ModelFLOPsParamsTop-1 Err.Top-5 Err.Pytorch Model
CondenseNet-74 (C=G=4)529M4.8M26.28.3Download (18.69M)
CondenseNet-74 (C=G=8)274M2.9M29.010.0Download (11.68M)

Results on CIFAR

ModelFLOPsParamsCIFAR-10CIFAR-100
CondenseNet-5028.6M0.22M6.22-
CondenseNet-7451.9M0.41M5.28-
CondenseNet-8665.8M0.52M5.0623.64
CondenseNet-9881.3M0.65M4.83-
CondenseNet-11098.2M0.79M4.63-
CondenseNet-122116.7M0.95M4.48-
CondenseNet-182*513M4.2M3.7618.47

(* trained 600 epochs)

Inference time on ARM platform

ModelFLOPsTop-1Time(s)
VGG-1615,300M28.5354
ResNet-181,818M30.28.14
1.0 MobileNet-224569M29.41.96
CondenseNet-74 (C=G=4)529M26.21.89
CondenseNet-74 (C=G=8)274M29.00.99

Contact

liushichen95@gmail.com
gh349@cornell.com

We are working on the implementation on other frameworks.
Any discussions or concerns are welcomed!