Awesome
DenseNet_lite
This implements the DenseNet architecture introduced in Densely Connected Convolutional Network.The original Torch implementation can be found at https://github.com/liuzhuang13/DenseNet, and please find more details about DenseNet there. The only difference here is that we write a customed container "DenseLayer.lua" to implement the dense connections in a more memory efficient way. This leads to ~25% reduction in memory consumption during training, while keeps the accuracy and training time the same.
Usage
-
Install Torch ResNet (https://github.com/facebook/fb.resnet.torch) following the instructions there. To reduce memory consumption, we recommend to install the optnet package.
-
Add the files
densenet_lite.lua
andDenseLayer.lua
to the folder models/; -
Insert
require 'models/DenseLayer
at Line.89 ofmodels/init.lua
, if you need to use multiple GPUs; -
Change the learning rate schedule at function learningRate() in
train.lua
(line 171/173), fromdecay = epoch >= 122 and 2 or epoch >= 81 and 1 or 0
todecay = epoch >= 225 and 2 or epoch >= 150 and 1 or 0
-
Train a DenseNet (L=40, k=12) on CIFAR-10+ using
th main.lua -netType densenet_lite -depth 40 -dataset cifar10 -batchSize 64 -nEpochs 300 -optnet true