Awesome

Quantized Convolutional networks using PyTorch

See https://github.com/eladhoffer/convNet.pytorch for updated version of this code

Code to replicate results in Scalable Methods for 8-bit Training of Neural Networks

e.g: running an 8-bit quantized resnet18 from the paper on ImageNet

python main.py --model resnet_quantized --model_config "{'depth': 18}" --save quantized_resnet18 --dataset imagenet --b 128

Dependencies

• pytorch
• torchvision to load the datasets, perform image transforms
• pandas for logging to csv
• bokeh for training visualization

Data

• Configure your dataset path at data.py.
• To get the ILSVRC data, you should register on their site for access: http://www.image-net.org/

Model configuration

Network model is defined by writing a <modelname>.py file in <code>models</code> folder, and selecting it using the <code>model</code> flag. Model function must be registered in <code>models/__init__.py</code> The model function must return a trainable network. It can also specify additional training options such optimization regime (either a dictionary or a function), and input transform modifications.

e.g for a model definition:

class Model(nn.Module):

    def __init__(self, num_classes=1000):
        super(Model, self).__init__()
        self.model = nn.Sequential(...)

        self.regime = [
            {'epoch': 0, 'optimizer': 'SGD', 'lr': 1e-2,
                'weight_decay': 5e-4, 'momentum': 0.9},
            {'epoch': 15, 'lr': 1e-3, 'weight_decay': 0}
        ]

        self.input_transform = {
            'train': transforms.Compose([...]),
            'eval': transforms.Compose([...])
        }
    def forward(self, inputs):
        return self.model(inputs)
        
 def model(**kwargs):
        return Model()