Awesome
SiMaN: Sign-to-Magnitude Network Binarization (Paper Link) 
.
Pytorch implementation of SiMaN (IEEE TPAMI 2022).
Its Conference of RBNN (NeurIPS 2020) is here (Paper, code)
Dependencies
- python 3.7
- pytorch 1.1.0
- torchvision 0.3.0
- numpy 1.17.2
Tips
We suggest reading our RBNN in NeurIPS'20 first (Paper, Code).
Any problem, please contact the first author (Email: lmbxmu@stu.xmu.edu.cn) or the third author (Email: ianhsu@stu.xmu.edu.cn).
Training on CIFAR-10
python -u main.py \
--gpus 0 \
--seed 123 \
--model resnet18_1w1a (or resnet20_1w1a or vgg_small_1w1a) \
--results_dir ./result \
--data_path [DATA_PATH] \
--dataset cifar10 \
--epochs 400 \
--lr 0.1 \
-b 256 \
-bt 128 \
--lr_type cos \
--warm_up \
--weight_decay 5e-4 \
Results on CIFAR-10.
| Quantized model Link | batch_size | batch_size_test | epochs | Top-1 |
|---|---|---|---|---|
| resnet18_1w1a | 256 | 128 | 400 | 92.5 |
| resnet20_1w1a | 256 | 128 | 400 | 87.4 |
| vgg_small_1w1a | 256 | 128 | 400 | 92.5 |
To ensure the reproducibility, please refer to our training details provided in the links for our quantized models.
To verify the performance of our quantized models on CIFAR-10, please use the following command:
python -u main.py \
--gpus 0 \
-e [best_model_path] \
--model resnet20_1w1a (resnet18_1w1a or vgg_small_1w1a) \
--data_path [DATA_PATH] \
--dataset cifar10 \
-bt 128 \
Training on ImageNet
python -u main.py \
--gpus 0,1 \
--model resnet18_1w1a (or resnet34_1w1a) \
--results_dir ./result \
--data_path [DATA_PATH] \
--dataset imagenet \
--epochs 150 \
--lr 0.1 \
-b 512 \
-bt 256 \
--lr_type cos \
--use_dali \
We provide two types of dataloaders by nvidia-dali and Pytorch respectively. They use the same data augmentations, including random crop and horizontal flip. We empirically find that the dataloader by Pytorch can offer a better accuracy performance. They may have different code implementations. Anyway, we haven't figured it out yet. However, nvidia-dali shows its extreme efficiency in processing data which well accelerates the network training. The reported experimental results are on the basis of nvidia-dali. If interested, you can try dataloader by Pytorch via removing the optional argument --use_dali to obtain a better performance.
Nvidia-dali package
# for CUDA 10
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist nvidia-dali-cuda100
# for CUDA 11
pip install --extra-index-url https://developer.download.nvidia.com/compute/redist nvidia-dali-cuda110
Results on ImageNet
| Quantized model Link | batch_size | batch_size_test | epochs | use_dali | Top-1 | Top-5 |
|---|---|---|---|---|---|---|
| resnet18_1w1a | 512 | 256 | 150 | ✔ | 60.1 | 82.3 |
| resnet34_1w1a | 512 | 256 | 150 | ✔ | 63.9 | 84.8 |
To ensure the reproducibility, please refer to our training details provided in the links for our quantized models.
Small tips for further boosting the performance of our method: (1) removing the optional argument --use_dali as discussed above; (2) increasing the training epochs; (3) enlarging the batch size for training.
To verify the performance of our quantized models on ImageNet, please use the following command:
python -u main.py \
--gpu 0 \
-e [best_model_path] \
--model resnet18_1w1a (or resnet34_1w1a)\
--dataset imagenet \
--data_path [DATA_PATH] \
-bt 256 \
--use dali