Awesome
ReActNet
This is the pytorch implementation of our paper "ReActNet: Towards Precise Binary NeuralNetwork with Generalized Activation Functions", published in ECCV 2020.
<div align=center> <img width=60% src="https://github.com/liuzechun0216/images/blob/master/reactnet_github.jpg"/> </div>In this paper, we propose to generalize the traditional Sign and PReLU functions to RSign and RPReLU, which enable explicit learning of the distribution reshape and shift at near-zero extra cost. By adding simple learnable bias, ReActNet achieves 69.4% top-1 accuracy on Imagenet dataset with both weights and activations being binary, a near ResNet-level accuracy.
Citation
If you find our code useful for your research, please consider citing:
@inproceedings{liu2020reactnet,
title={ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions},
author={Liu, Zechun and Shen, Zhiqiang and Savvides, Marios and Cheng, Kwang-Ting},
booktitle={European Conference on Computer Vision (ECCV)},
year={2020}
}
Run
1. Requirements:
- python3, pytorch 1.4.0, torchvision 0.5.0
2. Data:
- Download ImageNet dataset
3. Steps to run:
(1) Step1: binarizing activations
- Change directory to
./resnet/1_step1/or./mobilenet/1_step1/ - run
bash run.sh
(2) Step2: binarizing weights + activations
- Change directory to
./resnet/2_step2/or./mobilenet/2_step2/ - run
bash run.sh
Models
| Methods | Top1-Acc | FLOPs | Trained Model |
|---|---|---|---|
| XNOR-Net | 51.2% | 1.67 x 10^8 | - |
| Bi-Real Net | 56.4% | 1.63 x 10^8 | - |
| Real-to-Binary | 65.4% | 1.83 x 10^8 | - |
| ReActNet (Bi-Real based) | 65.9% | 1.63 x 10^8 | Model-ReAct-ResNet |
| ReActNet-A | 69.5% | 0.87 x 10^8 | Model-ReAct-MobileNet |
Contact
Zechun Liu, HKUST (zliubq at connect.ust.hk)
Zhiqiang Shen, CMU (zhiqians at andrew.cmu.edu)