Home

Awesome

Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples [NeurIPS 2021]

This repository is the official implementation of paper [Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples]. [Paper] [Slide]

Overview of Qimera

Requirements

To install requirements:

pip install -r requirements.txt

Training

For Imagenet training, change the path of the validation set in .hocon file. To train the model described in the paper, run one of this command:

./run_cifar10_4bit.sh
./run_cifar100_4bit.sh
./run_imgnet_resnet18_4bit.sh
./run_imgnet_resnet50_4bit.sh
./run_imgnet_mobilenet_v2_4bit.sh

Refer other_train_scripts folder for 5bit settings.

Evaluation

To evaluate trained model, run the command below after training:

./eval_cifar10_4bit.sh
./eval_cifar100_4bit.sh
./eval_imgnet_resnet18_4bit.sh
./eval_imgnet_resnet50_4bit.sh
./eval_imgnet_mobilenet_v2_4bit.sh

Visualizing Feature Space

Feature space visualization of real or synthetic images described in Figure 3.

python experiments.py --pca_source
python experiments.py --gdfq_generator_path GENERATOR_WEIGHT_PATH --pca_gdfq --image_gdfq
python experiments.py --qimera_generator_path GENERATOR_WEIGHT_PATH --pca_qimera --pca_mix --pca_path --image_gdfq --image_mix

Results

Our model achieves the following performance on :

DatasetModelTeacher Net Accuracy4bit Quantized Model Accuracy5bit Quantized Model Accuracy
Cifar-10ResNet-2093.89%91.26%93.46%
Cifar-100ResNet-2070.33%65.10%69.02%
ImageNetResNet-1871.47%63.84%69.29%
ImageNetResNet-5077.73%66.25%75.32%
ImageNetMobileNetV273.03%61.62%70.45%

Generated Synthetic Images for Cifar10 :
</br> Cifar10 Synthetic Images Generated By Qimera

License

This project is licensed under the terms of the GNU General Public License v3.0

Citation

@inproceedings{choi2021qimera,
  title={Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples},
  author={Choi, Kanghyun and Hong, Deokki and Park, Noseong and Kim, Youngsok and Lee, Jinho},
  booktitle={Thirty-Fifth Conference on Neural Information Processing Systems},
  year={2021}
}