Home

Awesome

OKDPH

This repository contains the code for CVPR2023 OKDPH: Generalization Matters: Loss Minima Flattening via Parameter Hybridization for Efficient Online Knowledge Distillation.

Data

3 datasets were used in the paper:

For downloaded data sets please place them in the 'dataset' folder.

dataset:

-- cifar-10-batches-py

-- cifar-100-python

Requirements

Run

cd src
bash OKDPH.sh

For the case of four students:

cd src
python OKDPH.py --omega 0.8 --beta 0.8 --gamma 0.5 --interval 1_epoch \
    --model_names resnet32 resnet32 resnet32 resnet32 \
    --transes hflip cutout augment auto_aug base \
    --log 21_cifar10_okdph_4stu_1ep

Please refer to the bash files for more running commands.

Baselines

cd src
bash baseline.sh

Experiment

Citation

If you find this work useful for your research, please cite our paper:

@inproceedings{zhang2023generalization,
  title={Generalization Matters: Loss Minima Flattening via Parameter Hybridization for Efficient Online Knowledge Distillation},
  author={Zhang, Tianli and Xue, Mengqi and Zhang, Jiangtao and Zhang, Haofei and Wang, Yu and Cheng, Lechao and Song, Jie and Song, Mingli},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={20176--20185},
  year={2023}
}

Contact

Please feel free to contact me via email (zhangtianli@zju.edu.cn) if you are interested in my research :)