Home

Awesome

PyCIL: A Python Toolbox for Class-Incremental Learning

<p align="center"> <a href="#Introduction">Introduction</a> • <a href="#Methods-Reproduced">Methods Reproduced</a> • <a href="#Reproduced-Results">Reproduced Results</a> • <a href="#how-to-use">How To Use</a> • <a href="#license">License</a> • <a href="#Acknowledgments">Acknowledgments</a> • <a href="#Contact">Contact</a> </p> <div align="center"> <img src="./resources/logo_v2.png" width="800px"> </div>
<div align="center">

LICENSEPython PyTorch method CIL visitors

</div>

Welcome to PyCIL, perhaps the toolbox for class-incremental learning with the most implemented methods. This is the code repository for "PyCIL: A Python Toolbox for Class-Incremental Learning" [paper] in PyTorch. If you use any content of this repo for your work, please cite the following bib entries:

@article{zhou2023pycil,
    author = {Da-Wei Zhou and Fu-Yun Wang and Han-Jia Ye and De-Chuan Zhan},
    title = {PyCIL: a Python toolbox for class-incremental learning},
    journal = {SCIENCE CHINA Information Sciences},
    year = {2023},
    volume = {66},
    number = {9},
    pages = {197101},
    doi = {https://doi.org/10.1007/s11432-022-3600-y}
  }

@article{zhou2024class,
    author = {Zhou, Da-Wei and Wang, Qi-Wei and Qi, Zhi-Hong and Ye, Han-Jia and Zhan, De-Chuan and Liu, Ziwei},
    title = {Class-Incremental Learning: A Survey},
    journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
    volume={46},
    number={12},
    pages={9851--9873},
    year = {2024}
}

@inproceedings{zhou2024continual,
    title={Continual learning with pre-trained models: A survey},
    author={Zhou, Da-Wei and Sun, Hai-Long and Ning, Jingyi and Ye, Han-Jia and Zhan, De-Chuan},
    booktitle={IJCAI},
    pages={8363-8371},
    year={2024}
}

What's New

Introduction

Traditional machine learning systems are deployed under the closed-world setting, which requires the entire training data before the offline training process. However, real-world applications often face the incoming new classes, and a model should incorporate them continually. The learning paradigm is called Class-Incremental Learning (CIL). We propose a Python toolbox that implements several key algorithms for class-incremental learning to ease the burden of researchers in the machine learning community. The toolbox contains implementations of a number of founding works of CIL, such as EWC and iCaRL, but also provides current state-of-the-art algorithms that can be used for conducting novel fundamental research. This toolbox, named PyCIL for Python Class-Incremental Learning, is open source with an MIT license.

For more information about incremental learning, you can refer to these reading materials:

Methods Reproduced

Reproduced Results

CIFAR-100

<div align="center"> <img src="./resources/cifar100.png" width="900px"> </div>

ImageNet-100

<div align="center"> <img src="./resources/ImageNet100.png" width="900px"> </div>

ImageNet-100 (Top-5 Accuracy)

<div align="center"> <img src="./resources/imagenet20st5.png" width="500px"> </div>

More experimental details and results can be found in our survey.

How To Use

Clone

Clone this GitHub repository:

git clone https://github.com/G-U-N/PyCIL.git
cd PyCIL

Dependencies

  1. torch 1.81
  2. torchvision 0.6.0
  3. tqdm
  4. numpy
  5. scipy
  6. quadprog
  7. POT

Run experiment

  1. Edit the [MODEL NAME].json file for global settings.
  2. Edit the hyperparameters in the corresponding [MODEL NAME].py file (e.g., models/icarl.py).
  3. Run:
python main.py --config=./exps/[MODEL NAME].json

where [MODEL NAME] should be chosen from finetune, ewc, lwf, replay, gem, icarl, bic, wa, podnet, der, etc.

  1. hyper-parameters

When using PyCIL, you can edit the global parameters and algorithm-specific hyper-parameter in the corresponding json file.

These parameters include:

Other parameters in terms of model optimization, e.g., batch size, optimization epoch, learning rate, learning rate decay, weight decay, milestone, and temperature, can be modified in the corresponding Python file.

Datasets

We have implemented the pre-processing of CIFAR100, imagenet100, and imagenet1000. When training on CIFAR100, this framework will automatically download it. When training on imagenet100/1000, you should specify the folder of your dataset in utils/data.py.

    def download_data(self):
        assert 0,"You should specify the folder of your dataset"
        train_dir = '[DATA-PATH]/train/'
        test_dir = '[DATA-PATH]/val/'

Here is the file list of ImageNet100 (or say ImageNet-Sub).

Awesome Papers using PyCIL

Our Papers

Other Awesome Works

License

Please check the MIT license that is listed in this repository.

Acknowledgments

We thank the following repos providing helpful components/functions in our work.

The training flow and data configurations are based on Continual-Learning-Reproduce. The original information of the repo is available in the base branch.

Contact

If there are any questions, please feel free to propose new features by opening an issue or contact with the author: Da-Wei Zhou(zhoudw@lamda.nju.edu.cn) and Fu-Yun Wang(wangfuyun@smail.nju.edu.cn). Enjoy the code.

Star History 🚀

Star History Chart