Home

Awesome

<p align="center"> <img src="./resources/logo.png" width="800px"> </p> <p align="center"> <a href="#-introduction">🎉Introduction</a> • <a href="#-methods-reproduced">🌟Methods Reproduced</a> • <a href="#-reproduced-results">📝Reproduced Results</a> <br /> <a href="#%EF%B8%8F-how-to-use">☄️How to Use</a> • <a href="#-acknowledgments">👨‍🏫Acknowledgments</a> • <a href="#-contact">🤗Contact</a> </p>
<p align="center"> <a href=""><img src="https://img.shields.io/badge/PILOT-v1.0-darkcyan"></a> <a href='https://arxiv.org/abs/2309.07117'><img src='https://img.shields.io/badge/Arxiv-2309.07117-b31b1b.svg?logo=arXiv'></a> <a href=""><img src="https://img.shields.io/github/stars/sun-hailong/LAMDA-PILOT?color=4fb5ee"></a> <a href=""><img src="https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2Fsun-hailong%2FLAMDA-PILOT&count_bg=%23FFA500&title_bg=%23555555&icon=&icon_color=%23E7E7E7&title=visitors&edge_flat=false"></a> <a href=""><img src="https://black.readthedocs.io/en/stable/_static/license.svg"></a> <a href=""><img src="https://img.shields.io/github/last-commit/sun-hailong/LAMDA-PILOT?color=blue"></a> </p>

🎉 Introduction

Welcome to PILOT, a pre-trained model-based continual learning toolbox <a href="https://arxiv.org/abs/2309.07117">[Paper]</a>. On the one hand, PILOT implements some state-of-the-art class-incremental learning algorithms based on pre-trained models, such as L2P, DualPrompt, and CODA-Prompt. On the other hand, PILOT also fits typical class-incremental learning algorithms (e.g., FOSTER, and MEMO) within the context of pre-trained models to evaluate their effectiveness.

If you use any content of this repo for your work, please cite the following bib entries:

@article{sun2023pilot,
  title={PILOT: A Pre-Trained Model-Based Continual Learning Toolbox},
  author={Sun, Hai-Long and Zhou, Da-Wei and Ye, Han-Jia and Zhan, De-Chuan},
  journal={arXiv preprint arXiv:2309.07117},
  year={2023}
}

@inproceedings{zhou2024continual,
    title={Continual learning with pre-trained models: A survey},
    author={Zhou, Da-Wei and Sun, Hai-Long and Ning, Jingyi and Ye, Han-Jia and Zhan, De-Chuan},
    booktitle={IJCAI},
    pages={8363-8371},
    year={2024}
}

@article{zhou2024class,
    author = {Zhou, Da-Wei and Wang, Qi-Wei and Qi, Zhi-Hong and Ye, Han-Jia and Zhan, De-Chuan and Liu, Ziwei},
    title = {Class-Incremental Learning: A Survey},
    journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
    volume={46},
    number={12},
    pages={9851--9873},
    year = {2024}
}

📰 What's New

🌟 Methods Reproduced

📝 Reproduced Results

CIFAR-100

<div align="center"> <img src="./resources/cifarb0inc10.jpg" width="600px"> </div>

ImageNet-R

<div align="center"> <img src="./resources/imagenetRb0inc20.jpg" width="600px"> </div>

For exemplar parameters, Coil, DER, iCaRL, MEMO, and FOSTER set the fixed_memory option to false and retain the memory_size of 2000 for CIFAR100, while setting fixed_memory option to true and retaining the memory_per_class of 20 for ImageNet-R. On the contrary, other models are exemplar-free.

☄️ how to use

🕹️ Clone

Clone this GitHub repository:

git clone https://github.com/sun-hailong/LAMDA-PILOT
cd LAMDA-PILOT

🗂️ Dependencies

  1. torch 2.0.1
  2. torchvision 0.15.2
  3. timm 0.6.12
  4. tqdm
  5. numpy
  6. scipy
  7. easydict

🔑 Run experiment

  1. Edit the [MODEL NAME].json file for global settings and hyperparameters.

  2. Run:

    python main.py --config=./exps/[MODEL NAME].json

  3. hyper-parameters

    When using PILOT, you can edit the global parameters and algorithm-specific hyper-parameter in the corresponding json file.

    These parameters include:

    • model_name: The model's name should be selected from the 11 methods listed above, i.e., finetune, icarl, coil, der, foster, memo, simplecil, l2p, dualprompt, coda-prompt and adam.
    • init_cls: The number of classes in the initial incremental stage. As the configuration of CIL includes different settings with varying class numbers at the outset, our framework accommodates diverse options for defining the initial stage.
    • increment: The number of classes in each incremental stage $i$, $i$ > 1. By default, the number of classes is equal across all incremental stages.
    • backbone_type: The backbone network of the incremental model. It can be selected from a variety of pre-trained models available in the Timm library, such as ViT-B/16-IN1K and ViT-B/16-IN21K. Both are pre-trained on ImageNet21K, while the former is additionally fine-tuned on ImageNet1K.
    • seed: The random seed is utilized for shuffling the class order. It is set to 1993 by default, following the benchmark setting iCaRL.
    • fixed_memory: a Boolean parameter. When set to true, the model will maintain a fixed amount of memory per class. Alternatively, when set to false, the model will preserve dynamic memory allocation per class.
    • memory_size: The total number of exemplars in the incremental learning process. If fixed_memory is set to false, assuming there are $K$ classes at the current stage, the model will preserve $\left[\frac{{memory-size}}{K}\right]$ exemplars for each class. L2P, DualPrompt, SimpleCIL, ADAM, and CODA-Prompt do not require exemplars. Therefore, parameters related to the exemplar are not utilized.
    • memory_per_class: If fixed memory is set to true, the model will preserve a fixed number of memory_per_class exemplars for each class.

🔎 Datasets

We have implemented the pre-processing datasets as follows:

These subsets are sampled from the original datasets. Please note that I do not have the right to distribute these datasets. If the distribution violates the license, I shall provide the filenames instead.

When training not on CIFAR100, you should specify the folder of your dataset in utils/data.py.

    def download_data(self):
        assert 0,"You should specify the folder of your dataset"
        train_dir = '[DATA-PATH]/train/'
        test_dir = '[DATA-PATH]/val/'

👨‍🏫 Acknowledgments

We thank the following repos providing helpful components/functions in our work.

🤗 Contact

If there are any questions, please feel free to propose new features by opening an issue or contact with the author: Hai-Long Sun(sunhl@lamda.nju.edu.cn) and Da-Wei Zhou(zhoudw@lamda.nju.edu.cn). Enjoy the code.

🚀 Star History

Star History Chart