Home

Awesome

CLass adaptive Linear Probing (CLAP)

The official implementation of A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models.<br/> Julio Silva-Rodriguez, Sina Hajimiri, Ismail Ben Ayed, Jose Dolz <br/> ÉTS Montreal <br/> | Project | Paper | Code | <br/>

When adapting CLIP using only few-shot, it is unrealistic to assume the presence of a validation subset to empirically fix a set of hyperparameters per task, i.e. model selection. We propose two solutions, which do not require any hyperparameter tuning, and thus is adapted strictly using only the support samples.

Installation

This repository requires to install the environment and datasets:

PS: You can also follow CoOp to perform the installation.

Usage

We present the basic usage here.

(a) Zero-shot initialized Linear Probe (ZS-LP):

(b) CLass adaptive Linear Probing (CLAP):

(c) Test domain generalization:

Acknowledgment

This repository is mainly based on CoOp and TaskRes code base. We sincerely thank prior authors on this topic for his awesome code base.

Citation

If you find this repository useful, please consider citing this paper:

@inproceedings{clap24,
    title={A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models},
    author={Julio Silva-Rodr\'iguez and Sina Hajimiri and Ismail Ben Ayed and Jose Dolz},
    booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2024}
    }