Home

Awesome

RankFeat

NeurIPS22 paper "RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection"

Expanded T-PAMI journal "RankFeat&RankWeight: Rank-1 Feature/Weight Removal for Out-of-distribution Detection"

ID/OOD Dataset Preparation

In-Distribution (ID) dataset. Please download ImageNet-1k and use its validation set as the ID set.

Out-of-Distribution (OOD) dataset. For the used OOD datasets (iNaturalist, SUN, Places, and Textures), please download them from the following links:

wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/iNaturalist.tar.gz
wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/SUN.tar.gz
wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/Places.tar.gz
wget https://www.robots.ox.ac.uk/~vgg/data/dtd/download/dtd-r1.0.1.tar.gz

For the large-scale Species dataset, please download the four subsets (Protozoa, Microorganisms, Plants, and Mollusks) from the official link.

Pre-trained Model Preparation

For SqueezeNet, it is already available in the Pytorch library. For BiT-S ResNetv2-101 and T2T-ViT-24, one can download the BiT-S ResNetv2-101 and T2T-ViT by the following links:

wget http://pages.cs.wisc.edu/~huangrui/finetuned_model/BiT-S-R101x1-flat-finetune.pth.tar
wget https://github.com/yitu-opensource/T2T-ViT/releases/download/main/82.6_T2T_ViTt_24.pth.tar

For ViT-B/16 and Swin-B, we use the pretrained model from torchvision. The download will start automatically when the test scripts are executed for the first time.

For more BiT pre-trained models, one can also refer to BiT-S pre-trained families.

Usage

Check test_resnetv2.sh, test_squeezenet.sh, test_t2tvit.sh, test_vit.sh and test_swin.sh in the scripts folder for the specific usages on each architecture.

Besides our proposed RankFeat, the previous post hoc methods are also implemented here, including MSP, ODIN, Energy, Mahalanobis, GradNorm, and ReAct.

OOD Evaluation Results

<div align=center><img src="journal_resnet.jpg" width="80%"/></div> <div align=center><img src="journal_t2t_sqz_vit_swin.jpg" width="80%"/></div> <div align=center><img src="journal_rankweight.jpg" width="80%"/></div> <div align=center><img src="journal_species.jpg" width="80%"/></div>

Citation

If you think the code is helpful to your research, please consider citing our paper:

@inproceedings{song2022rankfeat,
  title={RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection},
  author={Song, Yue and Sebe, Nicu and Wang, Wei},
  booktitle={NeurIPS},
  year={2022}
}
@article{song2024rankfeatweight,
  title={RankFeat&RankWeight: Rank-1 Feature/Weight Removal for Out-of-distribution Detection},
  author={Song, Yue and Wang, Wei and Sebe, Nicu},
  journal={IEEE TPAMI},
  year={2024},
  publisher={IEEE}
}

The code is built based on MOS and GradNorm. If you have any questions or suggestions, please feel free to contact me via yue.song@unitn.it.