Home

Awesome

<div align="center">

MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognition

Shuang Li, Kaixiong Gong, et al.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. [CVPR 2021 PDF]

Paper

</div>

This repository contains the code of our CVPR 2021 work "MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognition".

PWC PWC PWC PWC PWC PWC PWC

Abstract

Real-world training data usually exhibits long-tailed distribution, where several majority classes have a significantly larger number of samples than the remaining minority classes. This imbalance degrades the performance of typical supervised learning algorithms designed for balanced training sets. In this paper, we address this issue by augmenting minority classes with a recently proposed implicit semantic data augmentation (ISDA) algorithm, which produces diversified augmented samples by translating deep features along many semantically meaningful directions. Importantly, given that ISDA estimates the classconditional statistics to obtain semantic directions, we find it ineffective to do this on minority classes due to the insufficient training data. To this end, we propose a novel approach to learn transformed semantic directions with metalearning automatically. In specific, the augmentation strategy during training is dynamically optimized, aiming to minimize the loss on a small balanced validation set, which is approximated via a meta update step. Extensive empirical results on CIFAR-LT-10/100, ImageNet-LT, and iNaturalist2017/2018 validate the effectiveness of our method

<p align="center"> <img src="assets/illustration.png" alt="drawing" width="800"/> </p>

If you find this idea or code useful for your research, please consider citing our paper:

@inproceedings{li2021metasaug,
  title={Metasaug: Meta semantic augmentation for long-tailed visual recognition},
  author={Li, Shuang and Gong, Kaixiong and Liu, Chi Harold and Wang, Yulin and Qiao, Feng and Cheng, Xinjing},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={5212--5221},
  year={2021}
}

Prerequisite

Evaluation

We provide several trained models of MetaSAug for evaluation.

Testing on CIFAR-LT-10/100:

Testing on ImageNet and iNaturalist18:

The trained models are in Google Drive.

Getting Started

Dataset

Training

Training on CIFAR-LT-10/100:

CIFAR-LT-100, MetaSAug with LDAM loss
python3.6 MetaSAug_LDAM_train.py --gpu 0 --lr 0.1 --lam 0.75 --imb_factor 0.05 --dataset cifar100 --num_classes 100 --save_name MetaSAug_cifar100_LDAM_imb0.05 --idx 1

Or run the script:

sh scripts/MetaSAug_LDAM_train.sh

Training on ImageNet-LT:

CUDA_VISIBLE_DEVICES=0,1,2,3 python3 -m torch.distributed.launch --nproc_per_node=4 --master_port 53212 train.py  --lr 0.0003 --meta_lr 0.1 --workers 0 --batch_size 256 --epochs 20 --dataset ImageNet_LT --num_classes 1000 --data_root ../ImageNet

Or run the script:

sh ImageNet_iNat/scripts/train.sh

Note: Training on large scale datasets like ImageNet-LT and iNaturalist2017/2018 involves multiple gpus for faster speed. To achieve better generalizable representations, vanilla CE loss is used for training the network in the early training stage. For convenience, the training starts from the pre-trained models, e.g., ImageNet-LT, iNat18 (both from project cRT).

Results and models

CIFAR-LT-10

ModelImb.Top-1 ErrorDownloadModelImb.Top-1 ErrorDownload
MetaSAug+LDAM20022.65ResNet32MetaSAug+CE20023.11ResNet32
MetaSAug+LDAM10019.34ResNet32MetaSAug+CE10019.46ResNet32
MetaSAug+LDAM5015.66ResNet32MetaSAug+CE5015.97ResNet32
MetaSAug+LDAM2011.90ResNet32MetaSAug+CE2012.36ResNet32
MetaSAug+LDAM1010.32ResNet32MetaSAug+CE1010.56ResNet32

CIFAR-LT-100

ModelImb.Top-1 ErrorDownloadModelImb.Top-1 ErrorDownload
MetaSAug+LDAM20056.91ResNet32MetaSAug+CE20060.06ResNet32
MetaSAug+LDAM10051.99ResNet32MetaSAug+CE10053.13ResNet32
MetaSAug+LDAM5047.73ResNet32MetaSAug+CE5048.10ResNet32
MetaSAug+LDAM2042.47ResNet32MetaSAug+CE2042.15ResNet32
MetaSAug+LDAM1038.72ResNet32MetaSAug+CE1038.27ResNet32

ImageNet-LT

ModelTop-1 ErrorDownload
MetaSAug52.33ResNet50

iNaturalist18

ModelTop-1 ErrorDownload
MetaSAug30.50ResNet50

Acknowledgements

Some codes in this project are adapted from Meta-class-weight and cRT. We thank them for their excellent projects.