Awesome
Long-Tailed-Classification-Leaderboard
date: 2021/3/3(Updated 2021/3/3)<br />
auther: YW YSZ
1. Introduction
List of abbreviations:
Abbreviations | ReW | TrL | MeL | DeL | Aug | SeSu | OtM |
---|
Full names | Re-weighting | Transfer Learning | Meta Learning | Decoupling Learning | Data Augmentation | Self-Supervised Learning | Other methods |
2. Benchmark datasets
3. Leaderboard
3.1 CIFAR-10-LT
Evaluation metric: classification error rate.
IF
represents Imbalance factor
.
Backbone: ResNet-32
Backbone: ResNet-18
Method | Venue | Year | Backbone | Type | IF=10 | IF=50 | IF=100 | Code | Reported by |
---|
FSA | ECCV | 2020 | ResNet-18 | Aug | 8.25 | 15.29 | 19.43 | ---- | Source |
Backbone: ResNet-34
Method | Venue | Year | Backbone | Type | IF=10 | IF=50 | IF=100 | Code | Reported by |
---|
FSA | ECCV | 2020 | ResNet-34 | Aug | 8.8 | 15.51 | 17.94 | ---- | Source |
3.2 CIFAR-100- LT
Evaluation metric: classification error rate.
Backbone: ResNet-32
Backbone: ResNet-18
Method | Venue | Year | Backbone | Type | IF=10 | IF=50 | IF=100 | Code | Reported by |
---|
FSA | ECCV | 2020 | ResNet-18 | Aug | 34.92 | 48.1 | 53.43 | ---- | Source |
Backbone: ResNet-34
Method | Venue | Year | Backbone | Type | IF=10 | IF=50 | IF=100 | Code | Reported by |
---|
FSA | ECCV | 2020 | ResNet-34 | Aug | 34.71 | 47.83 | 51.49 | ---- | Source |
3.3 ImageNet-LT
Evaluation metric: closed-set setting/Top-1 classification accuracy.
Backbone: ResNet-10
Method | Venue | Year | Backbone | Type | Many-Shot | Medium-Shot | Few-Shot | ALL | Code | Reported by |
---|
OLTR | CVPR | 2019 | ResNet-10 | TrL | 43.2 | 35.1 | 18.5 | 35.6 | ---- | Source |
LWS | ICLR | 2020 | ResNet-10 | DeL | ----- | ----- | ---- | 41.4 | ---- | Source |
IEM | CVPR | 2020 | ResNet-10 | OtM | 48.9 | 44.0 | 24.4 | 43.2 | ---- | Source |
LFME+OLTR | ECCV | 2020 | ResNet-10 | TrL | 47.0 | 37.9 | 19.2 | 38.8 | ---- | Source |
FSA | ECCV | 2020 | ResNet-10 | Aug | 47.3 | 31.6 | 14.7 | 35.2 | ---- | Source |
BALMS | NeurIPS | 2020 | ResNet-10 | ReW | 50.3 | 39.5 | 25.3 | 41.8 | ---- | Source |
cRT + SSP | NeurIPS | 2020 | ResNet-10 | SeSu | ----- | ----- | ---- | 43.2 | ---- | Source |
Baseline + tricks | AAAI | 2021 | ResNet-10 | OtM | ----- | ----- | ---- | 43.31 | ---- | Source |
Backbone: ResNeXt-50
Method | Venue | Year | Backbone | Type | Many-Shot | Medium-Shot | Few-Shot | ALL | Code | Reported by |
---|
LWS | ICLR | 2020 | ResNeXt-50 | DeL | 60.2 | 47.2 | 30.3 | 49.9 | ---- | Source |
Backbone: ResNeXt-152
Method | Venue | Year | Backbone | Type | Many-Shot | Medium-Shot | Few-Shot | ALL | Code | Reported by |
---|
LWS | ICLR | 2020 | ResNeXt-152 | DeL | 63.5 | 50.4 | 34.2 | 53.3 | ---- | Source |
3.4 Places-LT
Evaluation metric: closed-set setting/Top-1 classification accuracy.
Backbone: ResNet-152
Method | Venue | Year | Backbone | Type | Many-Shot | Medium-Shot | Few-Shot | ALL | Code | Reported by |
---|
OLTR | CVPR | 2019 | ResNet-152 | TrL | 44.7 | 37 | 25.3 | 35.9 | ---- | Source |
LWS | ICLR | 2020 | ResNet-152 | DeL | 40.6 | 39.1 | 28.6 | 37.6 | ---- | Source |
τ -normalized | ICLR | 2020 | ResNet-152 | DeL | 37.8 | 40.7 | 31.8 | 37.9 | ---- | Source |
IEM | CVPR | 2020 | ResNet-152 | OtM | 46.8 | 39.2 | 28.0 | 39.7 | ---- | Source |
LFME+OLTR | ECCV | 2020 | ResNet-152 | TrL | 39.3 | 39.6 | 24.2 | 36.2 | ---- | Source |
FSA | ECCV | 2020 | ResNet-152 | Aug | 42.8 | 37.5 | 22.7 | 36.4 | ---- | Source |
Backbone: ResNet-10
Method | Venue | Year | Backbone | Type | Many-Shot | Medium-Shot | Few-Shot | ALL | Code | Reported by |
---|
BALMS | NeurIPS | 2020 | ResNet-10 | ReW | 41.2 | 39.8 | 31.6 | 38.7 | ---- | |
3.5 iNaturalist
Evaluation metric: Top-1 classification accuracy
Backbone: ResNet-50
Backbone: ResNet-152
Method | Venue | Year | Backbone | Type | iNat-2017(Top1) | iNat-2018(Top1) | Code | Reported by |
---|
CB Focal | CVPR | 2019 | ResNet-152 | ReW | 61.84 | 64.16 | ---- | Source |
LWS | ICLR | 2020 | ResNet-152 | DeL | ----- | 69.1/72.1 (90/200) | ---- | Source |
FSA | ECCV | 2020 | ResNet-152 | Aug | 66.58 | 69.08 | ---- | Source |
4. Contact
Yan Wang : yanwang@smail.nju.edu.cn
Yongshun Zhang: zhangys@lamda.nju.edu.cn
5. References
2019
- Shu et.al., Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting, NeurIPS 2019.
- Cao et.al., Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss, NeurIPS 2019.
- Cui et.al., Class-Balanced Loss Based on Effective Number of Samples, CVPR 2019.
2020
- Tang et.al., Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect, NeurIPS 2020.
- Yang et.al., Rethinking the Value of Labels for Improving Class-Imbalanced Learning, NeurIPS 2020.
- Ren et.al., Balanced Meta-Softmax for Long-Tailed Visual Recognition, NeurIPS 2020.
- Kang et.al., Decoupling Representation and Classifier for Long-Tailed Recognition, ICLR 2020.
- Kim et.al., M2m: Imbalanced Classification via Major-to-minor Translation, CVPR 2020.
- Zhou et.al., BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition, CVPR 2020.
- Jamal et.al., Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspective, CVPR 2020.
- Zhu et.al., Inflated Episodic Memory with Region Self-Attention for Long-Tailed Visual Recognition, CVPR 2020.
- Liu et.al., Large-Scale Long-Tailed Recognition in an Open World, CVPR 2019.
- Xiang et.al., Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification, ECCV 2020.
- Chu et.al., Feature Space Augmentation for Long-Tailed Data, ECCV 2020.
- Ren et.al., Balanced Activation for Long-tailed Visual Recognition, ECCV 2020.
- Chou et.al., Remix: Rebalanced Mixup, ECCV'2020 Workshop.
2021
- Zhang et.al., Bag of Tricks for Long-Tailed Visual Recognition with Deep Convolutional Neural Networks, AAAI 2021.