Awesome
Adaptive Split-Fusion Transformer
Update:
19/02/2024
- Release the pretrained model weights.
31/08/2022
- Add the code for ImageNet-21k pretrained weights loading.
- Add the code for the pyramid version model ASF-former_p.
27/04/2022
- The paper is posted on arXiv and the code is released.
1. Requirements
timm==0.3.4
torch==1.8.0
torchvision
pyyaml
Data Preparation
ImageNet with following folder structure:
│imagenet/
├──train/
│ ├── n01440764
│ │ ├── n01440764_10026.JPEG
│ │ ├── n01440764_10027.JPEG
│ │ ├── ......
│ ├── ......
├──val/
│ ├── n01440764
│ │ ├── ILSVRC2012_val_00000293.JPEG
│ │ ├── ILSVRC2012_val_00002138.JPEG
│ │ ├── ......
│ ├── ......
2. Pretrained Models
ImageNet-1K Training:
Model | Params | Top-1 |
---|---|---|
ASF-former-S | 19.3M | 82.7% |
ASF-former-B | 56.7M | 83.9% |
ASF-former_p-S | 21.3M | 83.0% |
ASF-former_p-B | 58.9M | 83.9% |
ImageNet-22K Pretraining + ImageNet-1K Fine-tuning:
Model | Params | Top-1 |
---|---|---|
ASF-former-B | 56.7M | 85.2% |
3. Training
Train ASF-former-S with 8 GPUs:
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 ./distributed_train.sh 8 path/to/data --model ASF_former_S -b 64 --lr 5e-4 --weight-decay .05 --amp --img-size 224
Train ASF-former-B with 8 GPUs:
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 ./distributed_train.sh 8 path/to/data --model ASF_former_B -b 64 --lr 5e-4 --weight-decay .065 --amp --img-size 224
Train ASF-former_p-S with 8 GPUs:
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 ./distributed_train.sh 8 /data/imagenet --model ASF_former_p_S -b 64 --lr 5e-4 --weight-decay .05 --amp --img-size 224
Train ASF-former_p-B with 8 GPUs:
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 ./distributed_train.sh 8 /data/imagenet --model ASF_former_p_B -b 64 --lr 5e-4 --weight-decay .065 --amp --img-size 22
4. Evaluation
Evaluate the ASF-former-S model:
CUDA_VISIBLE_DEVICES=0 python main.py path/to/data --model ASF_former_S -b 100 --eval_checkpoint path/to/checkpoint
Evaluate the ASF-former-B model:
CUDA_VISIBLE_DEVICES=0 python main.py path/to/data --model ASF_former_B -b 100 --eval_checkpoint path/to/checkpoint
5. Transfer Learning
Transfer ASF-former-S to CIFAR-10:
CUDA_VISIBLE_DEVICES=0,1 python transfer_learning.py --lr 0.025 --b 64 --num-classes 10 --img-size 224 --transfer-learning True --transfer-model path/to/model
Transfer ASF-former-B to CIFAR-10:
CUDA_VISIBLE_DEVICES=0,1 python transfer_learning.py --lr 0.025 --b 64 --num-classes 10 --img-size 224 --transfer-learning True --transfer-model path/to/model --model ASF_former_B
Transfer ASF-former-S to CIFAR-100:
CUDA_VISIBLE_DEVICES=0,1 python transfer_learning.py --lr 0.05 --b 64 --num-classes 100 --img-size 224 --transfer-learning True --transfer-model path/to/model --dataset cifar100
Transfer ASF-former-B to CIFAR-100:
CUDA_VISIBLE_DEVICES=0,1 python transfer_learning.py --lr 0.05 --b 64 --num-classes 100 --img-size 224 --transfer-learning True --transfer-model path/to/model --dataset cifar100 --model ASF_former_B
Citation
@article{Su2022AdaptiveST,
title={Adaptive Split-Fusion Transformer},
author={Zixuan Su and Hao Zhang and Jingjing Chen and Lei Pang and Chong-Wah Ngo and Yu-Gang Jiang},
journal={ArXiv},
year={2022},
volume={abs/2204.12196}
}
Our codes are based on T2T-ViT.