Home

Awesome

Vision GNN

By Kai Han, Yunhe Wang, Jianyuan Guo, Yehui Tang and Enhua Wu. NeurIPS 2022. [arXiv link]

image

Requirements

Visualization of Image and Heatmap

<table> <tr> <td>Raw Image</td> <td>Grad-Cam HearMap</td> </tr> <tr> <td><img src="./fig/ILSVRC2012_val_00000075.JPEG" width=425 ></td> <td><img src="./fig/cam.jpg" width=300></td> </tr> </table>

More details can be checked in Notebook.

Pretrained models

ModelParams (M)FLOPs (B)Top-1Github Release
ViG-Ti7.11.373.9Github Release
ViG-S22.74.580.4Github Release
ViG-B86.817.782.3Github Release
ModelParams (M)FLOPs (B)Top-1BaiduDisk URLGithub Release
Pyramid ViG-Ti10.71.778.5[BaiduDisk], Password: chaeGithub Release
Pyramid ViG-S27.34.682.1[BaiduDisk], Password: 81mgGithub Release
Pyramid ViG-M51.78.983.1[BaiduDisk], Password: prd3Github Release
Pyramid ViG-B82.616.883.7[BaiduDisk], Password: rgm4Github Release

Evaluation

Data preparation follows the official pytorch example

python train.py /path/to/imagenet/ --model pvig_s_224_gelu -b 128 --pretrain_path /path/to/pretrained/model/ --evaluate

Training

python -m torch.distributed.launch --nproc_per_node=8 train.py /path/to/imagenet/ --model vig_s_224_gelu --sched cosine --epochs 300 --opt adamw -j 8 --warmup-lr 1e-6 --mixup .8 --cutmix 1.0 --model-ema --model-ema-decay 0.99996 --aa rand-m9-mstd0.5-inc1 --color-jitter 0.4 --warmup-epochs 20 --opt-eps 1e-8 --repeated-aug --remode pixel --reprob 0.25 --amp --lr 2e-3 --weight-decay .05 --drop 0 --drop-path .1 -b 128 --output /path/to/save/models/
python -m torch.distributed.launch --nproc_per_node=8 train.py /path/to/imagenet/ --model pvig_s_224_gelu --sched cosine --epochs 300 --opt adamw -j 8 --warmup-lr 1e-6 --mixup .8 --cutmix 1.0 --model-ema --model-ema-decay 0.99996 --aa rand-m9-mstd0.5-inc1 --color-jitter 0.4 --warmup-epochs 20 --opt-eps 1e-8 --repeated-aug --remode pixel --reprob 0.25 --amp --lr 2e-3 --weight-decay .05 --drop 0 --drop-path .1 -b 128 --output /path/to/save/models/

Citation

@inproceedings{han2022vig,
  title={Vision GNN: An Image is Worth Graph of Nodes}, 
  author={Kai Han and Yunhe Wang and Jianyuan Guo and Yehui Tang and Enhua Wu},
  booktitle={NeurIPS},
  year={2022}
}

Acknowledgements

This repo partially uses code from deep_gcns_torch and timm. Our code is modified and adapted on these great repositories:

Citation

If you use this method or this code in your research, please cite as:

@inproceedings{han2022vig,
  title={Vision GNN: An Image is Worth Graph of Nodes}, 
  author={Kai Han and Yunhe Wang and Jianyuan Guo and Yehui Tang and Enhua Wu},
  booktitle={NeurIPS},
  year={2022}
}