Home

Awesome

NomMer Transformer

This is the official implementation of our NomMer: Nominate Synergistic Context in Vision Transformer for Visual Recognition, by Hao Liu, Xinghua Jiang, Xin Li, Zhimin Bao, Deqiang Jiang and Bo Ren.

03/02/2022: NomMer got accepted by CVPR 2022.

Introduction

<img src="./figures/nommer.png" width = "640" height = "628" alt="NomMer-transformer" align=center />

We propose a novel ViT architecture, termed NomMer, which can dynamically Nominate the synergistic global-local context in vision transforMer.

Benchmarking

Image Classification on ImageNet-1K

ModelPretrainResolutionacc@1#paramsFLOPs
NomMer-TIN-1K22482.622M5.4G
NomMer-SIN-1K22483.742M10.1G
NomMer-BIN-1K22484.573M17.6G

Getting Started

Evaluation

To evaluate a pre-trained NomMer on ImageNet val, run:

python -m torch.distributed.launch --nproc_per_node <num-of-gpus-to-use> --master_port 12345 main.py --eval \
--cfg <config-file> --batch-size <batch-size-per-gpu> --resume <checkpoint> --data-path <imagenet-path>

Training from scratch

To train a NomMer on ImageNet from scratch, run:

python -m torch.distributed.launch --nproc_per_node <num-of-gpus-to-use> --master_port 12345 main.py \
--cfg <config-file> --data-path <imagenet-path> --batch-size <batch-size-per-gpu> --output <output-directory>

Citation

If you find NomMer useful in your research, please consider citing:

@InProceedings{Liu_2022_CVPR,
    author    = {Liu, Hao and Jiang, Xinghua and Li, Xin and Bao, Zhimin and Jiang, Deqiang and Ren, Bo},
    title     = {NomMer: Nominate Synergistic Context in Vision Transformer for Visual Recognition},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {12073-12082}
}

Acknowledgement

Our codebase is built based on Swin-Transformer. We thank the authors for the nicely organized code!