Awesome
Adaptive Selection based Referring Image Segmentation
This is an official PyTorch implementation of ASDA (accepted by ACMMM 2024).
News
- [July 16, 2024] The paper is accepted by ACMMM 2024🎉.
- [Oct 22, 2024] Pytorch implementation of ASDA is released.
Main Results
Main results on RefCOCO
| Model | Backbone | val | test A | test B |
|---|---|---|---|---|
| CRIS | ResNet101 | 70.47 | 73.18 | 66.10 |
| ASDA | ViT-B | 75.06 | 77.14 | 71.36 |
Main results on RefCOCO+
| Model | Backbone | val | test A | test B |
|---|---|---|---|---|
| CRIS | ResNet101 | 62.27 | 68.08 | 53.68 |
| ASDA | ViT-B | 66.84 | 71.13 | 57.83 |
Main results on G-Ref
| Model | Backbone | val(U) | test(U) | val(G) |
|---|---|---|---|---|
| CRIS | ResNet101 | 59.87 | 60.36 | - |
| ASDA | ViT-B | 65.73 | 66.45 | 63.55 |
Quick Start
Environment preparation
conda create -n ASDA python=3.6 -y
conda activate ASDA
# install pytorch according to your cuda version
# don't change version of torch, or it may occur conflict
conda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 cudatoolkit=11.3 -c pytorch -c conda-forge
pip install -r requirements.txt
Dataset Preparation
1. Download the COCO train2014 to ASDA/ln_data/images.
wget https://pjreddie.com/media/files/train2014.zip
2. Download the RefCOCO, RefCOCO+, RefCOCOg to ASDA/ln_data.
mkdir ln_data && cd ln_data
# The original link bvisionweb1.cs.unc.edu/licheng/referit/data/refclef.zip is no longer valid, we have uploaded it to Google Drive (https://drive.google.com/file/d/1AnNBSL1gc9uG1zcdPIMg4d9e0y4dDSho/view?usp=sharing)
wget 'https://drive.usercontent.google.com/download?id=1AnNBSL1gc9uG1zcdPIMg4d9e0y4dDSho&export=download&authuser=0&confirm=t&uuid=be656478-9669-4b58-ab23-39f196f88c07&at=AN_67v3n4xwkPBdEQ9pMlwonmhrH%3A1729591897703' -O refcoco_all.zip
unzip refcoco_all.zip
3. Run data.sh to generate the annotations.
mkdir dataset && cd dataset
bash data.sh
Training & Testing
bash train.sh 0,1
bash test.sh 0
License
This project is under the MIT license. See LICENSE for details.
Acknowledgement
Thanks for a lot of codes from CRIS, VLT, ViTDet.
Citation
If you find our work useful in your research, please consider citing:
@inproceedings{yue2024adaptive,
title={Adaptive Selection based Referring Image Segmentation},
author={Yue, Pengfei and Lin, Jianghang and Zhang, Shengchuan and Hu, Jie and Lu, Yilin and Niu, Hongwei and Ding, Haixin and Zhang, Yan and JIANG, GUANNAN and Cao, Liujuan and others},
booktitle={ACM Multimedia 2024}
}