Home

Awesome

RIS-DMMI

This repository provides the PyTorch implementation of DMMI in the following papers:<br /> Beyond One-to-One: Rethinking the Referring Image Segmentation (ICCV2023) <br />

News

Dataset

We collect a new comprehensive dataset Ref-ZOM (Zero/One/Many), which contains image-text pairs in one-to-zero, one-to-one and one-to-many conditions. Similar to RefCOCO, RefCOCO+ and G-Ref, all the images in Ref-ZOM are selected from COCO dataset. Here, we provide the text, image and annotation information of Ref-ZOM, which should be utilized with COCO_trainval2014 together. <br /> Our dataset could be downloaded from:<br /> [Baidu Cloud] [Google Drive] <br /> Remember to download original COCO dataset from:<br /> [COCO Dowanload]<br />

Code

Prepare<br />

Train<br />

# Refcoco
--dataset refcoco, --splitBy unc, --split val
# Refcoco+
--dataset refcoco+, --splitBy unc, --split val
# Refcocog(umd)
--dataset refcocog, --splitBy umd, --split val
# Refcocog(google)
--dataset refcocog, --splitBy google, --split val
# Ref-zom
--dataset ref-zom, --splitBy final, --split test
sh train.sh

Test

sh test.sh

Parameter

Refcocog(umd)<br />

BackboneoIoUmIoUGoogle DriveBaidu Cloud
ResNet-10159.0262.59LinkLink
Swin-Base63.4666.48LinkLink

Ref-ZOM<br />

BackboneoIoUmIoUGoogle DriveBaidu Cloud
Swin-Base68.7768.25LinkLink

Acknowledgements

We strongly appreciate the wonderful work of LAVT. Our code is partially founded on this code-base. If you think our work is helpful, we suggest you refer to LAVT and cite it as well.<br />

Citation

If you find our work is helpful and want to cite our work, please use the following citation info.<br />

@InProceedings{Hu_2023_ICCV,
    author    = {Hu, Yutao and Wang, Qixiong and Shao, Wenqi and Xie, Enze and Li, Zhenguo and Han, Jungong and Luo, Ping},
    title     = {Beyond One-to-One: Rethinking the Referring Image Segmentation},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2023},
    pages     = {4067-4077}
}