Home

Awesome

Recognize Any Regions(NeurIPS 2024)

teaser

Recognize Any Regions
Haosen Yang, Chuofan Ma, Bin Wen, Yi Jiang, Zehuan Yuan, Xiatian Zhu

Updates

Models

MethodBox AP_rareBox AP_allMask AP_rareMask AP_allDownload
RegionSpot-BB19.120.917.517.8model
RegionSpot-BL26.023.722.820.2model
RegionSpot-BL@336px26.325.023.421.3model

Getting Started

The installation instruction and usage are in Getting Started with Recognize Any Regions.

Demo

First download a model checkpoint. Then the model can be used in just a few lines to get masks from a given prompt:

from regionspot.modeling.regionspot import build_regionspot_model
from regionspot import RegionSpot_Predictor
custom_vocabulary =  ['<custom>']
clip_type = <clip_type>
regionspot = build_regionspot_model(checkpoint="<path/to/checkpoint>", custom_vocabulary=custom_vocabulary, clip_type=clip_type)
predictor = RegionSpot_Predictor(regionspot)
predictor.set_image(<your_image>)
masks, mask_iou_score, class_score, class_index = predictor.predict(<input_prompts>)

See the demo.py on using RegionSpot with box prompts for more details. teaser

Citing Recognize Any Regions

If you use Recognize Any Regions in your research or wish to refer to the baseline results published here, please use the following BibTeX entry.

@inproceedings{RegionSpot,
  title={Recognize Any Regions},
  author={Yang, Haosen and Ma, Chuofan and Wen, Bin and Jiang, Yi and Yuan, Zehuan and Zhu, Xiatian},
  journal={arXiv preprint arXiv:2311.01373},
  year={2023}
}