Awesome
RbA: Segmenting Unknown Regions Rejected by All (ICCV 2023)
state-of-the-art on SegmentMeIfYouCan Benchmark
Nazir Nayal, Mısra Yavuz, João Henriques, Fatma Güney
<div align="center"> <img src="assets/overview.jpg" width="100%" height="100%"/> </div><br/>Updates
- 25.10.2024: Model Weights are available now as a release!
- code with training and evaluation scripts are available
Installation
See installation instructions for necessary installations and setup
Datasets Preparation
See Dataset Preparation for details on downloading and preparing datasets for both training and evaluation.
Model Zoo and Baselines
We provide the checkpoints and config files used to train RbA under different configurations in the RbA Model Zoo.
Training
In order to replicate any of our experiments, we provide the config files for all the models in the RbA Model Zoo. Given the config file, a training experiment can be run using the following command:
python train_net.py \
--config-file PATH_TO_CONFIG_FILE \
--num-gpus NUM_GPUS \
OUTPUT_DIR PATH_TO_STORE_CKPTS_AND_LOGS
For example, to train a Swin-B model with 1 decoder layer, on 4 gpus and store the outputs in a folder named model_logs/swin_b_1dl
you can use the following:
python train_net.py \
--config-file configs/cityscapes/semantic-segmentation/swin/single_decoder_layer/maskformer2_swin_base_IN21k_384_bs16_90k_1dl.yaml \
--num-gpus 4 \
OUTPUT_DIR model_logs/swin_b_1dl/
NOTE: some experiments require the model to be initialized from pretrained weights, make sure the required weights are available under the pretrained/
folder. Details about the require pretrained weights can be found in RbA Model Zoo.
Evaluation
We provide evaluate_ood.py
for evaluating on OoD datasets. A simple usage for the script is as follows:
python evaluate_ood.py
--out_path results_test/ \ # folder to store results as pkl files
--models_folder ckpts/ \
--datasets_folder PATH_TO_DATASETS_ROOT \
--model_mode all \ # evaluates all models in the models_folder
--dataset_mode all \ # evaluate on all datasets (RA & FS LaF)
The script assumes the following:
- The OoD datasets are setup as described in Datasets Prepration
- The parameter
--models_folder
is a path to a folder that contains multiple folders, where each folder corresponds to a model. In a model's folder the scripts expects to files: 1)config.yaml
and its checkpoint 2)model_final.pth
. Setting up the models is explained in RbA Model Zoo Introduction
The scripts supports more finegrained options like selecting subsets of the models in a folder or the datasets. Please check evaluate_ood.py
for descriptions of the options.
<a name="CitingRbA"></a>Citing RbA
If you use RbA in your research, please use the following BibTeX entry.
@InProceedings{nayal2023ICCV,
author = {Nazir Nayal and Mısra Yavuz and João F. Henriques and Fatma Güney},
title = {RbA: Segmenting Unknown Regions Rejected by All},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
year = {2023},
}
Acknowledgement
This repo is built mainly on top Mask2Former repo: (https://github.com/facebookresearch/Mask2Former).
Different code snippets are also adapted from the following repos:
We thank all of the authors of these repos for their contributions.