Awesome

Salient objects in clutter (TPAMI2022)

Authors: Deng-Ping Fan, Jing Zhang, Gang Xu, Ming-Ming Cheng, Ling Shao.

1. Preface

• This repository provides benchamrk for "Salient objects in clutter". (paper | 中文版)

• If you have any questions about our paper, feel free to contact me. And if you are using SOCbenchamrk results or evaluation toolbox for your research, please cite this paper (BibTeX).

2. Scope

Salient object detection (SOD) originated from the task of fixation prediction (FP), switching attention regions for accurate object-level regions. Current algorithms have been developed for 2D images of limited resolution (width or height $<$ 500 pixels), high-resolution (i.e., 1080p, 4K) and even remote sensing data. According to the supervision strategy, there are five types of SOD models: fully supervised, semi-supervised, weakly supervised, unsupervised, and self-supervised.

Recently, several interesting extensions of SOD have also been introduced, such as salient instance detection (SID), salient object subitizing (SOS), and saliency ranking. A taxonomy of the saliency detection task is shown below. Different from previous SOD reviews, we mainly focus on 2D salient object detection in a fully supervised manner. We highlight the scope of this study in gray. For other closely related 3D/4D SOD tasks, we refer readers to recent survey and benchmarking works such as RGB-D SOD, Event-RGB SOD (ERSOD), Light Field SOD,
Co-SOD, 360° Video SOD, and Video SOD.

Fig.1 Taxonomy of the saliency detection task. We highlight the scope of this study in gray.

3. Evaluation Code

Link: https://github.com/mczhuge/SOCToolbox Note: If you want to list your results on our web, please send your name, model name, paper title to us. Important Tips: Some works use the so-called same F-measure metric, while they do not explicitly describe which statistic (e.g., mean or max) they used, easily resulting in unfair comparison and inconsistent performance. Meanwhile, the different threshold strategies in F-measure (e.g., 255 varied thresholds, adaptive saliency threshold, and self-adaptive threshold) will result in different performance. Fairly comparing RGB-D based SOD models by extensively evaluating them with same metrics on standard benchmarks is highly desired. So you need to download all of our re-organized datasets, saliency maps of each model and then evaluate your model.

4. SOC Dataset

Fig.2 Examples from our new SOCdataset,including non-salient (first row) and salient object images (rows 2 to 4). For salient object images, an instance-level ground-truth map (different color), object attributes (Attr) and category labels are provided.

Fig.3 (a) Number of annotated instances per category in our SOC dataset. (b, c) Global and local color contrast statistics, respectively. (d) A set of saliency maps from our dataset and their overlay map. (e) Location distribution of the salient objects in SOC. (f) Distribution of instance sizes in the SOCand ILSO datasets. (g) Visual examples of attributes. Best view on screen and zoomed-in for details.

Object-level Ground-Truth of the SOC Test Set released. Baidu

Instance-level Ground-Truth of the SOC Test Set released. Baidu

Note that the Test Set only contains images and without ground truth. We will create the SOC Benchmark website soon and you can upload your result to obtain the final score in our website. Also, you can use the Validation Set as Test Set first. Note that the image file of COCO_train2014_000000080168.PNG should be changed with new file name COCO_train2014_000000080168.png to prevent some errors during training your model.

Download link: Baidu.

5. 2D RGB Saliency Detection Models

Note that: If the model used S-measure/E-measure will be marked with <strong>bold</strong>.

Traditional Methods (Updated: 2022-04-04)

Download link: Google.

Deep Learning Methods (Updated: 2022-04-05)

Download link: Google.

6. Attribute-Level Performance

Download link: Google.

7. Application Example

Demo1: https://dengpingfan.github.io/videos/SOD-App-BASNetAR.mp4

8. Citation

Please cite our paper if you use our results:

@article{fan2022salient,
   title={Salient objects in clutter},
   author={Fan, Deng-Ping and Zhang, Jing and Xu, Gang and Cheng, Ming-Ming and Shao, Ling},
   journal={IEEE TPAMI},
   year={2022}
}