Home

Awesome

SPSN: Superpixel Prototype Sampling Network for RGB-D Salient Object Detection (ECCV 2022)

PWC PWC PWC PWC PWC

Authors: Minhyeok Lee, Chaewon Park, Suhwan Cho, Sangyoun Lee

This repository provides code for paper "SPSN: Superpixel Prototype Sampling Network for RGB-D Salient Object Detection" accepted by the ECCV 2022 conference.

Our paper can be found [arXiv]

<img align="center" src="./images/main.png" width="800px" />

Prepared Datasets

Download the train and test dataset from Google Drive.

Requirements

For the superpixel algorithm we use fast_slic. You can install it like this:

pip install fast_slic

Training Model

  1. First, clone this repository.
git clone https://github.com/Hydragon516/SPSN
  1. Edit config.py. The data root path option and GPU index should be modified.
  2. Train the model.
python3 train.py

Evaluation

When training is complete, the prediction results for the test set are saved in the ./log folder. Two popular evaluation toolboxes are available. (Matlab version: https://github.com/DengPingFan/CODToolbox Python version: https://github.com/lartpang/PySODMetrics)

Result

<img align="center" src="./images/result.png" width="600px" />

./log directory structure

.
├── root
└── log/
    └── 2022-xx-xx xx:xx:xx/
        ├── model/
        │   └── best_model.pth
        ├── result/
        │   ├── gt/        # ground truth images
        │   │   ├── NJU2K
        │   │   ├── NLPR
        │   │   ├── DES
        │   │   ├── SIP
        │   │   └── STERE
        │   ├── pred/      # predicted mask images (only mask)
        │   │   ├── NJU2K
        │   │   ├── NLPR
        │   │   ├── DES
        │   │   ├── SIP
        │   │   └── STERE
        │   └── total/     # includes RGB, depth, GT, superpixel sampling maps, prediction mask, and more
        │       ├── NJU2K
        │       ├── NLPR
        │       ├── DES
        │       ├── SIP
        │       └── STERE
        └── train/
            └── config.py

An example of the resulting image is shown below.

<img align="center" src="./images/test.jpg"/>

The prediction mask results of our proposed model can be found here.

Citation

@article{lee2022spsn,
  title={SPSN: Superpixel Prototype Sampling Network for RGB-D Salient Object Detection},
  author={Lee, Minhyeok and Park, Chaewon and Cho, Suhwan and Lee, Sangyoun},
  journal={arXiv preprint arXiv:2207.07898},
  year={2022}
}