Home

Awesome

PICR-Net_ACMMM2023

Runmin Cong, Hongyu Liu, Chen Zhang*, Wei Zhang, Feng Zheng, Ran Song, and Sam Kwong, Point-aware Interaction and CNN-induced Refinement Network for RGB-D salient object detection, ACM International Conference on Multimedia (ACM MM), 2023.

Network

Our overall framework:

image

Relation Modeling

image

Requirement

Pleasure configure the environment according to the given version:

We also provide ".yaml" files for conda environment configuration, you can download it from [Link], code: mvpl, then use conda env create -f requirement.yaml to create a required environment.

Data Preprocessing

Please follow the tips to download the processed datasets and pre-trained model:

Download RGB-D SOD dataset from [Link], code: mvpl.

Download pretrained backbone weights from [Link], code: mvpl.

├── RGBD_dataset
    ├── train
        ├── RGB
        ├── depth
        ├── GT
    ├── val
        ├── RGB
        ├── depth
        ├── GT
    ├── test
        ├── NJU2K
            ├── RGB
            ├── depth
            ├── GT
        ├── NLPR
            ├── RGB
            ├── depth
            ├── GT
        ...

├── pretrain
    ├── swin_tiny_patch4_window7_224.pth
    ├── vgg16_bn-6c64b313.pth

Training and Testing

Training command :

python train.py

Testing command :

The trained model for PICR-Net can be download here: [Link], code: mvpl.

python test.py

Evaluation

We implement three metrics: MAE (Mean Absolute Error), F-Measure, S-Measure. We use Toolkit [Link] to obtain the test metrics.

Results

  1. Qualitative results: we provide the saliency maps, you can download them from [Link], code: mvpl.
  2. Quantitative results:

image

Bibtex

   @inproceedings{PICR-Net,
     title={Point-aware Interaction and {CNN}-induced Refinement Network for {RGB-D} salient object detection},
     author={Cong, Runmin and Liu, Hongyu and Zhang, Chen and Zhang, Wei and Zheng, Feng and Song, Ran and Kwong, Sam },
     journal={ACM International Conference on Multimedia (ACM MM) },
     year={2023},
    }
  

Contact Us

If you have any questions, please contact Runmin Cong at rmcong@sdu.edu.cn or Hongyu Liu at liu.hongyu@bjtu.edu.cn.