Awesome
RGB-D Salient Object Detection with Cross-Modality Modulation and Selection
Arxiv version: https://arxiv.org/abs/2007.07051
TensorFlow
TensorFlow implementation of cmMS for RGBD salient object detection
Requirements
Python 3
TensorFlow 1.x
Folder structure
The following shows the basic folder structure.
├── checkpoint
│ ├── coarse_224 # A pre-trained checkpoint (coarse.model-2850)
│ │ ├── checkpoint
│ │ ├── coarse.model-2850.data-00000-of-00001
│ │ ├── coarse.model-2850.index
│ │ ├── coarse.model-2850.meta
├── vgg_pretrained # vgg pretrained model
│ ├── imagenet-vgg-verydeep-16
│ ├── imagenet-vgg-verydeep-19
│ ├── vgg16.npy
│ ├── vgg19.npy
├── main_test.py # testing code
├── model.py # network
├── test_real. # put RGB images here
├── depth_real. # put depth images here
├── ops.py.py
├── utils.py
├── vgg.py
results
We rename the images, so the name of our result is different from the original data. For your evaluations, we also provide the corresponding renamed GT.
Google Drive: https://drive.google.com/file/d/1uu6Y_IDH6ukdkBkGN9zfYv4K-IYLc9aa/view?usp=sharing
Baidu Cloud: https://pan.baidu.com/s/1eXmx0Tm3K5rEn7OlyPDLOg Password: 1234
Preporcessing
- download the pretrained VGG model
Google Drive: https://drive.google.com/file/d/1IDzr2OqoQk2LdecWRoReGnoIsPYYSL-J/view?usp=sharing
Baidu Cloud: https://pan.baidu.com/s/1obO2IWLlkfVdXDj7gT5ogg Password: 1234
- download the checkpoint of our model, unzip, and put it to 'checkpoint/coarse_224' folder
Google Drive: https://drive.google.com/file/d/1YsQ4XBe1J3cho7BDM2hW85PpHX5m3-Yx/view?usp=sharing
Baidu Cloud: https://pan.baidu.com/s/1txRl_-xNctC6x3mZAwyRbQ Password: 1234
-
normalize the depth maps (note that the foreground should have higher value than the background in our method) input=(input-min(min(input)))/(max(max(input))-min(min(input))) The step is very important for accurate results.
-
resize the testing data to the size of 224*224 first normalize depth then resize will be better than first resize depth then normalize in our method. So please strictly follow our steps to generate testing data.
-
put your rgb images to 'test_real' folder and your depth maps to 'depth_real' folder (paired rgb image and depth map should have same name)
Test
python main_test.py
find the results in the 'test_real' folder with the same name as the input image + "_out".
You can use a script to resize the results back to the same size as the original RGB-D image, or just use the results with a size of 224*224 for evaluations. We did not find much differences for the evaluation results.
Bibtex
@inproceedings{cmMS,
author = {Li, Chongyi and Cong, Runmin and Piao Yongri and Xu Qianqian, and Loy, Chen Change},
title = {RGB-D salient object eetection with cross-modality modulation and selection},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
pages = {},
month = {},
year = {2020}
}
Contact
If you have any questions, please contact Chongyi Li at lichongyi25@gmail.com.