Awesome

LSNet

This project provides the code and results for 'LSNet: Lightweight Spatial Boosting Network for Detecting Salient Objects in RGB-Thermal Images', IEEE TIP, 2023. IEEE link <br>

Requirements

Python 3.7+, Pytorch 1.5.0+, Cuda 10.2+, TensorboardX 2.1, opencv-python <br> If anything goes wrong with the environment, please check requirements.txt for details.

Architecture and Details

<img src="https://user-images.githubusercontent.com/38373305/218299628-8b7bbdc5-39b2-4d68-9cdb-828e617c0bab.png" alt="drawing" width="400" height="400"/> <img src="https://user-images.githubusercontent.com/38373305/218299686-8a7e7cae-8970-4e56-a4b1-4986b872741f.png" alt="drawing" width="400" height="400"/>

Results

Data Preparation

• Download the RGB-T raw data from baidu, pin: sf9y / Google drive <br>
• Download the RGB-D raw data from baidu, pin: 7pi5 / Google drive <br>

Note that the depth maps of the raw data above are foreground is white.

Training & Testing

modify the train_root train_root save_path path in config.py according to your own data path.

• Train the LSNet:

  python train.py

modify the test_path path in config.py according to your own data path.

• Test the LSNet:

  python test.py

Note that task in config.py determines which task and dataset to use.

Evaluate tools

• You can select one of toolboxes to get the metrics CODToolbox / PySODMetrics

Saliency Maps

• RGB-T baidu pin: fxsk / Google drive<br>
• RGB-D baidu pin: 6352 / Google drive<br>

Note that we resize the testing data to the size of 224 * 224 for quicky evaluate. <br> please check our previous works APNet and CCAFNet.

Pretraining Models

• RGB-T baidu pin: wnoa / Google drive <br>
• RGB-D baidu pin: wnoa / Google drive <br>

Citation

    @ARTICLE{Zhou_2023_LSNet,
                author={Zhou, Wujie and Zhu, Yun and Lei, Jingsheng and Yang, Rongwang and Yu, Lu},
                journal={IEEE Transactions on Image Processing}, 
                title={LSNet: Lightweight Spatial Boosting Network for Detecting Salient Objects in RGB-Thermal Images}, 
                year={2023},
                volume={32},
                number={},
                pages={1329-1340},
                doi={10.1109/TIP.2023.3242775}}      
                

Acknowledgement

The implement of this project is based on the codebases bellow. <br>

• BBS-Net <br>
• Knowledge-Distillation-Zoo <br>
• Fps/speed test MobileSal
• Evaluate tools CODToolbox / PySODMetrics<br>

If you find this project helpful, Please also cite codebases above.

Contact

Please drop me an email for any problems or discussion: https://wujiezhou.github.io/ (wujiezhou@163.com) or zzzyylink@gmail.com.