Awesome
BMDENet
We proposed a bi-directional modality difference elimination network (BMDENet) to boost the segmentation performance. Concretely, the bi-directional modality difference elimination module (BMDEM) reduces the heterogeneity between RGB and thermal images in the prototype space. The residual attention fusion module (RAFM) mines the bimodal features to fully fuse the cross-modal information. In addition, the mainstay and subsidiary enhancement module (MSEM) enhances the fusion features for the existing problem of the advanced model. Extensive experiments on Tokyo Multi-Spectral-4i dataset prove that BMDENet achieves the state-of-the-art on both 1- and 5-shot settings.
Download the dataset and code
The dataset and code are available at:https://pan.baidu.com/s/1-hEbM0sRSAfpWH1QLk3kJQ?pwd=ofk3
Paper
https://ieeexplore.ieee.org/abstract/document/10130632
Citation
Y. Zhao, K. Song, Y. Zhang and Y. Yan, "BMDENet: Bi-directional Modality Difference Elimination Network for Few-shot RGB-T Semantic Segmentation," in IEEE Transactions on Circuits and Systems II: Express Briefs, doi: 10.1109/TCSII.2023.3278941.
Related Work of RGB-D/T Few-shot Semantic Segmentation
[1] Self-Enhanced Mixed Attention Network for Three-Modal Images Few-Shot Semantic Segmentation. Sensors 2023, 23, 6612. https://github.com/VDT-2048/SEMA
[2] Visible and Thermal Images Fusion Architecture for Few-shot Semantic Segmentation [J]. Journal of Visual Communication and Image Representation, 2021, 80, 103306. https://github.com/bbbbby-99/V-T-few-shot-semantic-segmentation
Related Work of Visible-Depth-Thermal Salient Object Detection
[1] A Novel Visible-Depth-Thermal Image Dataset of Salient Object Detection for Robotic Visual Perception [J]. IEEE/ASME Transactions on Mechatronics, 2023, 28(3), 1558-1569. https://github.com/VDT-2048/VDT-Dataset
[2] Lightweight Multi-level Feature Difference Fusion Network for RGB-D-T Salient Object Detection [J]. Journal of King Saud University - Computer and Information Sciences, 2023 https://github.com/VDT-2048/MFDF
[3] MFFNet: Multi-modal Feature Fusion Network for VDT Salient Object Detection[J]. IEEE Transactions on Multimedia, 2023. https://ieeexplore.ieee.org/abstract/document/10171982
Related Work of RGB-T Salient Object Detection
[1] Multiple Graph Affinity Interactive Network and A Variable Illumination Dataset for RGBT Image Salient Object Detection [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(7), 3104-3118. https://github.com/huanglm-me/VI-RGBT1500
[2] Thermal images-aware guided early fusion network for cross-illumination RGB-T salient object detection[J]. Engineering Applications of Artificial Intelligence, 2023, 118: 105640. https://github.com/VDT-2048/TAGFNet
Related Survey
RGB-T Image Analysis Technology and Application: A Survey [J]. Engineering Applications of Artificial Intelligence, 2023, 120, 105919. https://www.sciencedirect.com/science/article/abs/pii/S0952197623001033