Awesome

FusionMamba

Code for the paper: "FusionMamba: Efficient Remote Sensing Image Fusion with State Space Model", TGRS, 2024.
First application of the state space model (SSM) in the hyper-spectral pansharpening and hyper-spectral image super-resolution (HISR) tasks.
State-of-the-art (SOTA) performance in pansharpening, hyper-spectral pansharpening, and HISR tasks.

Paper

For a detailed understanding of our method, please refer to the paper: FusionMamba: Efficient Remote Sensing Image Fusion with State Space Model.
This paper has been published in the IEEE Transactions on Geoscience and Remote Sensing (TGRS).

Method

FusionMamba Block

FusionMamba

We expand the single-input Mamba block to accommodate dual inputs, creating the FusionMamba block, which can serve as a plug-and-play solution for information integration.

Experimental Results

Pansharpening

HPansharpening

Get Started

Dataset

Datasets for pansharpening: PanCollection. We recommend downloading datasets in the h5py format. The testing toolbox can be found here.
Datasets for hyper-spectral pansharpening: HyperPanCollection. We recommend downloading datasets in the h5py format.
Dataset for HISR: the CAVE dataset. You can find this dataset on the Internet.

Installation

Clone the repository:

git clone https://github.com/PSRben/FusionMamba.git

Install the Mamba implementation by following the instructions in the Mamba-block directory.
Install other packages:

pip install einops h5py opencv-python torchinfo scipy numpy

Usage

This repository is only for the pansharpening task.
The model weights trained on the WV3 dataset for 400 epochs can be found in the weights directory.

# train
python train.py --train_data_path ./path_to_data/train_WV3.h5 --val_data_path ./path_to_data/valid_WV3.h5
# test
python test.py --file_path ./path_to_data/name.h5 --save_dir ./path_to_dir --weight ./weights/epochs.pth

Citation

@ARTICLE{10750233,
  author={Peng, Siran and Zhu, Xiangyu and Deng, Haoyu and Deng, Liang-Jian and Lei, Zhen},
  journal={IEEE Transactions on Geoscience and Remote Sensing}, 
  title={FusionMamba: Efficient Remote Sensing Image Fusion With State Space Model}, 
  year={2024},
  volume={62},
  number={},
  pages={1-16},
  doi={10.1109/TGRS.2024.3496073}}

Contact

We are glad to hear from you. If you have any questions, please feel free to contact siran_peng@163.com.