Awesome
<div align="center"> <h1>HTD-Mamba </h1> <h3>Efficient Hyperspectral Target Detection with Pyramid State Space Model</h3>Dunbin Shen<sup>1</sup> ,Xuanbing Zhu<sup>1</sup> ,Jiacheng Tian<sup>1</sup>, Jianjun Liu<sup>2</sup>, Zhenrong Du<sup>1</sup>, Hongyu Wang<sup>1</sup>, Xiaorui Ma<sup>1 :email:</sup>
<sup>1</sup> Dalian University of Technology, <sup>2</sup> Jiangnan University
(<sup>:email:</sup>) Corresponding author.
ArXiv Preprint (arXiv 2407.06841)
</div>News
July 18th, 2024
: We released our codes and models.️
Abstract
Hyperspectral target detection (HTD) identifies objects of interest from complex backgrounds at the pixel level, playing a vital role in Earth observation. However, HTD faces challenges due to limited prior knowledge and spectral variation, leading to underfitting models and unreliable performance. To address these challenges, this paper proposes an efficient self-supervised HTD method with a pyramid state space model (SSM), named HTD-Mamba, which employs spectrally contrastive learning to distinguish between target and background based on the similarity measurement of intrinsic features. Specifically, to obtain sufficient training samples and leverage spatial contextual information, we propose a spatial-encoded spectral augmentation technique that encodes all surrounding pixels within a patch into a transformed view of the center pixel. Additionally, to explore global band correlations, we divide pixels into continuous group-wise spectral embeddings and introduce Mamba to HTD for the first time to model long-range dependencies of the spectral sequence with linear complexity. Furthermore, to alleviate spectral variation and enhance robust representation, we propose a pyramid SSM as a backbone to capture and fuse multiresolution spectral-wise intrinsic features. Extensive experiments conducted on four public datasets demonstrate that the proposed method outperforms state-of-the-art methods in both quantitative and qualitative evaluations.
Overview
<div align="center"> <img src="assets/architecture.png" /> </div>Installation
-
CUDA 11.7
- Make sure
/usr/local/cuda-11.7
exists. If not, you can install it from NVIDIA DEVELOPER (CUDA Toolkit). For example, Ubuntu 18.04 x86_64wget https://developer.download.nvidia.com/compute/cuda/11.7.1/local_installers/cuda_11.7.1_515.65.01_linux.run
sudo sh cuda_11.7.1_515.65.01_linux.run
sudo apt install gcc-9 g++-9
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 100
gcc -v
- See
nvcc -V
iscuda_11.7
. If not, you should modify the.bashrc
like this:-
vim ~/.bashrc
->i
, and add the following to the endexport CUDA_HOME=/usr/local/cuda-11.7
export PATH=$PATH:/usr/bin:/bin
export PATH=/usr/local/cuda-11.7/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-11.7/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
-
esc
->:wq
->source ~/.bashrc
, and seenvcc -V
iscuda_11.7
.
-
- Make sure
-
Python 3.10.x
conda create -n htd-mamba python=3.10
conda activate htd-mamba
-
Torch 2.0.1
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.7 -c pytorch -c nvidia
orpip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu117
-
Requirements: requirements.txt
cd /home/your_path/HTD-Mamba-main
pip install -r requirements.txt
-
Install
selective_scan_cuda
cd /home/your_path/HTD-Mamba-main
pip install .
-
Install
causal_conv1d
pip install --upgrade pip
pip install causal_conv1d>=1.1.0
Evaluation
To evaluate HTD-Mamba
on four public datasets, run:
CL_Main_Sandiego.py
CL_Main_Sandiego2.py
CL_Main_LosAngeles.py
CL_Main_Pavia.py
Training
- To train
HTD-Mamba
, change the state totrain
. After training, you can use the stateselect_best
to select the best model from all epochs. - To train
HTD-Mamba
on a new dataset, update thedataset
,path
, andband
. For optimal performance, fine-tune parameters such aspatch_size
,m
(group length), andchannel
(embedding size).
Acknowledgement
This project is based on Mamba
(paper, code), Vim
(paper, code). We thank the authors for their promising studies.
Citation
If you find HTD-Mamba
useful in your research or applications, please consider giving us a star 🌟 and citing it using the following BibTeX entry.
@article{shen2024htd,
title={HTD-Mamba: Efficient Hyperspectral Target Detection with Pyramid State Space Model},
author={Shen, Dunbin and Zhu, Xuanbing and Tian, Jiacheng and Liu, Jianjun and Du, Zhenrong and Wang, Hongyu and Ma, Xiaorui},
journal={arXiv preprint arXiv:2407.06841},
year={2024}
}