Awesome

Efficient Self-Supervised Video Hashing with Selective State Spaces

[toc]

1. Introduction

This repository contains the PyTorch implementation of our work at AAAI 2025:

Efficient Self-Supervised Video Hashing with Selective State Spaces. Jinpeng Wang, Niu Lian, Jun Li, Yuting Wang, Yan Feng, Bin Chen, Yongbing Zhang, Shu-Tao Xia.

We are happy to announce S5VH, the first Mamba-based video hashing model with an improved self-supervised learning paradigm. S5VH includes bidirectional Mamba layers for both the encoder and decoder, which are effective and efficient in capturing temporal relationships thanks to the data-dependent selective scanning mechanism with linear complexity. On hash learning strategy, we transform global semantics in the feature space into semantically consistent and *discriminative hash centers, followed by a center alignment loss as a global learning signal. Experiments show S5VH’s efficacy and efficiency under various setups. Our study suggests the strong potential of state-space models in video hashing, which we hope can inspire further research.

In the following, we will guide you how to use this repository step by step. 🤗

2. Preparation

git clone https://github.com/gimpong/AAAI25-S5VH.git
cd AAAI25-S5VH/

2.1 Requirements

• python==3.11.8
• numpy==1.26.4
• pytorch==2.0.1
• torchvision==0.15.2
• mamba-ssm==2.0.4
• scipy==1.5.4
• h5py==3.1.0
• addict==2.4.0

2.2 Download the video feature datasets and organize them properly

Before running the code, make sure that everything is ready. The working directory is expected to be organized as below:

You may downloaded video features from the following Baidu Cloud links and put them into dataset-specific folder under the data/ folder.

2.3 Pre-processing: hash center generation

Before model training, please make sure the hash centers have been generated. You may download our preprocessed version of the hash center files from the Baidu Cloud links (see the table above) and put them to the video feature folders of specific datasets. Otherwise, you can generate these files by re-executing the pre-processing code. For example, on ActivityNet:

python preprocess.py --gpu 0 --config configs/S5VH/act.py

To modify the configurations for different datasets, you can replace act.py with fcv.py, ucf.py or hmdb.py.

2.4 Train

The training command is as follows:

python train.py --config configs/<MODEL_NAME>/<DATASET_NAME>.py --gpu <GPU_ID>

Options:

• <MODEL_NAME>: S5VH, LSTM, RetNet, RWKV
• <DATASET_NAME>: act, fcv, ucf, hmdb
• <GPU_ID>: specify the gpu id

The logs, model checkpoints will be generated under the logs/ and checkpoint/ folders, respectively.

2.5 Test

We provide the evaluation code for model checkpoints (if exist). The test command is as follows:

python eval.py --configs/<MODEL_NAME>/<DATASET_NAME>.py --gpu <GPU_ID>

3. Results

4. References

If you find our code useful or use the toolkit in your work, please consider citing:

@inproceedings{Wang25_S5VH,
  author={Wang, Jinpeng and Lian, Niu and Li, Jun and Wang, Yuting and Feng, Yan and Chen, Bin and Zhang, Yongbing and Xia, Shu-Tao},
  title={Efficient Self-Supervised Video Hashing with Selective State Spaces},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  year={2025}
}

5. Acknowledgements

This code is based on our previous work ConMH at AAAI'23. We are also grateful for other teams for open-sourcing codes that inspire our work, including SSVH, BTH, MCMSH, BerVAE, DKPH, and SHC-IR.

6. Contact

If you have any question, you can raise an issue or email Jinpeng Wang (wjp20@mails.tsinghua.edu.cn). We will reply you soon.