Home

Awesome

Contrastive Masked Autoencoders for Self-Supervised Video Hashing

This repository is the official PyTorch implementation of our AAAI 2023 paper Contrastive Masked Autoencoders for Self-Supervised Video Hashing.

Catalogue <br>

Getting Started

1. Clone this repository:

git clone https://github.com/haungmozhi9527/ConMH.git
cd ConMH

2. Create a conda environment and install the dependencies:

conda create -n conmh python=3.6
conda activate conmh
conda install pytorch==1.6.0 cudatoolkit=10.1 -c pytorch -c conda-forge
pip install -r requirements.txt

3. Download Datasets: VGG features of FCVID and YFCC are kindly uploaded by the authors of SSVH. ResNet50 features of ActivityNet are kindly provided by the authors of BTH. You can download them from Baiduyun disk.

DatasetLink
FCVIDBaidu disk
ActivityNetBaidu disk
YFCCBaidu disk

4. Set data_root and home_root in config files (e.g., ./configs/conmh_fcv.py).

Train

To train ConMH on FCVID:

python train.py --gpu 0 --config configs/conmh_fcv.py

To train ConMH on ActivityNet:

python train.py --gpu 0 --config configs/conmh_act.py

To train ConMH on YFCC:

python train.py --gpu 0 --config configs/conmh_yfcc.py

Test

To test ConMH on FCVID:

python eval.py --gpu 0 --config configs/conmh_fcv.py

To test ConMH on ActivityNet:

python eval.py --gpu 0 --config configs/conmh_act.py

To test ConMH on YFCC:

python eval.py --gpu 0 --config configs/conmh_yfcc.py

Trained Models

We provide trained ConMH checkpoints. You can download them from Baiduyun disk.

Dataset16 bits32 bits64 bits
FCVIDBaidu diskBaidu diskBaidu disk
ActivityNetBaidu diskBaidu diskBaidu disk
YFCCBaidu diskBaidu diskBaidu disk

Results

Quantitative Results

For this repository, the expected performance is:

DatasetBitsmAP@5mAP@20mAP@40mAP@60mAP@80mAP@100
FCVID160.3500.2520.2160.1960.1810.169
FCVID320.4760.3320.2870.2630.2450.230
FCVID640.5240.3730.3260.3010.2830.267
ActivityNet160.1560.0810.0500.0360.0290.024
ActivityNet320.2290.1240.0750.0540.0420.035
ActivityNet640.2670.1500.0920.0660.0510.042
YFCC160.2250.1460.1220.1130.1080.104
YFCC320.3410.1820.1480.1350.1280.123
YFCC640.3680.1940.1580.1430.1350.130

Citation

If you find this repository useful, please consider citing our work:

@inproceedings{wang2023contrastive,
  title={Contrastive Masked Autoencoders for Self-Supervised Video Hashing},
  author={Wang, Yuting and Wang, Jinpeng and Chen, Bin and Zeng, Ziyun and Xia, Shu-Tao},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={37},
  number={3},
  pages={2733--2741},
  year={2023}
}