Awesome
Contrastive Masked Autoencoders for Self-Supervised Video Hashing
This repository is the official PyTorch implementation of our AAAI 2023 paper Contrastive Masked Autoencoders for Self-Supervised Video Hashing.
Catalogue <br>
Getting Started
1. Clone this repository:
git clone https://github.com/haungmozhi9527/ConMH.git
cd ConMH
2. Create a conda environment and install the dependencies:
conda create -n conmh python=3.6
conda activate conmh
conda install pytorch==1.6.0 cudatoolkit=10.1 -c pytorch -c conda-forge
pip install -r requirements.txt
3. Download Datasets: VGG features of FCVID and YFCC are kindly uploaded by the authors of SSVH. ResNet50 features of ActivityNet are kindly provided by the authors of BTH. You can download them from Baiduyun disk.
Dataset | Link |
---|---|
FCVID | Baidu disk |
ActivityNet | Baidu disk |
YFCC | Baidu disk |
4. Set data_root and home_root in config files (e.g., ./configs/conmh_fcv.py).
Train
To train ConMH on FCVID:
python train.py --gpu 0 --config configs/conmh_fcv.py
To train ConMH on ActivityNet:
python train.py --gpu 0 --config configs/conmh_act.py
To train ConMH on YFCC:
python train.py --gpu 0 --config configs/conmh_yfcc.py
Test
To test ConMH on FCVID:
python eval.py --gpu 0 --config configs/conmh_fcv.py
To test ConMH on ActivityNet:
python eval.py --gpu 0 --config configs/conmh_act.py
To test ConMH on YFCC:
python eval.py --gpu 0 --config configs/conmh_yfcc.py
Trained Models
We provide trained ConMH checkpoints. You can download them from Baiduyun disk.
Dataset | 16 bits | 32 bits | 64 bits |
---|---|---|---|
FCVID | Baidu disk | Baidu disk | Baidu disk |
ActivityNet | Baidu disk | Baidu disk | Baidu disk |
YFCC | Baidu disk | Baidu disk | Baidu disk |
Results
Quantitative Results
For this repository, the expected performance is:
Dataset | Bits | mAP@5 | mAP@20 | mAP@40 | mAP@60 | mAP@80 | mAP@100 |
---|---|---|---|---|---|---|---|
FCVID | 16 | 0.350 | 0.252 | 0.216 | 0.196 | 0.181 | 0.169 |
FCVID | 32 | 0.476 | 0.332 | 0.287 | 0.263 | 0.245 | 0.230 |
FCVID | 64 | 0.524 | 0.373 | 0.326 | 0.301 | 0.283 | 0.267 |
ActivityNet | 16 | 0.156 | 0.081 | 0.050 | 0.036 | 0.029 | 0.024 |
ActivityNet | 32 | 0.229 | 0.124 | 0.075 | 0.054 | 0.042 | 0.035 |
ActivityNet | 64 | 0.267 | 0.150 | 0.092 | 0.066 | 0.051 | 0.042 |
YFCC | 16 | 0.225 | 0.146 | 0.122 | 0.113 | 0.108 | 0.104 |
YFCC | 32 | 0.341 | 0.182 | 0.148 | 0.135 | 0.128 | 0.123 |
YFCC | 64 | 0.368 | 0.194 | 0.158 | 0.143 | 0.135 | 0.130 |
Citation
If you find this repository useful, please consider citing our work:
@inproceedings{wang2023contrastive,
title={Contrastive Masked Autoencoders for Self-Supervised Video Hashing},
author={Wang, Yuting and Wang, Jinpeng and Chen, Bin and Zeng, Ziyun and Xia, Shu-Tao},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
volume={37},
number={3},
pages={2733--2741},
year={2023}
}