Awesome
Spatio-temporal Relation Modeling for Few-shot Action Recognition (CVPR 2022)
Anirudh Thatipelli, Sanath Narayan, Salman Khan, Rao Muhammad Anwer, Fahad Shahbaz Khan, Bernard Ghanem
Installation
The codebase is built on PyTorch 1.9.0 and tested on Ubuntu 18.04 environment (Python3.8.8, CUDA11.0) and trained on 4 GPUs. Build a conda environment using the requirements given in environment.yaml.
Attention Visualization
<img src = "https://imgur.com/CZym9q1.png" width="900">Results
| Method | Kinetics | SSv2 | HMDB | UCF |
|---|---|---|---|---|
| CMN-J | 78.9 | - | - | - |
| TARN | 78.5 | - | - | - |
| ARN | 82.4 | - | 60.6 | 83.1 |
| OTAM | 85.8 | 52.3 | - | - |
| HF-AR | - | 55.1 | 62.2 | 86.4 |
| TRX | 85.9 | 64.6 | 75.6 | 96.1 |
| STRM [Ours] | 86.7 | 68.1 | 77.3 | 96.8 |
Training and Evaluation
Step 1 : Data preparation
Prepare the datasets according to the splits provided.
Step 2 : Training
Use the scripts given in scripts
Step 3 : Evaluation
-
Use the evaluation script as given in eval_strm_ssv2.sh
-
Download the checkpoints from these links: SSV2, Kinetics, HMDB, UCF
Citation
If you find this repository useful, please consider giving a star :star: and citation :confetti_ball::
@inproceedings{thatipelli2021spatio,
title={Spatio-temporal Relation Modeling for Few-shot Action Recognition},
author={Thatipelli, Anirudh and Narayan, Sanath and Khan, Salman and Anwer, Rao Muhammad and Khan, Fahad Shahbaz and Ghanem, Bernard},
booktitle={CVPR},
year={2022}
}
Acknowledgements
The codebase was built on top of trx. Many thanks to Toby Perrett for previous work.
Contact
Should you have any question, please contact :e-mail: thatipellianirudh@gmail.com or message me on Linkedin.