Awesome
RSKP
Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation (CVPR 2022)<br> Linjiang Huang (CUHK), Liang Wang (CASIA), Hongsheng Li (CUHK)
Overview
The experimental results on THUMOS14 are as below.
| Method \ mAP(%) | @0.1 | @0.2 | @0.3 | @0.4 | @0.5 | @0.6 | @0.7 | AVG |
|---|---|---|---|---|---|---|---|---|
| UntrimmedNet | 44.4 | 37.7 | 28.2 | 21.1 | 13.7 | - | - | - |
| STPN | 52.0 | 44.7 | 35.5 | 25.8 | 16.9 | 9.9 | 4.3 | 27.0 |
| W-TALC | 55.2 | 49.6 | 40.1 | 31.1 | 22.8 | - | 7.6 | - |
| AutoLoc | - | - | 35.8 | 29.0 | 21.2 | 13.4 | 5.8 | - |
| CleanNet | - | - | 37.0 | 30.9 | 23.9 | 13.9 | 7.1 | - |
| MAAN | 59.8 | 50.8 | 41.1 | 30.6 | 20.3 | 12.0 | 6.9 | 31.6 |
| CMCS | 57.4 | 50.8 | 41.2 | 32.1 | 23.1 | 15.0 | 7.0 | 32.4 |
| BM | 60.4 | 56.0 | 46.6 | 37.5 | 26.8 | 17.6 | 9.0 | 36.3 |
| RPN | 62.3 | 57.0 | 48.2 | 37.2 | 27.9 | 16.7 | 8.1 | 36.8 |
| DGAM | 60.0 | 54.2 | 46.8 | 38.2 | 28.8 | 19.8 | 11.4 | 37.0 |
| TSCN | 63.4 | 57.6 | 47.8 | 37.7 | 28.7 | 19.4 | 10.2 | 37.8 |
| EM-MIL | 59.1 | 52.7 | 45.5 | 36.8 | 30.5 | 22.7 | 16.4 | 37.7 |
| BaS-Net | 58.2 | 52.3 | 44.6 | 36.0 | 27.0 | 18.6 | 10.4 | 35.3 |
| A2CL-PT | 61.2 | 56.1 | 48.1 | 39.0 | 30.1 | 19.2 | 10.6 | 37.8 |
| ACM-BANet | 64.6 | 57.7 | 48.9 | 40.9 | 32.3 | 21.9 | 13.5 | 39.9 |
| HAM-Net | 65.4 | 59.0 | 50.3 | 41.1 | 31.0 | 20.7 | 11.1 | 39.8 |
| ACSNet | - | - | 51.4 | 42.7 | 32.4 | 22.0 | 11.7 | - |
| WUM | 67.5 | 61.2 | 52.3 | 43.4 | 33.7 | 22.9 | 12.1 | 41.9 |
| AUMN | 66.2 | 61.9 | 54.9 | 44.4 | 33.3 | 20.5 | 9.0 | 41.5 |
| CoLA | 66.2 | 59.5 | 51.5 | 41.9 | 32.2 | 22.0 | 13.1 | 40.9 |
| ASL | 67.0 | - | 51.8 | - | 31.1 | - | 11.4 | - |
| TS-PCA | 67.6 | 61.1 | 53.4 | 43.4 | 34.3 | 24.7 | 13.7 | 42.6 |
| UGCT | 69.2 | 62.9 | 55.5 | 46.5 | 35.9 | 23.8 | 11.4 | 43.6 |
| CO2-Net | 70.1 | 63.6 | 54.5 | 45.7 | 38.3 | 26.4 | 13.4 | 44.6 |
| D2-Net | 65.7 | 60.2 | 52.3 | 43.4 | 36.0 | - | - | - |
| FAC-Net | 67.6 | 62.1 | 52.6 | 44.3 | 33.4 | 22.5 | 12.7 | 42.2 |
| Ours | 71.3 | 65.3 | 55.8 | 47.6 | 38.2 | 25.4 | 12.5 | 45.1 |
Prerequisites
Recommended Environment
- Python 3.6
- Pytorch 1.5
- Tensorboard Logger
- CUDA 10.1
Note: Our code works with different PyTorch and CUDA versions, for high version of Pytorch, you need to change one line of our code according to this issue.
Data Preparation
-
Prepare THUMOS'14 dataset.
- We recommend using features and annotations provided by this repo.
-
Place the features and annotations inside a
dataset/Thumos14reduced/folder.
Usage
Training
You can easily train the model by running the provided script.
-
Refer to
options.py. Modify the argument ofdataset-rootto the path of yourdatasetfolder. -
Run the command below.
$ python main.py --run-type 0 --model-id 1
Models are saved in ./ckpt/dataset_name/model_id/
Evaulation
The trained model can be found here. (This saved model's result is slightly different from the one reported in our paper.)
Please put it into ./ckpt/dataset_name/model_id/.
- Run the command below.
$ python main.py --pretrained --run-type 1 --model-id 1 --load-epoch xxx
Please refer to the log in the same folder of saved models to set the load epoch of the best model.
Make sure you set the right model-id that corresponds to the model-id during training.
References
We referenced the repos below for the code.
Citation
@InProceedings{rskp,
title={Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation},
author={Huang, Linjiang and Wang, Liang and Li, Hongsheng},
booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2022}
}
Contact
If you have any question or comment, please contact the first author of the paper - Linjiang Huang (ljhuang524@gmail.com).