Awesome
ReAct: Temporal Action Detection with Relational Queries
This repo holds the code for React, which is accept to ECCV2022. If you have any question, welcome to contact at "shidingfeng at buaa . edu. cn".
Installation
We build our code based on the MMaction2 project (1.3.10 version). See here for more details if you are interested. MMCV is needed before install MMaction2, which can be install with:
pip install mmcv-full-f https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html
# For example, to install the latest mmcv-full with CUDA 11.1 and PyTorch 1.9.0, use the following command:
pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.html
For other CUDA or pytorch version, please refer here to get a matched link.
Then, our code can be built by
git clone https://github.com/sssste/React.git
cd React
pip3 install -e .
Then, Install the 1D Grid Sampling and RoI Align operators.
cd React/model
python setup.py build_ext --inplace
Data preparing
We used the TSN feature (Google Drive Link) provied by G-TAD for our model. Please put all the files in the datasets/thumos14/
fold (or you can put them in any place and modify the data path in the config file in React/configs/thumos_tsn_feature.py
)
Training
Our model can be trained with
python tools/train.py React/configs/thumos_tsn_feature.py --validate
We recommend to set the --validate
flag to monitor the training process.
Test
If you want to test the pretrained model, please use the following code.
python tools/test.py React/configs/thumos_tsn_feature.py PATH_TO_MODEL_PARAMETER_FILE
We provide the pretrained weights for React (THUMOS14) . Our code supports test with a batch of videos for efficient. If you want to change the batch size, you can change the number of workers_per_gpu
in thumos_tsn_feature.py
.
Then, you can run the test by
python tools/test.py React/configs/thumos_tsn_feature.py react_thumos_pretrained_weight.pth
The results (mAP at tIoUs, %) should be
Method | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | Avg |
---|---|---|---|---|---|---|
React | 70.8 | 65.9 | 57.8 | 47.2 | 34.2 | 55.2 |
Citation
If you feel this work useful, please cite our paper! Thank you!
@inproceedings{shi2022react,
title = {ReAct: Temporal Action Detection with Relational Queries},
author = {Shi, Dingfeng and Zhong, Yujie and Cao, Qiong and Zhang, Jing and Ma, Lin and Li, Jia and Tao, Dacheng},
year={2022},
booktitle = {European conference on computer vision}
}