Home

Awesome

[CVPR2023] TriDet: Temporal Action Detection with Relative Boundary Modeling

PWC PWC PWC PWC

Image Title

Overview

This repository contains the code for TriDet: Temporal Action Detection with Relative Boundary Modeling paper, which has been accepted for CVPR2023. Our code is built upon the codebase from ActionFormer and Detectron2, and we would like to express our gratitude for their outstanding work.

To quickly get start with the model architecture, you can focus mainly on the following files:

Update Log

Installation

  1. Please ensure that you have installed PyTorch and CUDA. (This code requires PyTorch version >= 1.11. We use version=1.11.0 in our experiments)

  2. Install the required packages by running the following command:

pip install  -r requirements.txt
  1. Install NMS
cd ./libs/utils
python setup.py install --user
cd ../..
  1. Done! We are ready to get start!

Data Preparation

Quick Start

We provide a list of scripts that allow you to reproduce our results with just one click. These scripts are located in the ./tools folder and include:

To easily reproduce our results, simply run the following command:

bash SCRIPT_PATH GPU_NUM

For example, if you want to train and eval our model on THUMOS14 dataset using the first GPU on you machine, you can run:

bash tools/thumos_i3d_script.sh 0

The mean average precision (mAP) results for each dataset are:

Dataset0.30.40.50.60.7Avg
THUMOS1483.6280.0772.9462.3547.3569.27
Dataset0.10.20.30.40.5Avg
EPIC-KITCHEN-noun27.3826.2824.6022.2318.2823.76
Dataset0.10.20.30.40.5Avg
EPIC-KITCHEN-verb28.7227.5726.1924.2620.8325.51
Dataset0.50.750.95Avg
HACS56.9039.3311.2438.69
Dataset0.50.750.95Avg
ActivityNet54.7138.018.3536.77

*There has been a slight improvement in the results of some datasets compared to those reported in the paper. *Note: We conduct all our experiments on a single A100-40G GPU and the training results may vary depending on the type of GPU used.

Test

We offer pre-trained models for each dataset, which you can download the chechpoints from Google Drive. The command for test is

python eval.py ./configs/CONFIG_FILE PATH_TO_CHECKPOINT

Contact

If you have any questions about the code, feel free to contact shidingfeng at buaa dot edu dot cn.

References

If you find this work helpful, please consider citing our paper

@inproceedings{shi2023tridet,
  title={TriDet: Temporal Action Detection with Relative Boundary Modeling},
  author={Shi, Dingfeng and Zhong, Yujie and Cao, Qiong and Ma, Lin and Li, Jia and Tao, Dacheng},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={18857--18866},
  year={2023}
}