Home

Awesome

Causal inference framework for Weakly supervised temporal action localization

Our paper The Blessings of Unlabeled Background in Untrimmed Videos has been accepted by CVPR 2021.

Due to the excellent performance of WUM, the released code is TS-PCA+WUM on the THUMOS dataset.

Recommended Environment

Data Preparation

├── dataset
   └── THUMOS14
       ├── gt.json
       ├── split_train.txt
       ├── split_test.txt
       └── features
           ├── train
               ├── rgb
                   ├── video_validation_0000051.npy
                   ├── video_validation_0000052.npy
                   └── ...
               └── flow
                   ├── video_validation_0000051.npy
                   ├── video_validation_0000052.npy
                   └── ...
           └── test
               ├── rgb
                   ├── video_test_0000004.npy
                   ├── video_test_0000006.npy
                   └── ...
               └── flow
                   ├── video_test_0000004.npy
                   ├── video_test_0000006.npy
                   └── ...
├── WTAL_result_numpy
   ├── video-id_cas.npy
   ├── video-id_feat_act.npy
   ├── video-id_feat_bkg.npy
   ├── video-id_features.npy
   ├── video-id_score.npy
   └── ...

Usage

Running

You could optionally specify your preferred parameters with options.py.

Train and evaluate the TS-PCA confounder:

$ sh run.sh

Evaulation

Test with a pretrained model:

$ sh run_eval.sh

References

We referenced the repos below for the code.

Citation

Please cite the following paper if you feel our paper useful to your research.

@inproceedings{wtal_blessing,
  author    = {Yuan Liu and
               Jingyuan Chen and
               Zhenfang Chen and
               Bing Deng and
               Jianqiang Huang and 
               Hanwang Zhang},
  title     = {The Blessings of Unlabeled Background in Untrimmed Videos},
  booktitle = {CVPR},
  year      = {2021},
}