Home

Awesome

A PyTorch Reproduction of HCN

Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation. Chao Li, Qiaoyong Zhong, Di Xie, Shiliang Pu, IJCAI 2018.

Arxiv Preprint

Features

1. Dataset

2. Tasks

3. Visualization

Prerequisites

Our code is based on Python3.5. There are a few dependencies to run the code in the following:

Usage

Data preparation

NTU RGB+D

To transform raw NTU RGB+D data into numpy array (memmap format ) by this command:

python ./feeder/ntu_gendata.py --data_path <path for raw skeleton dataset> --out_folder <path for new dataset>
Other Datasets

Not supported now.

Training

Before you start the training, you have to launch visdom server.

python -m visdom

To train the model, you should note that:

python main.py --dataset_dir <parents path for all the datasets> --mode train --model_name HCN --dataset_name NTU-RGB-D-CV --num 01

To run a new trial with different parameters, you need to:

Testing

python main.py --dataset_dir <parents path for all the datasets> --mode test --load True --model_name HCN --dataset_name NTU-RGB-D-CV --num 01

Load and Training

You also can load a half trained model, and start training it from a specific checkpoint by the following command:

python main.py --dataset_dir <parents path for all the datasets> --mode load_train --load True --model_name HCN --dataset_name NTU-RGB-D-CV --num 01 --load_model <path for  trained model>

Results

Table

The expected Top-1 accuracy of the model for NTU-RGD+D are shown here (There is an accuracy gap. I am not the author of original HCN paper, the repo was reproduced according to the paper text and have not been tuned carefully):

ModelNormalized <br> Sequence <br> LengthFC <br> Neuron <br> NumbersNTU RGB+D <br> Cross Subject (%)NTU RGB+D <br> Cross View (%)
HCN[1]3225686.591.1
HCN3225684.289.2
HCN6451284.9*90.9*

[1] http://arxiv.org/pdf/1804.06055.pdf

Figures

<div align="center"> <img src="resource/info/Loss.png" width="500" height="400"> <img src="resource/info/Accuracy.png"width="500" height="400"> </div>

Confusion matrix

<div align="center"> <img src="resource/info/ConfMatrix_train.png" width="400" height="400"> <img src="resource/info/ConfMatrix_test.png" width="400" height="400"> </div> - Loss & accuracy[CS] <div align="center"> <img src="resource/info/Loss2.png" width="500" height="400"> <img src="resource/info/Accuracy2.png"width="500" height="400"> </div>

Reference

[1] Chao Li, Qiaoyong Zhong, Di Xie, Shiliang Pu. Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation. IJCAI 2018.

[2] yysijie/st-gcn: referred for some code of dataset processing.