Awesome

Learning Trajectory Dependencies for Human Motion Prediction

This is the code for the paper

Wei Mao, Miaomiao Liu, Mathieu Salzmann, Hongdong Li. Learning Trajectory Dependencies for Human Motion Prediction. In ICCV 19.

Dependencies

cuda 9.0
Python 3.6
Pytorch 0.3.1.
progress 1.5

Get the data

Human3.6m in exponential map can be downloaded from here.

UPDATE 2023-09: It seems the above link does not work any more. Please try to download the dataset from here. Please follow the license of the dataset.

CMU mocap was obtained from the repo of ConvSeq2Seq paper.

3DPW from their official website.

Quick demo and visualization

For a quick demo, you can train for a few epochs and visualize the outputs of your model.

To train, run

python main.py --epoch 5 --input_n 10 --output 10 --dct_n 20 --data_dir [Path To Your H36M data]/h3.6m/dataset/

Visualize the results of pretrained model for predictions on angle space on H36M dataset.

change the model path
then run the command below

python demo.py --input_n 10 --output_n 10 --dct_n 20 --data_dir [Path To Your H36M data]/h3.6m/dataset/

Training commands

All the running args are defined in opt.py. We use following commands to train on different datasets and representations. To train on angle space,

python main.py --data_dir "[Path To Your H36M data]/h3.6m/dataset/" --input_n 10 --output_n 10 --dct_n 20 --exp [where to save the log file]

python main_cmu.py --data_dir_cmu "[Path To Your CMU data]/cmu_mocap/" --input_n 10 --output_n 25 --dct_n 35 --exp [where to save the log file]

python main_3dpw.py --data_dir_3dpw "[Path To Your 3DPW data]/3DPW/sequenceFiles/" --input_n 10 --output_n 30 --dct_n 40 --exp [where to save the log file]

To train on 3D space,

python3 main_3d.py --data_dir "[Path To Your H36M data]/h3.6m/dataset/" --input_n 10 --output_n 10 --dct_n 15 --exp [where to save the log file]

python main_cmu_3d.py --data_dir_cmu "[Path To Your CMU data]/cmu_mocap/" --input_n 10 --output_n 25 --dct_n 30 --exp [where to save the log file]

python main_3dpw_3d.py --data_dir_3dpw "[Path To Your 3DPW data]/3DPW/sequenceFiles/" --input_n 10 --output_n 30 --dct_n 35 --exp [where to save the log file]

Results

We re-run our code 2 more times under different setups and the overall average results at different time are reported below.

Human3.6-short-term prediction on angle space (top) and 3D coordinate (bottom)

	80ms	160ms	320ms	400ms
pre-trained	0.27	0.51	0.83	0.95
test_run_1	0.28	0.52	0.84	0.96
test_run_2	0.28	0.52	0.84	0.96
----------------	------	------	------	------
pre-trained	12.1	25.0	51.0	61.3
test_run_1	12.1	24.6	50.4	61.1
test_run_2	12.1	24.8	50.5	61.2

Human3.6-long-term prediction

	560ms	1000ms
pre-trained	0.90	1.27
test_run_1	0.91	1.25
test_run_2	0.92	1.27
-------------	--------	------
pre-trained	50.4	71.0
test_run_1	51.2	71.6
test_run_2	51.6	70.9

CMU-mocap

	80ms	160ms	320ms	400ms	1000ms
pre-trained	0.25	0.39	0.68	0.79	1.33
test_run_1	0.26	0.41	0.72	0.84	1.35
test_run_2	0.26	0.41	0.71	0.83	1.38
-------------	------	-------	-------	-------	--------
pre-trained	11.5	20.4	37.8	46.8	96.5
test_run_1	11.3	19.8	36.9	45.5	92.7
test_run_2	11.3	19.7	37.2	46.0	94.0

3DPW

	200ms	400ms	600ms	800ms	1000ms
pre-trained	0.64	0.95	1.12	1.22	1.27
test_run_1	0.64	0.97	1.12	1.22	1.28
test_run_2	0.64	0.95	1.11	1.21	1.27
-------------	-------	-------	-------	-------	--------
pre-trained	35.6	67.8	90.6	106.9	117.8
test_run_1	36.7	69.6	90.8	105.0	115.3
test_run_2	35.8	69.1	93.2	110.9	121.7

Citing

If you use our code, please cite our work

@inproceedings{wei2019motion,
  title={Learning Trajectory Dependencies for Human Motion Prediction},
  author={Wei, Mao and Miaomiao, Liu and Mathieu, Salzemann and Hongdong, Li},
  booktitle={ICCV},
  year={2019}
}

Acknowledgments

Some of our evaluation code and data process code was adapted/ported from Residual Sup. RNN by Julieta. The overall code framework (dataloading, training, testing etc.) is adapted from 3d-pose-baseline.

Licence

MIT