Home

Awesome

Integral-Human-Pose-Regression-for-3D-Human-Pose-Estimation

<p align="center"> <img src="assets/1.png" width="400" height="250"> <img src="assets/2.png" width="400" height="250"> </p>

Introduction

This repo is PyTorch implementation of Integral Human Pose Regression (ECCV 2018) of MSRA for 3D human pose estimation from a single RGB image.

What this repo provides:

Dependencies

This code is tested under Ubuntu 16.04, CUDA 9.0, cuDNN 7.1 environment with two NVIDIA 1080Ti GPUs.

Python 3.6.5 version with Anaconda 3 and PyTorch 1.0.0 is used for development.

Directory

Root

The ${POSE_ROOT} is described as below.

${POSE_ROOT}
|-- data
|-- common
|-- main
|-- tool
`-- output

Data

You need to follow directory structure of the data as below.

${POSE_ROOT}
|-- data
|-- |-- MPII
|   `-- |-- annotations
|       |   |-- train.json
|       |   `-- test.json
|       `-- images
|           |-- 000001163.jpg
|           |-- 000003072.jpg
|-- |-- Human36M
|   `-- |-- data
|       |   |-- s_01_act_02_subact_01_ca_01
|       |   |-- s_01_act_02_subact_01_ca_02

Output

You need to follow the directory structure of the output folder as below.

${POSE_ROOT}
|-- output
|-- |-- log
|-- |-- model_dump
|-- |-- result
`-- |-- vis

Running code

Start

Train

In the main folder, set training set in config.py. Note that trainset must be list type and 0th dataset is the reference dataset.

In the main folder, run

python train.py --gpu 0-1

to train the network on the GPU 0,1.

If you want to continue experiment, run

python train.py --gpu 0-1 --continue

--gpu 0,1 can be used instead of --gpu 0-1.

Test

In the main folder, set testing set in config.py. Note that testset must be str type.

Place trained model at the output/model_dump/.

In the main folder, run

python test.py --gpu 0-1 --test_epoch 16

to test the network on the GPU 0,1 with 16th epoch trained model. --gpu 0,1 can be used instead of --gpu 0-1.

Results

Here I report the performance of the model from this repo and the original paper. Also, I provide pre-trained 3d human pose estimation models.

Results on Human3.6M dataset

The tables below are PA MPJPE and MPJPE on Human3.6M dataset. Provided config.py file is used to achieve below results. It's currently slightly worse than the performance of the original paper, however I'm trying to achieve the same performance. I think training schedule has to be changed.

Protocol 2 (training subjects: 1,5,6,7,8, testing subjects: 9, 11), PA MPJPE

The PA MPJPEs of the paper are from protocol 1, however, note that protocol 2 uses smaller training set.

MethodsDir.Dis.EatGre.Phon.PosePur.Sit.Sit D.Smo.Phot.WaitWalkWalk D.Walk P.Avg
my repo39.038.644.142.540.635.338.249.959.441.0046.137.630.340.835.541.5
original paper36.936.240.640.441.934.935.750.159.440.444.939.030.839.836.740.6

Protocol 2 (training subjects: 1,5,6,7,8, testing subjects: 9, 11), MPJPE

MethodsDir.Dis.EatGre.Phon.PosePur.Sit.Sit D.Smo.Phot.WaitWalkWalk D.Walk P.Avg
my repo50.852.354.857.952.847.052.162.073.752.658.350.440.954.145.153.9
original paper47.547.749.550.251.443.846.458.965.749.455.847.838.949.043.849.6

Troubleshooting

If you get an extremely large error, disable cudnn for batch normalization. This typically occurs in low version of PyTorch.

# PYTORCH=/path/to/pytorch
# for pytorch v0.4.0
sed -i "1194s/torch\.backends\.cudnn\.enabled/False/g" ${PYTORCH}/torch/nn/functional.py
# for pytorch v0.4.1
sed -i "1254s/torch\.backends\.cudnn\.enabled/False/g" ${PYTORCH}/torch/nn/functional.py

Acknowledgement

This repo is largely modified from Original PyTorch repo of IntegralHumanPose.

Reference

[1] Sun, Xiao and Xiao, Bin and Liang, Shuang and Wei, Yichen. "Integral human pose regression". ECCV 2018.