Home

Awesome

[CVPR 2022] AdaptPose: Cross-Dataset Adaptation for 3D Human Pose Estimation by Learnable Motion Generation

PWC Open In Colab

This implementation is based on VideoPsoe3D and PoseAug. Experiments on 4 datasets: Human3.6M, 3DHP, 3DPW, and Ski are provided. Adaptpose is aimed to improve the accuracy of 3D pose estimators in cross-dataset scenarios.

<p align="center">. <img src="Figures/Tiser.jpg" width="600"> <p/>

Google Colab

If you do not have a suitable environment to run this project then you could give Google Colab a try. It allows you to run the project in the cloud, free of charge. You may try our Colab demo using the notebook we have prepared: Open In Colab

Environment

cd AdaptPose
conda create -n adaptpose python=3.6.9
conda activate adaptpose

Install pytorch3d following the instructions here.

pip install -r requirements.txt

Dataset setup

Due to license issues, we can not share the Huamn3.6m dataset. Please refer to here for instructions on downloading and processing Human3.6M. After downloaing you need to have two files for Human3.6M: data_3d_h36m.npz data_2d_h36m_gt.npz. Here we provide processed data of 3DHP, 3PW, and SKi dataset:

source scripts/prepare_data.sh

Experiments:

Download the pretraind models:

source scripts/pretrained_models.sh

1. Cross-dataset Evaluation of Pretrained Model on 3DHP dataset

Source:Human3.6M/Target:3DHP

python3  run_evaluate.py --posenet_name 'videopose' --keypoints gt --evaluate  'checkpoint/adaptpose/videopose/gt/3dhp/ckpt_best_dhp_p1.pth.tar' --dataset_target 3dhp --keypoints_target 'gt' --pad 13  --pretrain_path  'checkpoint/pretrain_baseline/videopose/gt/3dhp/ckpt_best.pth.tar'

2. Cross-dataset Training of Pretrained Model on 3DHP dataset

Source:Human3.6M/Target:3DHP

python3 run_adaptpose.py --note poseaug --posenet_name 'videopose' --lr_p 1e-4 --checkpoint './checkpoint/adaptpose' --keypoints gt --keypoints_target gt --dataset_target '3dhp'  --pretrain_path './checkpoint/pretrain_baseline/videopose/gt/3dhp/ckpt_best.pth.tar'  --pad 13

3. Cross-dataset Evaluation of Pretrained Model on 3DPW dataset

Source:Human3.6M/Target:3DPW

python3  run_evaluate.py --posenet_name 'videopose' --keypoints gt --evaluate  'checkpoint/adaptpose/videopose/gt/3dpw/ckpt_best_dhp_p1.pth.tar' --dataset_target 3dpw --keypoints_target 'gt' --pad 13  --pretrain_path  'checkpoint/pretrain_baseline/videopose/gt/3dpw/ckpt_best.pth.tar'

4. Cross-dataset Training of Pretrained Model on 3DPW dataset

Source:Human3.6M/Target:3DPW

ppython3 run_adaptpose.py --note poseaug --posenet_name 'videopose' --lr_p 1e-4 --checkpoint './checkpoint/adaptpose' --keypoints gt --keypoints_target gt --dataset_target '3dpw'  --pretrain_path './checkpoint/pretrain_baseline/videopose/gt/3dhp/ckpt_best.pth.tar'  --pad 13

Citations:

@InProceedings{Gholami_2022_CVPR,
    author    = {Gholami, Mohsen and Wandt, Bastian and Rhodin, Helge and Ward, Rabab and Wang, Z. Jane},
    title     = {AdaptPose: Cross-Dataset Adaptation for 3D Human Pose Estimation by Learnable Motion Generation},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {13075-13085}
}