Awesome
POCO: Pose and Shape Estimation with Confidence [3DV 2024]
Sai Kumar Dwivedi, Cordelia Schmid, Hongwei Yi, Michael J. Black, Dimitrios Tzionas
International Conference on 3D Vision (3DV 2024)
<div style="display:flex;"> <img src="assets/run_lola.gif" width="45%" style="margin-right: 1%;"> <img src="assets/yt_solo.gif" width="45%"> </div>Setup and Installation
Clone the repository:
git clone https://github.com/saidwivedi/POCO.git
Create fresh conda environment and install all the dependencies:
conda create -n poco python=3.8
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge
pip install -r requirements.txt
Pretrained Models
To run the demo, please download the pretrained models and necessary files from here. Note that the Downloads url works only after sign in -- you need to register first and agree with our license.
After downloading, unzip the file and ensure that the contents are placed in a folder named ./data
.
Demo
We provide two versions of POCO: POCO-PARE and POCO-CLIFF. Note that POCO-CLIFF is more suitable for in-the-wild scenarios. To run a specific model, make sure to change the --cfg
and --ckpt
parameters accordingly i.e pare
for POCO-PARE and cliff
for POCO-CLIFF.
Run the demo on a video
python demo.py --mode video --vid_file demo_data/friends.mp4 --cfg configs/demo_poco_cliff.yaml --ckpt data/poco_cliff.pt --output_folder out
Run the demo on image folder
python demo.py --mode folder --image_folder demo_data/images --cfg configs/demo_poco_cliff.yaml --ckpt data/poco_cliff.pt --output_folder out
Acknowledgements
Parts of the code are taken or adapted from the following repos:
We thank Partha Ghosh and Haiwen Feng for insightful discussions, Priyanka Patel for the CLIFF implementation, and Peter Kulits, Shashank Tripathi, Muhammed Kocabas, and the Perceiving Systems department for their feedback. SKD acknowledges support from the International Max Planck Research School for Intelligent Systems (IMPRS-IS). This work was partially supported by the German Federal Ministry of Education and Research (BMBF): Tubingen AI Center, FKZ: 01IS18039B.
Citing
If you find this code useful for your research, please consider citing the following paper:
@inproceedings{dwivedi_3dv2023_poco,
title={{POCO}: {3D} Pose and Shape Estimation using Confidence},
author={Dwivedi, Sai Kumar and Schmid, Cordelia and Yi, Hongwei and Black, Michael J. and Tzionas, Dimitrios},
booktitle={International Conference on 3D Vision (3DV)},
year={2024},
}
License
This code is available for non-commercial scientific research purposes as defined in the LICENSE file. By downloading and using this code you agree to the terms in the LICENSE. Third-party datasets and software are subject to their respective licenses.
Contact
For code related questions, please contact sai.dwivedi@tuebingen.mpg.de
For commercial licensing (and all related questions for business applications), please contact ps-licensing@tue.mpg.de.