Home

Awesome

STAR: Sparse Trained Articulated Human Body Regressor

<!-- TODO: Replace with our arxiv link --> <!-- [![report](https://img.shields.io/badge/arxiv-report-red)](https://arxiv.org/abs/1912.05656) -->

[Project Page] [Paper] [Supp. Mat.]

<p align="center"> <img src="./images/main_teaser.png" /> </p>

Table of Contents

License

Software Copyright License for non-commercial scientific research purposes. Please read carefully the LICENSE file and any accompanying documentation before you download and/or use the STAR model and software, (the "Data & Software"). By downloading and/or using the Data & Software (including downloading, cloning, installing, and any other use of the corresponding github repository), you acknowledge that you have read these terms and conditions in the LICENSE file, understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not download and/or use the Data & Software. Any infringement of the terms of this agreement will automatically terminate your rights under this License

Description

STAR - A Sparse Trained Articulated Human Body Regressor is a generateive 3D human body model, that is designed to be a drop-in replacement for the widely used SMPL model. STAR is trained on a large dataset of 14,000 human subjects, with a learned set of sparse and spatially local pose corrective blend shapes. In the Figure below, a single joint movement only influences a sparse set of the model vertices. The mesh vertices in gray are not affected by the joint movement. In contrast, for SMPL, bending the left elbow causes a bulge in the right elbow. <br/> STAR is publicly avaiable with the full 300 principal-component shape space for research purposes from our website https://star.is.tue.mpg.de/

<p align="center"> <img src="./images/sparse_pose_correctives.png" /> </p>

For more details, please see our ECCV paper STAR: Sparse Trained Articulated Human Body Regressor.

Content

This repository contains the model loader for the following auto-differention frameworks:

Code tested on Python 3.69, CUDA 10.1, CuDNN 7.6.5 and PyTorch 1.6.0, Tensorflow 2.3, Chumpy 0.69 on Ubuntu 18.04

Installation

Install

We recommend doing the following in a python3 virtual environment.

  1. Clone the repository:
git clone git@github.com:ahmedosman/STAR.git
  1. Install your favorite framework <br/> Chumpy
pip install chumpy==0.69
pip install opencv-python

PyTorch

pip install pytorch==1.6

Tensorflow

pip install tensorflow-gpu==2.3

  1. Download the models from our website https://star.is.tue.mpg.de/

  2. Update the model paths in the config.py file.

path_male_star = '/mypath/male/model.npz'
path_female_star = '/mypath/female/model.npz'
path_neutral_star = '/mypath/neutral/model.npz'
  1. Install with pip
pip install .

Usage

Under demos/* there are scripts demonstrating how to load and use the model in all frameworks.

    $PATH_TO_REPO/
    ├── demos
    │   │
    │   ├── compare_frameworks.py #Unit test script constructing the model with three frameworks and comparing the output
    │   └── load_chumpy.py        #A script demonstrating loading the model in chumpy
    │   └── load_tf.py            #A script demonstrating loading the model in Tensorflow
    │   └── load_torch.py         #A script demonstrating loading the model in PyTorch
    │   └── profile_tf.py         #A script profiling the STAR graph as a function of batch Size in Tensorflow
    |   └── profile_torch.py      #A script profiling the STAR graph as a function of batch Size in PyTorch

SMPL Comparison

STAR is designed to be a drop in replacement for SMPL, similar to SMPL it is parameterised with pose and shape parameters, with the same template resolution and kinematic tree.

<p align="center"> <img src="./images/smpl_vs_star.jpeg" /> </p>

STAR Kinematic Tree

<p align="center"> <img src="./images/star_kinematic_tree.png" /> </p>

Citation

If you find this Model & Software useful in your research we would kindly ask you to cite:

      author = {Osman, Ahmed A A and Bolkart, Timo and Black, Michael J.},
      title = {{STAR}: A Sparse Trained Articulated Human Body Regressor},
      booktitle = {European Conference on Computer Vision (ECCV)},
      pages = {598--613},
      year = {2020},
      url = {https://star.is.tue.mpg.de}
}

Acknowledgments

We thank Naureen M. Mahmood, Talha Zaman, Nikos Athanasiou, Joachim Tesch, Muhammed Kocabas, Nikos Kolotouros and Vassilis Choutas for the discussions and Sai Kumar Dwivedi, Lea Muller, Amir Ahmad and Nitin Saini for proof reading the script and Mason Landry for the video voice over and Benjamin Pellkofer for the IT support.

Contact

For questions, please contact star@tue.mpg.de.

For commercial licensing (and all related questions for business applications), please contact ps-license@tue.mpg.de.