Awesome

Awesome 3D Human Resources List

A curated list of papers & resources linked to 3D Human research including face, body, hand and motion modeling.

Contents

• Papers

  • Face Modeling
    • Face Reconstriction (Geometry Only)
    • Face Reconstruction (With Texture)
    • Audio-Driven Face Animation
    • Other 3D Face Tasks
  • Body Modeling
    • Body Reconstruction (Geometry Only)
    • Body Reconstruction (With Texture)
    • Human Capture
    • 3D Pose Estimation
  • Hand Modeling
    • Hand Shape Estimation
    • 3D Hand Pose Estimation
  • Motion Modeling
    • Motion Prediction/Synthesis

• Contributing

<a name="papers"></a>

Papers

<a name="face"></a>

Face

<a name="papers-face-geometry-only"></a>

Face Reconstruction (Geometry Only)

High-Quality Face Capture Using Anatomical Muscles. Michael Bao, Matthew Cong, Stephane Grabli, Ronald Fedkiw. CVPR 2019.

Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision. Soubhik Sanyal, Timo Bolkart, Haiwen Feng, Michael J. Black. CVPR 2019.

MVF-Net: Multi-View 3D Face Morphable Model Regression. Fanzi Wu, Linchao Bao, Yajing Chen, Yonggen Ling, Yibing Song, Songnan Li, King Ngi Ngan, Wei Liu. CVPR 2019.

Generating 3D Faces using Convolutional Mesh Autoencoders. Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, Michael J. Black. ECCV 2018.

Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network. Yao Feng, Fan Wu, Xiaohu Shao, Yanfeng Wang, Xi Zhou. ECCV 2018.

3D Face Reconstruction from Light Field Images: A Model-free Approach. Mingtao Feng, Syed Zulqarnain Gilani, Yaonan Wang, Ajmal Mian. ECCV 2018.

Mesoscopic Facial Geometry Inference Using Deep Neural Networks. Loc Huynh, Weikai Chen, Shunsuke Saito, Jun Xing, Koki Nagano, Andrew Jones, Paul Debevec, Hao Li. CVPR 2018.

Extreme 3D Face Reconstruction: Seeing Through Occlusions. Anh Tuấn Trần, Tal Hassner, Iacopo Masi, Eran Paz, Yuval Nirkin, Gérard Medioni. CVPR 2018.

Sparse Photometric 3D Face Reconstruction Guided by Morphable Models. Xuan Cao, Zhang Chen, Anpei Chen, Xin Chen, Shiying Li, Jingyi Yu. CVPR 2018.

Disentangling Features in 3D Face Shapes for Joint Face Reconstruction and Recognition. Feng Liu, Ronghang Zhu, Dan Zeng, Qijun Zhao, Xiaoming Liu. CVPR 2018.

Probabilistic Joint Face-Skull Modelling for Facial Reconstruction. Dennis Madsen, Marcel Lüthi, Andreas Schneider, Thomas Vetter. CVPR 2018.

Large Pose 3D Face Reconstruction From a Single Image via Direct Volumetric CNN Regression. Aaron S. Jackson, Adrian Bulat, Vasileios Argyriou, Georgios Tzimiropoulos. ICCV 2017.

Learning Detailed Face Reconstruction From a Single Image. Elad Richardson, Matan Sela, Roy Or-El, Ron Kimmel. CVPR 2017.

End-To-End 3D Face Reconstruction With Deep Neural Networks. Pengfei Dou, Shishir K. Shah, Ioannis A. Kakadiaris. CVPR 2017.

Parametric T-Spline Face Morphable Model for Detailed Fitting in Shape Subspace. Weilong Peng, Zhiyong Feng, Chao Xu, Yong Su. CVPR 2017.

<a name="papers-face-texture"></a>

Face Reconstruction (With Texture)

FML: Face Model Learning From Videos. Ayush Tewari, Florian Bernard, Pablo Garrido, Gaurav Bharaj, Mohamed Elgharib, Hans-Peter Seidel, Patrick Perez, Michael Zollhofer, Christian Theobalt. CVPR 2019.

Dense 3D Face Decoding Over 2500FPS: Joint Texture & Shape Convolutional Mesh Decoders. Yuxiang Zhou, Jiankang Deng, Irene Kotsia, Stefanos Zafeiriou. CVPR 2019.

Towards High-Fidelity Nonlinear 3D Face Morphable Model. Luan Tran, Feng Liu, Xiaoming Liu. CVPR 2019.

GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction. Baris Gecer, Stylianos Ploumpis, Irene Kotsia, Stefanos Zafeiriou. CVPR 2019.

PAGAN: Real-time Avatars Using Dynamic Textures. Koki Nagano, Jaewoo Seo, Jun Xing, Lingyu Wei, Zimo Li, Shunsuke Saito, Aviral Agarwal, Jens Fursund, Hao Li. SIGGRAPH Asia 2018.

High-fidelity Facial Reflectance And Geometry Inference From An Unconstrained Image. Shugo Yamaguchi, Shunsuke Saito, Koki Nagano, Yajie Zhao, Weikai Chen, Kyle Olszewski, Shigeo Morishima, Hao Li. SIGGRAPH 2018.

Self-Supervised Multi-Level Face Model Learning for Monocular Reconstruction at Over 250 Hz. Ayush Tewari, Michael Zollhöfer, Pablo Garrido, Florian Bernard, Hyeongwoo Kim, Patrick Pérez, Christian Theobalt. CVPR 2018.

InverseFaceNet: Deep Monocular Inverse Face Rendering. Hyeongwoo Kim, Michael Zollhöfer, Ayush Tewari, Justus Thies, Christian Richardt, Christian Theobalt. CVPR 2018.

Nonlinear 3D Face Morphable Model. Luan Tran, Xiaoming Liu. CVPR 2018.

SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wild. Soumyadip Sengupta, Angjoo Kanazawa, Carlos D. Castillo, David W. Jacobs. CVPR 2018.

MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction. Ayush Tewari, Michael Zollhofer, Hyeongwoo Kim, Pablo Garrido, Florian Bernard, Patrick Perez, Christian Theobalt. ICCV 2017.

3D Face Morphable Models "In-The-Wild". James Booth, Epameinondas Antonakos, Stylianos Ploumpis, George Trigeorgis, Yannis Panagakis, Stefanos Zafeiriou. CVPR 2017.

Fast 3D Reconstruction of Faces With Glasses. Fabio Maninchedda, Martin R. Oswald, Marc Pollefeys. CVPR 2017.

<a name="papers-face-audio"></a>

Audio-Driven Face Animation

Capture, Learning, and Synthesis of 3D Speaking Styles. Daniel Cudeiro, Timo Bolkart, Cassidy Laidlaw, Anurag Ranjan, Michael J. Black. CVPR 2019.

Audio-Driven Facial Animation by Joint End-to-End Learning of Pose and Emotion. Tero Karras, Timo Aila, Samuli Laine, Antti Herva, Jaakko Lehtinen. SIGGRAPH 2017.

<a name="papers-face-others"></a>

Other 3D Face Tasks

Dense Semantic and Topological Correspondence of 3D Faces without Landmarks. Zhenfeng Fan, Xiyuan Hu, Chen Chen, Silong Peng. ECCV 2018.

Learning From Millions of 3D Scans for Large-Scale 3D Face Recognition. Syed Zulqarnain Gilani, Ajmal Mian. CVPR 2018.

<a name="papers-body"></a>

Body

<a name="papers-body-geometry-only"></a>

Body Reconstruction (Geometry Only)

HoloPose: Holistic 3D Human Reconstruction In-The-Wild. Riza Alp Guler, Iasonas Kokkinos. CVPR 2019.

Detailed Human Shape Estimation From a Single Image by Hierarchical Mesh Deformation. Hao Zhu, Xinxin Zuo, Sen Wang, Xun Cao, Ruigang Yang. CVPR 2019.

Convolutional Mesh Regression for Single-Image Human Shape Reconstruction. Nikos Kolotouros, Georgios Pavlakos, Kostas Daniilidis. CVPR 2019.

Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimation. Mohamed Omran, Christoph Lassner, Gerard Pons-Moll, Peter V. Gehler, Bernt Schiele. 3DV 2018.

End-to-End Recovery of Human Shape and Pose. Angjoo Kanazawa, Michael J. Black, David W. Jacobs, Jitendra Malik. CVPR 2018.

Learning to Estimate 3D Human Pose and Shape From a Single Color Image. Georgios Pavlakos, Luyang Zhu, Xiaowei Zhou, Kostas Daniilidis. CVPR 2018.

Detailed, Accurate, Human Shape Estimation From Clothed 3D Scan Sequences. Chao Zhang, Sergi Pujades, Michael J. Black, Gerard Pons-Moll. CVPR 2017.

Human Shape From Silhouettes Using Generative HKS Descriptors and Cross-Modal Neural Networks. Endri Dibra, Himanshu Jain, Cengiz Oztireli, Remo Ziegler, Markus Gross. CVPR 2017.

Unite the People: Closing the Loop Between 3D and 2D Human Representations. Christoph Lassner, Javier Romero, Martin Kiefel, Federica Bogo, Michael J. Black, Peter V. Gehler. CVPR 2017.

Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image. Federica Bogo*, Angjoo Kanazawa*, Christoph Lassner, Peter Gehler,Javier Romero, Michael J. Black. ECCV 2016.

SMPL: A Skinned Multi-Person Linear Model. Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, Michael J. Black. 2015.

The Stitched Puppet: A Graphical Model of 3D Human Shape and Pose. Silvia Zuffi, Michael J. Black. CVPR 2015.

<a name="papers-body-texture"></a>

Body Reconstruction (With Texture)

PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization. Shunsuke Saito*, Zeng Huang*, Ryota Natsume*, Shigeo Morishima, Angjoo Kanazawa, Hao Li. ICCV 2019.

SiCloPe: Silhouette-Based Clothed People. Ryota Natsume, Shunsuke Saito, Zeng Huang, Weikai Chen, Chongyang Ma, Hao Li, Shigeo Morishima. CVPR 2019.

Learning to Reconstruct People in Clothing From a Single RGB Camera. Thiemo Alldieck, Marcus Magnor, Bharat Lal Bhatnagar, Christian Theobalt, Gerard Pons-Moll. CVPR 2019.

<a name="papers-body-capture"></a>

Human Capture

Expressive Body Capture: 3D Hands, Face, and Body From a Single Image. Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, Michael J. Black. CVPR 2019.

Volumetric Capture of Humans With a Single RGBD Camera via Semi-Parametric Learning. Rohit Pandey, Anastasia Tkach, Shuoran Yang, Pavel Pidlypenskyi, Jonathan Taylor, Ricardo Martin-Brualla, Andrea Tagliasacchi, George Papandreou, Philip Davidson, Cem Keskin, Shahram Izadi, Sean Fanello. CVPR 2019.

SimulCap : Single-View Human Performance Capture With Cloth Simulation. Tao Yu, Zerong Zheng, Yuan Zhong, Jianhui Zhao, Qionghai Dai, Gerard Pons-Moll, Yebin Liu. CVPR 2019.

Monocular Total Capture: Posing Face, Body, and Hands in the Wild. Donglai Xiang, Hanbyul Joo, Yaser Sheikh. CVPR 2019.

HybridFusion: Real-Time Performance Capture Using a Single Depth Sensor and Sparse IMUs. Zerong Zheng, Tao Yu, Hao Li, Kaiwen Guo, Qionghai Dai, Lu Fang, Yebin Liu. ECCV 2018.

DoubleFusion: Real-Time Capture of Human Performances With Inner Body Shapes From a Single Depth Sensor. Tao Yu, Zerong Zheng, Kaiwen Guo, Jianhui Zhao, Qionghai Dai, Hao Li, Gerard Pons-Moll, Yebin Liu. CVPR 2018.

Deep Volumetric Video From Very Sparse Multi-View Performance Capture. Zeng Huang, Tianye Li, Weikai Chen, Yajie Zhao, Jun Xing, Chloe LeGendre, Linjie Luo, Chongyang Ma, Hao Li. ECCV 2018.

Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies. Hanbyul Joo, Tomas Simon, Yaser Sheikh. CVPR 2018.

BodyFusion: Real-Time Capture of Human Motion and Surface Geometry Using a Single Depth Camera. Tao Yu, Kaiwen Guo, Feng Xu, Yuan Dong, Zhaoqi Su, Jianhui Zhao, Jianguo Li, Qionghai Dai, Yebin Liu. ICCV 2017.

<a name="papers-body-correspondence"></a>

Body Correspondence

4D Human Body Correspondences From Panoramic Depth Maps. Zhong Li, Minye Wu, Wangyiteng Zhou, Jingyi Yu. CVPR 2018.

Dynamic FAUST: Registering Human Bodies in Motion. Federica Bogo, Javier Romero, Gerard Pons-Moll, Michael J. Black. CVPR 2017.

FAUST: Dataset and evaluation for 3D mesh registration. Federica Bogo, Javier Romero, Matthew Loper, Michael J. Black. CVPR 2016.

<a name="papers-body-pose"></a>

3D Pose Estimation

3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training. Dario Pavllo, Christoph Feichtenhofer, David Grangier, Michael Auli. CVPR 2019.

RepNet: Weakly Supervised Training of an Adversarial Reprojection Network for 3D Human Pose Estimation. Bastian Wandt, Bodo Rosenhahn. CVPR 2019.

Generating Multiple Hypotheses for 3D Human Pose Estimation With Mixture Density Network. Chen Li, Gim Hee Lee. CVPR 2019.

Weakly-Supervised Discovery of Geometry-Aware Representation for 3D Human Pose Estimation. Xipeng Chen, Kwan-Yee Lin, Wentao Liu, Chen Qian, Liang Lin. CVPR 2019.

Self-Supervised Learning of 3D Human Pose Using Multi-View Geometry. Muhammed Kocabas, Salih Karagoz, Emre Akbas. CVPR 2019.

Exploiting Temporal Context for 3D Human Pose Estimation in the Wild. Anurag Arnab, Carl Doersch, Andrew Zisserman. CVPR 2019.

Semantic Graph Convolutional Networks for 3D Human Pose Regression. Long Zhao, Xi Peng, Yu Tian, Mubbasir Kapadia, Dimitris N. Metaxas. CVPR 2019.

3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training. Dario Pavllo, Christoph Feichtenhofer, David Grangier, Michael Auli. CVPR 2019.

3D Human Pose Estimation in the Wild by Adversarial Learning. Wei Yang, Wanli Ouyang, Xiaolong Wang, Jimmy Ren, Hongsheng Li, Xiaogang Wang. CVPR 2018.

Ordinal Depth Supervision for 3D Human Pose Estimation. Georgios Pavlakos, Xiaowei Zhou, Kostas Daniilidis. CVPR 2018.

Learning Monocular 3D Human Pose Estimation From Multi-View Images. Helge Rhodin, Jörg Spörri, Isinsu Katircioglu, Victor Constantin, Frédéric Meyer, Erich Müller, Mathieu Salzmann, Pascal Fua. CVPR 2018.

Coarse-To-Fine Volumetric Prediction for Single-Image 3D Human Pose. Georgios Pavlakos, Xiaowei Zhou, Konstantinos G. Derpanis, Kostas Daniilidis. CVPR 2017.

3D Human Pose Estimation = 2D Pose Estimation + Matching. Ching-Hang Chen, Deva Ramanan. CVPR 2017.

<a name="papers-hand"></a>

Hand

<a name="papers-hand-shape"></a>

Hand Shape Estimation

Learning Joint Reconstruction of Hands and Manipulated Objects. Yana Hasson, Gul Varol, Dimitrios Tzionas, Igor Kalevatykh, Michael J. Black, Ivan Laptev, Cordelia Schmid. CVPR 2019.

3D Hand Shape and Pose Estimation From a Single RGB Image. Liuhao Ge, Zhou Ren, Yuncheng Li, Zehao Xue, Yingying Wang, Jianfei Cai, Junsong Yuan. CVPR 2019.

3D Hand Shape and Pose From Images in the Wild. Adnane Boukhayma, Rodrigo de Bem, Philip H.S. Torr. CVPR 2019.

<a name="papers-hand-pose"></a>

3D Hand Pose Estimation

Pushing the Envelope for RGB-Based Dense 3D Hand Pose Estimation via Neural Rendering. Seungryul Baek, Kwang In Kim, Tae-Kyun Kim. CVPR 2019.

Self-Supervised 3D Hand Pose Estimation Through Training by Fitting. Chengde Wan, Thomas Probst, Luc Van Gool, Angela Yao. CVPR 2019.

V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation From a Single Depth Map. Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee. CVPR 2018.

Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals. Shanxin Yuan, Guillermo Garcia-Hernando, Björn Stenger, Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee, Pavlo Molchanov, Jan Kautz, Sina Honari, Liuhao Ge, Junsong Yuan, Xinghao Chen, Guijin Wang, Fan Yang, Kai Akiyama, Yang Wu, Qingfu Wan, Meysam Madadi, Sergio Escalera, Shile Li, Dongheui Lee, Iason Oikonomidis, Antonis Argyros, Tae-Kyun Kim. CVPR 2018.

Dense 3D Regression for Hand Pose Estimation. Chengde Wan, Thomas Probst, Luc Van Gool, Angela Yao. CVPR 2018.

Hand PointNet: 3D Hand Pose Estimation Using Point Sets. Liuhao Ge, Yujun Cai, Junwu Weng, Junsong Yuan. CVPR 2018.

Point-to-Point Regression PointNet for 3D Hand Pose Estimation. Liuhao Ge, Zhou Ren, Junsong Yuan. ECCV 2018.

Weakly-supervised 3D Hand Pose Estimation from Monocular RGB Images. Yujun Cai, Liuhao Ge, Jianfei Cai, Junsong Yuan. ECCV 2018.

3D Convolutional Neural Networks for Efficient and Robust Hand Pose Estimation From Single Depth Images. Liuhao Ge, Hui Liang, Junsong Yuan, Daniel Thalmann. CVPR 2017.

<a name="papers-motion"></a>

Motion Modeling

<a name="papers-motion-pred"></a>

Motion Prediction/Synthesis

Towards Natural and Accurate Future Motion Prediction of Humans and Animals. Zhenguang Liu, Shuang Wu, Shuyuan Jin, Qi Liu, Shijian Lu, Roger Zimmermann, Li Cheng. CVPR 2019.

A Neural Temporal Model for Human Motion Prediction. Anand Gopalakrishnan, Ankur Mali, Dan Kifer, Lee Giles, Alexander G. Ororbia. CVPR 2019.

Learning 3D Human Dynamics From Video. Angjoo Kanazawa, Jason Y. Zhang, Panna Felsen, Jitendra Malik. CVPR 2019.

On the Continuity of Rotation Representations in Neural Networks. Yi Zhou*, Connelly Barnes*, Jingwan Lu, Jimei Yang, Hao Li. CVPR 2019.

Convolutional Sequence to Sequence Model for Human Dynamics. Chen Li, Zhen Zhang, Wee Sun Lee, Gim Hee Lee. CVPR 2018.

Auto-conditioned LSTM Network For Extended Complex Human Motion Synthesis. Yi Zhou*, Zimo Li*, Shuangjio Xiao, Chong He, Zeng Huang, Hao Li. ICLR 2018.

Forecasting Human Dynamics From Static Images. Yu-Wei Chao, Jimei Yang, Brian Price, Scott Cohen, Jia Deng. CVPR 2017.

On Human Motion Prediction Using Recurrent Neural Networks. Julieta Martinez, Michael J. Black, Javier Romero. CVPR 2017.

Deep Representation Learning for Human Motion Prediction and Classification. Judith Butepage, Michael J. Black, Danica Kragic, Hedvig Kjellstrom. CVPR 2017.

Contributing

Please see CONTRIBUTING for details.