Home

Awesome

Awesome ML Character

A collection of papers about characters generation with machine learning

Awesome made-with-Markdown

Contents

Non-Machine Learning

It's odd that non-ML is included in the ML list. The non-ML papers serve as a foundation and important to know.

Skinning

Real-time skeletal skinning with optimized centers of rotation.<br> Le, Binh Huy, and Jessica K. Hodgins<br> 2016. [PDF]

Learning from the Artist: Theory and Practice of Example-Based Character Deformation.<br> J Lewis<br> 2016. [PDF]

SMPL: A Skinned Multi-Person Linear Model.<br> Loper, Matthew, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black<br> 2015. [PDF]

Delta Mush: smoothing deformations while preserving detail.<br> J Mancewicz, ML Derksen, H Rijpkema, et al.<br> 2014. [PDF]

Pose-space animation and transfer of facial details.<br> Bickel, Bernd, Manuel Lang, Mario Botsch, Miguel A. Otaduy, and Markus Gross<br> 2008. [PDF]

Skinning with dual quaternions.<br> Kavan, Ladislav, Steven Collins, Jiří Žára, and Carol O'Sullivan<br> 2007. [PDF]

Real‐time weighted pose‐space deformation on the GPU.<br> Rhee, Taehyun, John P. Lewis, and Ulrich Neumann<br> 2006. [PDF]

Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation.<br> Lewis, John P., Matt Cordner, and Nickson Fong<br> 2000. [PDF]

Joint-dependent local deformations for hand animation and object grasping.<br> Magnenat-Thalmann, Nadia, Richard Laperrire, and Daniel Thalmann<br> 1988. [PDF]

Physics Simulation

Enriching facial blendshape rigs with physical simulation.<br> Kozlov, Yeara, Derek Bradley, Moritz Bächer, Bernhard Thomaszewski, Thabo Beeler, and Markus Gross<br> 2017. [PDF]

Blendforces: A dynamic framework for facial animation.<br> Barrielle, Vincent, Nicolas Stoiber, and Cédric Cagniart<br> 2016. [PDF]

Reconstructing personalized anatomical models for physics-based body animation.<br> Kadleček, Petr, Alexandru-Eugen Ichim, Tiantian Liu, Jaroslav Křivánek, and Ladislav Kavan<br> 2016. [PDF]

Efficient elasticity for character skinning with contact and collisions.<br> McAdams, Aleka, Yongning Zhu, Andrew Selle, Mark Empey, Rasmus Tamstorf, Joseph Teran, and Eftychios Sifakis<br> 2011. [PDF]

Efficient Simulation of Inextensible Cloth.<br> Goldenthal, Rony, David Harmon, Raanan Fattal, Michel Bercovier, and Eitan Grinspun<br> 2007. [PDF]

Invertible Finite Elements For Robust Simulation of Large Deformation.<br> Irving, Geoffrey, Joseph Teran, and Ronald Fedkiw<br> 2004. [PDF]

Miscellaneous

Doug Roble - YouTube channel<br> [Channel]

Ladislav Kavan - YouTube channel<br> [Channel]

Parametric Model

Learning a model of facial shape and expression from 4D scans.<br> Li, Tianye, Timo Bolkart, Michael J. Black, Hao Li, and Javier Romero<br> 2017. [PDF]

Building Statistical Shape Spaces for 3D Human Modeling.<br> Pishchulin, Leonid, Stefanie Wuhrer, Thomas Helten, Christian Theobalt, and Bernt Schiele.<br> 2017. [PDF]

SMPL: A skinned multi-person linear model.<br> Loper, Matthew, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black.<br> SIGGRAPH 2015. [PDF][Course]

Parametric modeling of 3D human body shape—A survey.<br> Cheng, Zhi-Quan, Yin Chen, Ralph R. Martin, Tong Wu, and Zhan Song.<br> 2015. [Website]

A Statistical Model of Human Pose and Body Shape.<br> Hasler, Nils, Carsten Stoll, Martin Sunkel, Bodo Rosenhahn, and H‐P. Seidel. <br> 2009. [PDF]

SCAPE: Shape Completion and Animation of People.<br> Anguelov, Dragomir, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis.<br> SIGGRAPH 2005. [PDF]

The space of human body shapes: reconstruction and parameterization from range scans.<br> Allen, Brett, Brian Curless, and Zoran Popović.<br> 2003. [PDF]

A morphable model for the synthesis of 3D faces.<br> Blanz, Volker, and Thomas Vetter.<br> 1999. [PDF]

Sparse Decomposition

Generating 3D faces using convolutional mesh autoencoders.<br> Ranjan, Anurag, Timo Bolkart, Soubhik Sanyal, and Michael J. Black.<br> 2018. [PDF]

Mesh-based Autoencoders for Localized Deformation Component Analysis.<br> Tan, Qingyang, Lin Gao, Yu-Kun Lai, Jie Yang, and Shihong Xia.<br> 2018.[PDF]

Sparse Data Driven Mesh Deformation.<br> Lin Gao, Yu-Kun Lai, Jie Yang, Ling-Xiao Zhang, Leif Kobbelt, Shihong Xia.<br> 2017. [PDF]

Linear Shape Deformation Models with Local Support using Graph-based Structured Matrix Factorisation.<br> Bernard, Florian, Peter Gemmar, Frank Hertel, Jorge Goncalves, and Johan Thunberg.<br> 2016. [PDF]

Sparse Localized Decomposition of Deformation Gradients.<br> Huang, Zhichao, Junfeng Yao, Zichun Zhong, Yang Liu, and Xiaohu Guo.<br> SPLOCS with deformation gradients<br> 2014. [PDF]

Sparse localized deformation components.<br> Introduces SPLOCS.<br> Neumann, Thomas, Kiran Varanasi, Stephan Wenger, Markus Wacker, Marcus Magnor, and Christian Theobalt.<br> 2013. [PDF]

Avatar Creation

SMPLpix: Neural Avatars from 3D Human Models.<br> Prokudin, Sergey, Michael J. Black, and Javier Romero.<br> 2021. [PDF][GitHub]

Expressive body capture: 3d hands, face, and body from a single image.<br> Pavlakos, Georgios, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed AA Osman, Dimitrios Tzionas, and Michael J. Black.<br> 2019. [PDF][GitHub]

Direct Manipulation Blendshapes.<br> Lewis, John P., and Ken-ichi Anjyo.<br> 2010. [PDF]

Rigging

Learning Skeletal Articulations with Neural Blend Shapes.<br> Li, Peizhuo, Kfir Aberman, Rana Hanocka, Libin Liu, Olga Sorkine-Hornung, and Baoquan Chen.<br> SIGGRAPH 2021. [PDF]

HeterSkinNet: A Heterogeneous Network for Skin Weights Prediction.<br> Pan, Xiaoyu, Jiancong Huang, Jiaming Mai, He Wang, Honglin Li, Tongkui Su, Wenjun Wang, and Xiaogang Jin.<br> I3D 2021. [PDF]

Rignet: Neural rigging for articulated characters.<br> Xu, Zhan, Yang Zhou, Evangelos Kalogerakis, Chris Landreth, and Karan Singh.<br> SIGGRAPH 2020. [PDF]

NiLBS: Neural Inverse Linear Blend Skinning.<br> Jeruzalski, Timothy, David IW Levin, Alec Jacobson, Paul Lalonde, Mohammad Norouzi, and Andrea Tagliasacchi.<br> SIGGRAPH 2020. [PDF]

Deformation

PBNS: Physically Based Neural Simulator for Unsupervised Garment Pose Space Deformation.<br> Bertiche, Hugo, Meysam Madadi, and Sergio Escalera.<br> 2020.[PDF]

Fast and deep deformation approximations.<br> Bailey, Stephen W., Dave Otte, Paul Dilorenzo, and James F. O'Brien.<br> 2018.[PDF]

Fast and deep facial deformations.<br> Bailey, Stephen W., Dalton Omens, Paul Dilorenzo, and James F. O'Brien.<br> 2020. [PDF]

Body Simulation

SoftSMPL: Data-driven Modeling of Nonlinear Soft-tissue Dynamics for Parametric Humans.<br> Santesteban, Igor, Elena Garces, Miguel A. Otaduy, and Dan Casas.<br> Eurographics 2020.[PDF]

Cloth Simulation

SNUG: Self-Supervised Neural Dynamic Garments.<br> Igor Santesteban, Miguel A. Otaduy, and Dan Casas.<br> 2022. [PDF]

Self-Supervised Collision Handling via Generative 3D Garment Models for Virtual Try-On.<br> Igor Santesteban, Nils Thuerey, Miguel A. Otaduy and Dan Casas.<br> 2021. [PDF]

Swish: Neural Network Cloth Simulation on Madden NFL 21.<br> Lewin, Chris, James Power, and James Cobb.<br> 2021.[PDF]

TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style.<br> Patel, Chaitanya, Zhouyingcheng Liao, and Gerard Pons-Moll.<br> 2020. [PDF]

Learning Mesh-Based Simulation with Graph Networks.<br> Pfaff, Tobias, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W. Battaglia.<br> 2020. [PDF]

Subspace neural physics: Fast data-driven interactive simulation.<br> Holden, Daniel, Bang Chi Duong, Sayantan Datta, and Derek Nowrouzezahrai..<br> 2020. [PDF]

Motion Controller

Data-Driven Controller

ProtoRes: Proto-Residual Architecture for Deep Modeling of Human Pose.<br> Oreshkin, Boris N., Florent Bocquelet, Félix H. Harvey, Bay Raitt, and Dominic Laflamme.<br> 2021. [PDF][Website]

Neural animation layering for synthesizing martial arts movements.<br> Starke, Sebastian, Yiwei Zhao, Fabio Zinno, and Taku Komura.<br> 2021. [PDF]

AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control.<br> Peng, Xue Bin, Ze Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa.<br> 2021. [PDF]

A GAN-Like Approach for Physics-Based Imitation Learning and Interactive Character Control.<br> Xu, Pei, and Ioannis Karamouzas.<br> 2021. [PDF]

Learned motion matching.<br> Xu, Pei, and Ioannis Karamouzas.<br> 2020. [PDF]

Local motion phases for learning multi-contact character movements.<br> Starke, Sebastian, Yiwei Zhao, Taku Komura, and Kazi Zaman.<br> 2020. [PDF]

Neural state machine for character-scene interactions.<br> Starke, Sebastian, He Zhang, Taku Komura, and Jun Saito.<br> SIGGRAPH Asia 2019. [PDF]

Mode-adaptive neural networks for quadruped motion control .<br> Zhang, He, Sebastian Starke, Taku Komura, and Jun Saito.<br> 2018. [PDF]

Phase-functioned neural networks for character control.<br> Holden, Daniel, Taku Komura, and Jun Saito.<br> 2017. [PDF]

Physics-Based Controller

CARL: Controllable Agent with Reinforcement Learning for Quadruped Locomotion.<br> Luo, Ying-Sheng, Jonathan Hans Soeseno, Trista Pei-Chun Chen, and Wei-Chao Chen.<br> 2020. [PDF]

Motion Generation

From Motor Control to Team Play in Simulated Humanoid Football.<br> Liu, Siqi, Guy Lever, Zhe Wang, Josh Merel, S. M. Eslami, Daniel Hennes, Wojciech M. Czarnecki et al.<br> 2021.[PDF]

Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning.<br> Peng, Xue Bin, Glen Berseth, KangKang Yin, and Michiel Van De Panne.<br> 2017.[PDF]

Emergence of locomotion behaviours in rich environments.<br> Heess, Nicolas, Dhruva TB, Srinivasan Sriram, Jay Lemmon, Josh Merel, Greg Wayne, Yuval Tassa et al..<br> 2017.[PDF]

Flexible muscle-based locomotion for bipedal creatures.<br> Geijtenbeek, Thomas, Michiel Van De Panne, and A. Frank Van Der Stappen<br> 2013.[PDF]

Motion Inbetweening

Recurrent transition networks for character locomotion.<br> Harvey, Félix G., and Christopher Pal<br> SIGGRAPH Asia 2018.[PDF]

Robust motion in-betweening.<br> Harvey, Félix G., Mike Yurick, Derek Nowrouzezahrai, and Christopher Pal<br> SIGGRAPH 2020.[PDF]

Motion Retargeting

Skeleton-Aware Networks for Deep Motion Retargeting.<br> Aberman, Kfir, Peizhuo Li, Dani Lischinski, Olga Sorkine-Hornung, Daniel Cohen-Or, and Baoquan Chen<br> SIGGRAPH 2020.[PDF]

Dataset Generation

Learning from Synthetic Humans.<br> Varol, Gul, Javier Romero, Xavier Martin, Naureen Mahmood, Michael J. Black, Ivan Laptev, and Cordelia Schmid.<br> 2017. [PDF]

Datasets

LAFAN1 - Ubisoft La Forge Animation Dataset.<br> Félix G. Harvey and Mike Yurick and Derek Nowrouzezahrai and Christopher Pal.<br> 2020. [Website]

MPII Human Pose - 2D Human Pose Estimation: New Benchmark and State of the Art Analysis.<br> Mykhaylo Andriluka and Leonid Pishchulin and Peter Gehler and Schiele, Bernt.<br> 2014. [Website]

Human3.6M Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments.<br> Ionescu, Catalin, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu.<br> 2013. [Website]

HumanEva I/II<br> Ionescu, Catalin, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu.<br> [Website]

Caesar<br> [Website]

DensePose<br> Facebook Research<br> 50K humans, collecting more then 5 million manually annotated correspondences<br> [Website]

SURREAL : Synthetic hUmans foR REAL tasks<br> 50K humans, collecting more then 5 million manually annotated correspondences<br> 6M RGB frames of synthetic humans.<br> [Website]

Interisting Links

Contribute

Contributions welcome! Read the contribution guidelines first.