Home

Awesome

Face-Pose-Net

Teaser <sub>Extreme face alignment examples: Faces rendered to a 45 degrees yaw angle (aligned to half profile) using our FacePoseNet. Images were taken from the IJB-A collection and represent extreme viewing conditions, including near profile views, occlusions, and low resolution. Such conditions are often too hard for existing face landmark detection methods to handle yet easily aligned with our FacePoseNet.</sub> <br/> <br/> This page contains DCNN model and python code to robustly estimate 6 degrees of freedom, 3D face pose from an unconstrained image, without the use of face landmark detectors. The method is described in the paper:

F.-J. Chang, A. Tran, T. Hassner, I. Masi, R. Nevatia, G. Medioni, "FacePoseNet: Making a Case for Landmark-Free Face Alignment", in 7th IEEE International Workshop on Analysis and Modeling of Faces and Gestures, ICCV Workshops, 2017 [1].

This release bundles up our FacePoseNet (FPN) with the Face Renderer from Masi et al. [2,5], which is available separately from this project page.

The result is an end-to-end pipeline that seamlessly estimates facial pose and produces multiple rendered views to be used for face alignment and data augmentation.

Teaser

Updates (Modified and New features, 12/20/2018)

$ python main_predict_6DoF.py <gpu_id> <input-list-path>

$ python main_predict_ProjMat.py <gpu_id> <input-list-path>

We provide a sample input list available here.

<FILE_NAME, FACE_X, FACE_y, FACE_WIDTH, FACE_HEIGHT>

where <FACE_X, FACE_y, FACE_WIDTH, FACE_HEIGHT> is the x,y coordinates of the upper-left point, the width, and the height of the tight face bounding box, either obtained manually, by the face detector or by the landmark detector. The predicted 6DoF and 11DoF results would be saved in output_6DoF folder and output_ProjMat folder respectively. The output 3D shapes and landmarks by 6DoF and 11DoF are saved in output_6DoF folder and in output_ProjMat folder respectively. You can visualize the 3D shapes and landmarks via Matlab.

Features

Dependencies

The code has been tested on Linux only. On Linux you can rely on the default version of python, installing all the packages needed from the package manager or on Anaconda Python and install required packages through conda.

Note: no landmarks are used in our method, although you can still project the landmarks on the input image using the estimated pose. See the paper for further details.

Usage

Run it

The alignment and rendering can be used from the command line in the following, different ways.

To run it directly on a list of images (software will run FPN to estimate the pose and then render novel views based on the estimated pose):

$ python main_fpn.py <input-list-path>

We provide a sample input list available here.

<ID, FILE, FACE_X, FACE_y, FACE_WIDTH, FACE_HEIGHT>

where <FACE_X, FACE_y, FACE_WIDTH, FACE_HEIGHT> is the face bounding box information, either obtained manually or by the face detector.

Sample Results

Please see the input images here and rendered outputs here.

input:

sbj10

rendering:

sbj10 sbj10 sbj10 sbj10 sbj10

Current Limitations

FPN is currently trained with a single 3D generic shape, without accounting for facial expressions. Addressing these is planned as future work.

Citation

Please cite our paper with the following bibtex if you use our face renderer:

@inproceedings{chang17fpn,
      title={{F}ace{P}ose{N}et: Making a Case for Landmark-Free Face Alignment},
      booktitle = {7th IEEE International Workshop on Analysis and Modeling of Faces and Gestures, ICCV Workshops},
      author={
      Feng-ju Chang
      and Anh Tran 
      and Tal Hassner 
      and Iacopo Masi 
      and Ram Nevatia
      and G\'{e}rard Medioni},
      year={2017},
    }

References

[1] F.-J. Chang, A. Tran, T. Hassner, I. Masi, R. Nevatia, G. Medioni, "FacePoseNet: Making a Case for Landmark-Free Face Alignment", in 7th IEEE International Workshop on Analysis and Modeling of Faces and Gestures, ICCV Workshops, 2017

[2] I. Masi*, A. Tran*, T. Hassner*, J. Leksut, G. Medioni, "Do We Really Need to Collect Million of Faces for Effective Face Recognition? ", ECCV 2016, * denotes equal authorship

[3] I. Masi, S. Rawls, G. Medioni, P. Natarajan "Pose-Aware Face Recognition in the Wild", CVPR 2016

[4] T. Hassner, S. Harel, E. Paz and R. Enbar "Effective Face Frontalization in Unconstrained Images", CVPR 2015

[5] I. Masi, T. Hassner, A. Tran, and G. Medioni, "Rapid Synthesis of Massive Face Sets for Improved Face Recognition", FG 2017

Changelog

Disclaimer

The SOFTWARE PACKAGE provided in this page is provided "as is", without any guarantee made as to its suitability or fitness for any particular use. It may contain bugs, so use of this tool is at your own risk. We take no responsibility for any damage of any sort that may unintentionally be caused through its use.

Contacts

If you have any questions, drop an email to fengjuch@usc.edu, anhttran@usc.edu, iacopo.masi@usc.edu or hassner@isi.edu or leave a message below with GitHub (log-in is needed).