Home

Awesome

Generating Talking Face Landmarks

The code for the paper "Generating Talking Face Landmarks from Speech."

You can find the project page here.

An improved version of this project can be found here.

Installation

The project depends on the following Python packages:

It also depends on the following packages:

The code has been tested on Ubuntu 16.04 and OS X Sierra and High Sierra.

Code Example

The generation code has the following arguments:

You can run the following code to test the system:

python generate.py -i test_samples/test1.flac -m models/D40_C3.h5 -d 1 -c 3 -o results/D40_C3_test1

Feature Extraction

You can run featureExtractor.py to extract features from videos directly. The arguments are as follows:

Usage:

python featureExtractor.py -vp path-to-video-files/ -sp path-to-shape-predictor-68-face-landmarks-dat -o output-file-name.hdf5

Training

The training code has the following arguments:

Usage:

python train.py -i path-to-hdf5-train-file/ -u number-of-hidden-units -d number-of-delay-frames -c number-of-context-frames -o output-folder-to-save-model-file

Citation

@inproceedings{eskimez2018generating,
  title={Generating talking face landmarks from speech},
  author={Eskimez, Sefik Emre and Maddox, Ross K and Xu, Chenliang and Duan, Zhiyao},
  booktitle={International Conference on Latent Variable Analysis and Signal Separation},
  pages={372--381},
  year={2018},
  organization={Springer}
}