Home

Awesome

Notes 2023-01-12

What is FACSvatar? (v0.4.0-Alpha)

FACSvatar is An Open Source Modular Framework for Real-Time FACS based Facial Animation

Or in plain English:

Track facial expressions with any software and visualize that data on any avatar in real-time, powered by the FACS representation. No more need to modify your avatar to support your tracking software. All written in your favorite programming language, on any OS, and across machines.

Diagram FACS advantage Muscle image source.

FACSvatar demo 2018-09

(Above demo video link: https://www.youtube.com/watch?v=J2FvrIl-ypU)

Message to:

FACSvatar is already operable with:

Disclaimers: This is an open-source project, hopefully being flexible enough for your facial animation needs. This is not software supported by a company / commercially, but by users like you. If you need some new capability, you likely have to code it yourself (or ask/hire someone), but questions for guidance are always welcome (make a GitHub issue)! For commercial usage, please check the license page. Read more about FACSvatar's limitations (TODO doc link).

Full documentation

Read the Docs: https://facsvatar.readthedocs.io/

Paper

Please cite the following paper when using this framework in a paper:

van der Struijk, Stef and Huang, Hung-Hsuan and Mirzaei, Maryam Sadat and Nishida, Toyoaki "FACSvatar: An Open Source Modular Framework for Real-Time FACS based Facial Animation" In Proceedings of 18th ACM International Conference on Intelligent Virtual Agents (pp. 159-164). ACM, 2018.

New in v0.4.0-alpha (2020-07-??) TODO UNFINISHED

See all changelogs

Quickstart

FACSvatar is tested on Ubuntu and Windows, but should work on MacOS.

This quickstart has 2 parts:

  1. Start FACSvatar modules using Docker - modules in containers (see here for Python instructions)
  2. Visualize in Unity3D or Blender

Dockerized modules

<!-- <details><summary>Expand (CLICK ME)</summary> <p> -->
  1. Downloads - Go to the release page of this GitHub repo and download:

    • (Real-time only) openface_2.1.0_zeromq.zip
      • Unzip and execute download_models.sh or .ps1 to download trained models
    • Windows 7 / 8 / 10 Home version <2004 : unity_FACSvatar_standalone_docker-ip.zip
    • Windows 10 Home v2004+ / Pro / Enterprise / Education: unity_FACSvatar_standalone.zip
    • Windows / Linux / Mac: Unity3D editor (documentation)
    • Source code (zip / tar.gz) or download this repository with:
      • git clone https://github.com/NumesSanguis/FACSvatar.git
      • Press the green Clone or Download button on this page --> Download ZIP
  2. Docker Install - Let's you execute applications without worrying about OS or programming language.

  3. Docker Modules - Open a terminal (W7/8: cmd.exe / W10: PowerShell) and navigate to folder FACSvatar/modules, then execute:

    1. docker-compose pull (Downloads FACSvatar Docker containers)
    2. docker-compose up (Starts downloaded Docker containers)
  4. See visualization engine instructions

Offline version:

  1. Open a 2nd terminal in folder FACSvatar/modules and execute: docker-compose exec facsvatar_facsfromcsv bash
  2. Inside Docker container - Start facial animation with: python main.py --pub_ip facsvatar_bridge

With webcam for real-time (Windows-only for now):

  1. Navigate inside folder openface_x.x.x_zeromq
  2. (Windows 7/8/10 Home version <2004 - only) Get Docker machine ip by opening a 2nd terminal and execute: docker-machine ip (likely to be 192.168.99.100)
  3. (Windows 7/8/10 Home version <2004 - only) Open config.xml, change <IP>127.0.0.1</IP> to <IP>machine ip from step 3</IP> (<IP>192.168.99.100</IP>) and save and close.
  4. Double click OpenFaceOffline.exe –> menu: File –> Open Webcam
<!-- </p> </details> -->

Visualization engines

Unity3D

Tested on version: 2018.2.20f1

  1. Open the folder unity_FACSvatar as a project with Unity3D
  2. Press play (now it's waiting for facial data)

OR (Windows-only TODO):

  1. Navigate inside unzipped folder unity_FACSvatar_standalone(_docker-ip) and double-click unity_FACSvatar.exe

Extra: Use the numbers 0, 1, 2 on your keyboard to change camera.

FACSvatar Blender add-on

Follow instructions here: https://github.com/NumesSanguis/FACSvatar-Blender

Quickstart video

See the quickstart video (:warning: note that the Blender script part is outdated (from 15:15) due the new FACSvatar Blender add-on):

FACSvatar Quickstart 2019-01 (v0.3.4)

Find out more

Full documentation

Read the FACSvatar documentation!

2017 promotion poster (English & 日本語)

FACSvatar details in English and 日本語