Home

Awesome

Facial Emotion Recognition with Apache MXNet

This repository demonstrates the implementation and deployoment of a facial expression recognition deep learning model using Apache MXNet, based on the FER+ paper by Barsoum et. al..

The repository consists of the following resources:

  1. Scripts for data pre-processing as suggested in the paper.
  2. Notebook for model building and training, using Apache MXNet.
  3. Model deployment for online inference, using MXNet Model Server

Check out a working FER+ web application demo, powered by AWS.

Before you start

Pre-requisites

# Install MXNet

pip install mxnet-mkl # for CPU machines
pip install mxnet-cu92 # for GPU machines with CUDA 9.2
    
# Other Dependencies

pip install Pillow # For image processing
pip install graphviz # For MXNet network visualization
pip install matplotlib # For plotting training graphs

# Install MXNet Model Server and required dependencies for inference model serving

pip install mxnet-model-server
pip install scikit-image
pip install opencv-python

Note: please refer to MXNet installation guide for more detailed installation instructions.

Data preparation

Clone this repository

git clone https://github.com/TalkAI/facial-emotion-recognition-gluon
cd facial-emotion-recognition-gluon

Download FER dataset fer2013.tar.gz from the FER Kaggle competition.

Note: You cannot download the dataset with wget. You will have to register on Kaggle, and then login to download the dataset.

Once downloaded:

We will now generate FER+ train/test/validation dataset from the downloaded FER data by executing the command below:

# In this step, we read the raw FER data, correct the labels using FER+ labels, and save as png images.

# -d : path to "data" folder in this repository. It has folder for Train/Test/Validation data with corrected labels.
# -fer : path to fer dataset that you have extracted.
# -ferplus : path to fer2013new.csv file that comes with this repository in the data folder
    
python utils/prepare_data.py -d ./data -fer ./data/fer2013.csv -ferplus ./data/fer2013new.csv

Lastly, we will process the FER+ train/test/validation dataset

# This script reads the FER+ dataset (png images) we prepared in the previous step, applies the transformation suggested in the FER+ paper, and saves the processed images as NumPy binaries (npy files).

# -d : path to data folder. This is where we have created data from the previous step.

python utils/process_data.py -d ./data

Deep Learning Basics You Will Need

Go over this notebook that provides basic overview and intuitiion for various deep learning concepts and techniaues used in building and training the model.

Model Building, Training and Deployment

Head over to the FER+ tutorial, to go over the process for building, training and deploying FER+ model. It is best to run as a live Jupyter Notebook, and you would need a GPU machine to complete training in a reasonable time.

Advanced Stuff - Left To The Reader

Below are few areas of improvements and next steps for the advanced reader. Contributions back to the repository are welcomed!

Contributors

Citation / Credits

Resources