Home

Awesome

SSR-Net

[IJCAI18] SSR-Net: A Compact Soft Stagewise Regression Network for Age Estimation

Code Author: Tsun-Yi Yang

Last update: 2019/09/19 (Renew the morph2 dataset link)

<img src="https://media.giphy.com/media/ygBDe4FIU4Cybbfh2N/giphy.gif" height="240"/> <img src="https://media.giphy.com/media/bZvHMOp2hBsusr96fa/giphy.gif" height="240"/>

<img src="https://github.com/shamangary/SSR-Net/blob/master/demo/TGOP_tvbs.png" height="240"/> <img src="https://github.com/shamangary/SSR-Net/blob/master/demo/the_flash_cast.png" height="240"/>

<img src="https://github.com/shamangary/SSR-Net/blob/master/table1.png" height="240"/>

Real-time webcam demo

<img src="https://media.giphy.com/media/AhXTjtGO9tnsyi2k6Q/giphy.gif" height="240"/> <img src="https://github.com/shamangary/SSR-Net/blob/master/age_gender_demo.png" height="240"/>

Paper

PDF

https://github.com/shamangary/SSR-Net/blob/master/ijcai18_ssrnet_pdfa_2b.pdf

Paper authors

Tsun-Yi Yang, Yi-Husan Huang, Yen-Yu Lin, Pi-Cheng Hsiu, and Yung-Yu Chuang

Abstract

This paper presents a novel CNN model called Soft Stagewise Regression Network (SSR-Net) for age estimation from a single image with a compact model size. Inspired by DEX, we address age estimation by performing multi-class classification and then turning classification results into regression by calculating the expected values. SSR-Net takes a coarse-to-fine strategy and performs multi-class classification with multiple stages. Each stage is only responsible for refining the decision of the previous stage. Thus, each stage performs a task with few classes and requires few neurons, greatly reducing the model size. For addressing the quantization issue introduced by grouping ages into classes, SSR-Net assigns a dynamic range to each age class by allowing it to be shifted and scaled according to the input face image. Both the multi-stage strategy and the dynamic range are incorporated into the formulation of soft stagewise regression. A novel network architecture is proposed for carrying out soft stagewise regression. The resultant SSR-Net model is very compact and takes only 0.32 MB. Despite of its compact size, SSR-Net’s performance approaches those of the state-of-the-art methods whose model sizes are more than 1500x larger.

Platform

Dependencies

pip install mtcnn

conda install -c conda-forge moviepy
conda install -c cogsci pygame
conda install -c conda-forge requests
conda install -c conda-forge pytables

Codes

There are three different section of this project.

  1. Data pre-processing
  2. Training and testing
  3. Video demo section We will go through the details in the following sections.

This repository is for IMDB, WIKI, and Morph2 datasets.

1. Data pre-processing

cd ./data
python TYY_IMDBWIKI_create_db.py --db imdb --output imdb.npz
python TYY_IMDBWIKI_create_db.py --db wiki --output wiki.npz
python TYY_MORPH_create_db.py --output morph_db_align.npz

2. Training and testing

<img src="https://github.com/shamangary/SSR-Net/blob/master/merge_val_morph2.png" height="300"/>

The experiments are done by randomly choosing 80% of the dataset as training and 20% of the dataset as validation (or testing). The details of the setting in each dataset is in the paper.

For MobileNet and DenseNet:

cd ./training_and_testing
sh run_all.sh

For SSR-Net:

cd ./training_and_testing
sh run_ssrnet.sh

Plot the results: For example, after the training of IMDB dataset, you want to plot the curve and the results. Copy "plot.sh", "ssrnet_plot.sh", and "plot_reg.py" into "./imdb_models". The following command should plot the results of the training process.

sh plot.sh
sh ssrnet_plot.sh

3. Video demo section

Pure CPU demo command:

cd ./demo
KERAS_BACKEND=tensorflow CUDA_VISIBLE_DEVICES='' python TYY_demo_mtcnn.py TGOP.mp4

# Or you can use

KERAS_BACKEND=tensorflow CUDA_VISIBLE_DEVICES='' python TYY_demo_mtcnn.py TGOP.mp4 '3'

Real-time webcam demo

Considering the face detection process (MTCNN or Dlib) is not fast enough for real-time demo. We show a real-time webcam version by using lbp face detector.

cd ./demo
KERAS_BACKEND=tensorflow CUDA_VISIBLE_DEVICES='' python TYY_demo_ssrnet_lbp_webcam.py

4. Extension

Training the gender model

We can reformulate binary classification problem into regression problem, and SSR-Net can be used to predict the confidence. For example, we provide gender regression and demo in the code for the extension.

Training the gender network:

cd ./training_and_testing
sh run_ssrnet_gender.sh

Note that the score can be between [0,1] and the 'V' inside SSR-Net can be changed into 1 for general propose regression.

Third Party Implementation