Awesome
ibug.face_alignment
2D facial landmark detector based on FAN [1] with some pretrained weights. Our training code is available in this repostory: https://github.com/hhj1897/fan_training.
Prerequisites
- Numpy:
$pip3 install numpy
- OpenCV:
$pip3 install opencv-python
- PyTorch:
$pip3 install torch torchvision
- ibug.face_detection (only needed by the test script): See this repository for details: https://github.com/hhj1897/face_detection.
How to Install
git clone https://github.com/hhj1897/face_alignment.git
cd face_alignment
pip install -e .
How to Test
- To test on live video:
python face_alignment_test.py [-i webcam_index]
- To test on a video file:
python face_alignment_test.py [-i input_file] [-o output_file]
How to Use
# Import the libraries
import cv2
from ibug.face_detection import RetinaFacePredictor
from ibug.face_alignment import FANPredictor
from ibug.face_alignment.utils import plot_landmarks
# Create a RetinaFace detector using Resnet50 backbone, with the confidence
# threshold set to 0.8
face_detector = RetinaFacePredictor(
threshold=0.8, device='cuda:0',
model=RetinaFacePredictor.get_model('resnet50'))
# Create a facial landmark detector
landmark_detector = FANPredictor(
device='cuda:0', model=FANPredictor.get_model('2dfan2_alt'))
# Load a test image. Note that images loaded by OpenCV adopt the B-G-R channel
# order.
image = cv2.imread('test.png')
# Detect faces from the image
detected_faces = face_detector(image, rgb=False)
# Detect landmarks from the faces
# Note:
# 1. The input image must be a byte array of dimension HxWx3.
# 2. The input face boxes must be a array of dimension Nx4, N being the
# number of faces. More columns are allowed, but only the first 4
# columns will be used (which should be the left, top, right, and
# bottom coordinates of the face).
# 3. The returned landmarks are stored in a Nx68x2 arrays, each row giving
# the X and Y coordinates of a landmark.
# 4. The returned scores are stored in a Nx68 array. Scores are usually
# within the range of 0 to 1, but could go slightly beyond.
landmarks, scores = landmark_detector(image, detected_faces, rgb=False)
# Draw the landmarks onto the image
for lmks, scs in zip(landmarks, scores):
plot_landmarks(image, lmks, scs, threshold=0.2)
References
[1] Bulat, Adrian, and Georgios Tzimiropoulos. "How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks)." In Proceedings of the IEEE International Conference on Computer Vision, pp. 1021-1030. 2017.