Home

Awesome

AdaFace: Quality Adaptive Margin for Face Recognition

Official github repository for AdaFace: Quality Adaptive Margin for Face Recognition. The paper (https://arxiv.org/abs/2204.00964) is presented in CVPR 2022 (Oral).

Abstract: Recognition in low quality face datasets is challenging because facial attributes are obscured and degraded. Advances in margin-based loss functions have resulted in enhanced discriminability of faces in the embedding space. Further, previous studies have studied the effect of adaptive losses to assign more importance to misclassified (hard) examples. In this work, we introduce another aspect of adaptiveness in the loss function, namely the image quality. We argue that the strategy to emphasize misclassified samples should be adjusted according to their image quality. Specifically, the relative importance of easy and hard samples should be based on the sample's image quality. We propose a new loss function that emphasizes samples of different difficulty based on their image quality. Our method achieves this in the form of an adaptive margin function by approximating the image quality with feature norms. Extensive experiments show that our method, AdaFace, improves the face recognition performance over the state-of-the-art (SoTA) on four datasets (IJB-B, IJB-C, IJB-S and TinyFace).

@inproceedings{kim2022adaface,
  title={AdaFace: Quality Adaptive Margin for Face Recognition},
  author={Kim, Minchul and Jain, Anil K and Liu, Xiaoming},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2022}
}
<img src="assets/main_figure.png"/>

PWC PWC PWC PWC PWC

Repository Updates

News

5 Minute Video Presentation

IMAGE ALT TEXT

Demo Comparison between AdaFace and ArcFace on Low Quality Images

Demo The demo shows a comparison between AdaFace and ArcFace on a live video. To show how model performs with low quality images, we show original, blur+ and blur++ setting where blur++ means it is heavily blurred. The numbers with colorbox show the cosine similarity between the live image and the cloest matching gallery image. The statistics on the bottom show the cumulative count of true positive match for blur++ setting. AdaFace has high true positive rate. It also shows it is less prone to making false positive (red) mistakes as sometimes observed in ArcFace.

Usage

import torch
from head import AdaFace

# typical inputs with 512 dimension
B = 5
embbedings = torch.randn((B, 512)).float()  # latent code
norms = torch.norm(embbedings, 2, -1, keepdim=True)
normalized_embedding  = embbedings / norms
labels =  torch.randint(70722, (B,))

# instantiate AdaFace
adaface = AdaFace(embedding_size=512,
                  classnum=70722,
                  m=0.4,
                  h=0.333,
                  s=64.,
                  t_alpha=0.01,)

# calculate loss
cosine_with_margin = adaface(normalized_embedding, norms, labels)
loss = torch.nn.CrossEntropyLoss()(cosine_with_margin, labels)

Installation

conda create --name adaface pytorch==1.8.0 torchvision==0.9.0 cudatoolkit=10.2 -c pytorch
conda activate adaface
conda install scikit-image matplotlib pandas scikit-learn 
pip install -r requirements.txt

Train (Preapring Dataset and Training Scripts)

Pretrained Models

Note that our pretrained model takes the input in BGR color channel. This is different from the InsightFace released model which uses RGB color channel.

ArchDatasetLink
R18CASIA-WebFacegdrive
R18VGGFace2gdrive
R18WebFace4Mgdrive
R50CASIA-WebFacegdrive
R50WebFace4Mgdrive
R50MS1MV2gdrive
R100MS1MV2gdrive
R100MS1MV3gdrive
R100WebFace4Mgdrive
R100WebFace12Mgdrive

Inferece

Example using provided sample images

AdaFace takes input images that are preproccsed. The preprocessing step involves

  1. aligned with facial landmark (using MTCNN) and
  2. cropped to 112x112x3 size whose color channel is BGR order.

We provide the code for performing the preprocessing step. For using pretrained AdaFace model for inference,

  1. Download the pretrained adaface model and place it in pretrained/

  2. For using pretrained AdaFace on below 3 images, run

python inference.py
img1img2img3
<img src="face_alignment/test_images/img1.jpeg" width="215" /><img src="face_alignment/test_images/img2.jpeg" width="130" /><img src="face_alignment/test_images/img3.jpeg" width="191" />

The similarity score result should be

tensor([[ 1.0000,  0.7334, -0.0655],
        [ 0.7334,  1.0000, -0.0277],
        [-0.0655, -0.0277,  1.0000]], grad_fn=<MmBackward0>)

General Inference Guideline

In a nutshell, inference code looks as below.

from face_alignment import align
from inference import load_pretrained_model, to_input

model = load_pretrained_model('ir_50')
path = 'path_to_the_image'
aligned_rgb_img = align.get_aligned_face(path)
bgr_input = to_input(aligned_rgb_img)
feature, _ = model(bgr_input)

Validation

High Quality Image Validation Sets (LFW, CFPFP, CPLFW, CALFW, AGEDB)

For evaluation on 5 HQ image validation sets with pretrained models, refer to

bash validation_hq/eval_5valsets.sh
ArchDatasetMethodLFWCFPFPCPLFWCALFWAGEDBAVG
R18CASIA-WebFaceAdaFace0.99130.92590.87000.92650.92720.9282
R18VGGFace2AdaFace0.99470.97130.91720.93900.94070.9526
R18WebFace4MAdaFace0.99530.97260.92280.95520.96470.9621
R50CASIA-WebFaceAdaFace0.99420.96410.89970.93230.94380.9468
R50MS1MV2AdaFace0.99820.97860.92830.96070.97850.9688
R50WebFace4MAdaFace0.99780.98970.94170.95980.97780.9734
R100MS1MV2AdaFace0.99820.98490.93530.96080.98050.9719
R100MS1MV3AdaFace0.99780.98910.93930.96020.98170.9736
R100WebFace4MAdaFace0.99800.99170.94630.96050.97900.9751
R100WebFace12MAdaFace0.99820.99260.94570.96120.98000.9755

Comparison with Other Methods

ArchDatasetMethodLFWCFPFPCPLFWCALFWAGEDBAVG
R50CASIA-WebFaceAdaFace0.99420.96410.89970.93230.94380.9468
R50CASIA-WebFace(ArcFace)0.99450.9521NANA0.9490NA
R100MS1MV2AdaFace0.99820.98490.93530.96080.98050.9719
R100MS1MV2(ArcFace)0.9982NA0.92080.9545NANA

Mixed Quality Scenario (IJBB, IJBC Dataset)

For IJBB, IJBC validation, refer to

cd validation_mixed
bash eval_ijb.sh
ArchDatasetMethodIJBB TAR@FAR=0.01%IJBC TAR@FAR=0.01%
R18VGG2AdaFace90.6792.95
R18WebFace4MAdaFace93.0394.99
R50WebFace4MAdaFace95.4496.98
R50MS1MV2AdaFace94.8296.27
R100MS1MV2AdaFace95.6796.89
R100MS1MV3AdaFace95.8497.09
R100WebFace4MAdaFace96.0397.39
R100WebFace12MAdaFace96.4197.66

Comparison with Other Methods

ArchDatasetMethodVenueIJBB TAR@FAR=0.01%IJBC TAR@FAR=0.01%
R100MS1MV2AdaFaceCVPR2295.6796.89
R100MS1MV2(MagFace)CVPR2194.5195.97
R100MS1MV2(SCF-ArcFace)CVPR2194.7496.09
R100MS1MV2(BroadFace)ECCV2094.9796.38
R100MS1MV2(CurricularFace)CVPR2094.8096.10
R100MS1MV2(MV-Softmax)AAAI2093.6095.20
R100MS1MV2(AFRN)ICCV1988.5093.00
R100MS1MV2(ArcFace)CVPR1994.2596.03
R100MS1MV2(CosFace)CVPR1894.8096.37
ArchDatasetMethodIJBC TAR@FAR=0.01%
R100WebFace4MAdaFace97.39
R100WebFace4M(CosFace)96.86
R100WebFace4M(ArcFace)96.77
R100WebFace4M(CurricularFace)97.02
ArchDatasetMethodIJBC TAR@FAR=0.01%
R100WebFace12MAdaFace97.66
R100WebFace12M(CosFace)97.41
R100WebFace12M(ArcFace)97.47
R100WebFace12M(CurricularFace)97.51

Low Quality Scenario (IJBS)

For IJBB, IJBC validation, refer to

cd validation_lq
python validate_IJB_S.py

Comparison with Other Methods

Sur-to-SingleSur-to-BookSur-to-SurTinyFace
ArchMethodDatasetRank1Rank51%Rank1Rank51%Rank1Rank51%rank1rank5
R100AdaFaceWebFace12M71.3576.2459.3971.9376.5659.3736.7150.034.6272.2974.97
R100AdaFaceWebFace4M70.4275.2958.2770.9376.1158.0235.0548.224.9672.0274.52
R100AdaFaceMS1MV265.2670.5351.6666.2771.6150.8723.7437.472.5068.2171.54
R100(CurricularFace)MS1MV262.4368.6847.6863.8169.7447.5719.5432.802.5363.6867.65
R100(URL)MS1MV258.9465.4837.5761.9867.1242.73NANANA63.8968.67
R100(ArcFace)MS1MV257.3564.4241.8557.3664.9541.23NANANANANA
R100(PFE)MS1MV250.1658.3331.8853.6061.7535.999.2020.820.84NANA