Home

Awesome

Local Connectivity-Based Density Estimation for Face Clustering

This repo contains an official implementation for CVPR'23 paper "Local Connectivity-Based Density Estimation for Face Clustering".

Introduction

The proposed clustering method employs density-based clustering, which maintains edges that have higher density. For this purpose, we propose a reliable density estimation algorithm based on local connectivity between K nearest neighbors (KNN). We effectively exclude negative pairs from the KNN graph based on the reliable density while maintaining sufficient positive pairs. Furthermore, we develop a pairwise connectivity estimation network to predict the connectivity of the selected edges.

Requirements

conda install pytorch==1.10.2 cudatoolkit=11.3 -c pytorch
pip install -r requirements.txt

Dataset

The data directory is constructed as follows:

data
  ├── ms1m
  |    ├── features
  |    |    ├── part1_test.bin
  |    |    ├── ...
  |    |    └── part9.test.bin
  |    ├── labels
  |    |    ├── part1_test.meta
  |    |    ├── ...
  |    |    └── part9.test.meta
  |    └── knns
  |         ├── part1_test
  |         |    └── faiss_k_80.npz
  |         ├── ...
  |         └── part9_test
  |              └── faiss_k_80.npz
  ├── deepfashion
  |    ├── features
  |    |    └── deepfashion_test.bin
  |    ├── labels
  |    |    └── deepfashion_test.meta
  |    └── knns
  |         └── deepfashion_test
  |              └── deepfashion_k40.npz
  └── ijb-b
       ├── 512.fea.npy
       ├── 512.labels.npy
       ├── knn.graph.512.bf.npy
       └── ...

We have used the data from following repositories.

For DeepFashion dataset, we construct kNN graph using faiss.

Run

To test for each dataset, simply run shell scripts.

sh inference_{dataset_name}.sh

Results on MS-Celeb-1M

1, 3, 5, 7, 9 mean different subset of the clustering benchmark. Detailed settings are on our paper.

Pairwise F-Score

Methods13579
CDP75.0270.7569.5168.6268.06
L-GCN78.6875.8374.2973.7172.99
LTC85.6682.4180.3278.9877.87
GCN(V+E)87.9384.0482.1080.4579.30
Clusformer88.2084.6082.7981.0379.91
STAR-FC91.9788.2886.1784.7083.46
Pair-Cls90.6786.9185.0683.5182.41
Ada-NETS92.7989.3387.5085.4083.99
Chen et al.93.2290.5189.0987.9386.94
Ours94.6491.9090.2788.6987.35

BCubed F-Score

Methods13579
CDP78.7075.8274.5873.6272.92
L-GCN84.3781.6180.1179.3378.60
LTC85.5283.0181.1079.8478.86
GCN(V+E)86.0982.8481.2480.0979.25
Clusformer87.1784.0582.3080.5179.95
STAR-FC-86.2684.1382.6381.47
Pair-Cls89.5486.2584.5583.4982.40
Ada-NETS91.4087.9886.0384.4883.28
Chen et al.92.1889.4388.0086.9286.06
Ours93.3690.7889.2888.1587.28

Results on IJB-B

$F_{512}, F_{1024}, F_{1845}$ mean different subset of the clustering benchmark. Detailed settings are on our paper.

Pairwise F-Score

Methods$F_{512}$$F_{1024}$$F_{1845}$
Pair-Cls84.483.382.7
Chen et al.80.873.259.1
Ours93.092.790.8

BCubed F-Score

Methods$F_{512}$$F_{1024}$$F_{1845}$
L-GCN83.383.381.4
DANet83.483.382.8
Chen et al.79.678.176.7
Ours85.485.284.8

Results on DeepFashion

MethodsPairwise F-ScoreBCubed F-Score
CDP28.2857.83
L-GCN28.8558.91
LTC29.1459.11
GCN(V+E)38.4760.06
Pair-Cls37.6762.17
Ada-NETS39.3061.05
Chen et al.40.9163.61
Ours41.7664.56

Acknowledgement

Some codes are based on the publicly available codebase https://github.com/yl-1993/learn-to-cluster.

Citation

@inproceedings{shin2023local,
  title={Local Connectivity-Based Density Estimation for Face Clustering},
  author={Shin, Junho and Lee, Hyo-Jun and Kim, Hyunseop and Baek, Jong-Hyeon and Kim, Daehyun and Koh, Yeong Jun},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2023}
}