Home

Awesome

Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning

Official PyTorch implementation of "Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning" (AAAI 2021)

Geonmo Gu*<sup>1</sup>, Byungsoo Ko*<sup>1</sup>, Han-Gyu Kim<sup>2</sup> (* Authors contributed equally.)

<sup>1</sup>@NAVER/LINE Vision, <sup>2</sup>@NAVER Clova Speech

Overview

Proxy Synthesis

<img src="figures/overview.png">

How it works?

<img src="figures/tsne.png">

Experimental results

<p align="center"> <img src="figures/evaluation.png" width="400"> </p>

Getting Started

Installation

  1. Clone the repository locally
$ git clone https://github.com/navervision/proxy-synthesis
  1. Create conda virtual environment
$ conda create -n proxy_synthesis python=3.7 anaconda
$ conda activate proxy_synthesis
  1. Install pytorch
$ conda install pytorch torchvision cudatoolkit=<YOUR_CUDA_VERSION> -c pytorch
  1. Install faiss
$ conda install faiss-gpu cudatoolkit=<YOUR_CUDA_VERSION> -c pytorch
  1. Install requirements
$ pip install -r requirements.txt

Prepare Data

$ wget http://ai.stanford.edu/~jkrause/car196/cars_annos.mat
$ tar zxvf car_ims.tgz -C ./dataset
# Dataset structure
/dataset/carDB/
  train/
    class1/
      img1.jpeg
    class2/
      img2.jpeg
  test/
    class1/
      img3.jpeg
    class2/
      img4.jpeg
# Rearrange dataset structure
$ python dataset/prepare_cars.py

Train models

Norm-SoftMax loss with CARS196

# Norm-SoftMax
$ python main.py --gpu=0 \
--save_path=./logs/CARS196_norm_softmax \
--data=./dataset/carDB --data_name=cars196 \
--dim=512 --batch_size=128 --epochs=130 \
--freeze_BN --loss=Norm_SoftMax \
--decay_step=50 --decay_stop=50 --n_instance=1 \
--scale=23.0 --check_epoch=5

PS + Norm-SoftMax loss with CARS196

# PS + Norm-SoftMax
$ python main.py --gpu=0 \
--save_path=./logs/CARS196_PS_norm_softmax \
--data=./dataset/carDB --data_name=cars196 \
 --dim=512 --batch_size=128 --epochs=130 \
--freeze_BN --loss=Norm_SoftMax \
--decay_step=50 --decay_stop=50 --n_instance=1 \
--scale=23.0 --check_epoch=5 \
--ps_alpha=0.40 --ps_mu=1.0

Proxy-NCA loss with CARS196

# Proxy-NCA
$ python main.py --gpu=0 \
--save_path=./logs/CARS196_proxy_nca \
--data=./dataset/carDB --data_name=cars196 \
--dim=512 --batch_size=128 --epochs=130 \
--freeze_BN --loss=Proxy_NCA \
--decay_step=50 --decay_stop=50 --n_instance=1 \
--scale=12.0 --check_epoch=5

PS + Proxy-NCA loss with CARS196

# PS + Proxy-NCA
$ python main.py --gpu=0 \
--save_path=./logs/CARS196_PS_proxy_nca \
--data=./dataset/carDB --data_name=cars196 \
--dim=512 --batch_size=128 --epochs=130 \
--freeze_BN --loss=Proxy_NCA \
--decay_step=50 --decay_stop=50 --n_instance=1 \
--scale=12.0 --check_epoch=5 \
--ps_alpha=0.40 --ps_mu=1.0

Check Test Results

$ tensorboard --logdir=logs --port=10000

Experimental results

Recall@1

Loss12345678Mean ± std
Norm-SoftMax83.3883.2583.2583.1883.0582.9082.8382.7983.08 ± 0.21
PS + Norm-SoftMax84.6984.5884.4584.3584.2283.9583.9183.8984.25 ± 0.31
Proxy-NCA83.7483.6983.6283.3283.0683.0082.9782.8483.28 ± 0.36
PS + Proxy-NCA84.5284.3984.3284.2984.2284.1283.9483.8884.21 ± 0.21

RP

Loss12345678Mean ± std
Norm-SoftMax35.8535.5135.2835.2835.2434.9534.8734.8435.23 ± 0.34
PS + Norm-SoftMax37.0136.9836.9236.7436.7436.7336.5436.4536.76 ± 0.20
Proxy-NCA36.0835.8535.7935.6635.6635.6335.4735.4335.70 ± 0.21
PS + Proxy-NCA36.9736.8436.7236.6436.6336.6036.4336.4136.66 ± 0.18

MAP

Loss12345678Mean ± std
Norm-SoftMax25.5625.5625.0024.9324.9024.5924.5724.5624.92 ± 0.35
PS + Norm-SoftMax26.7126.6726.6526.5626.5326.5226.3026.1726.51 ± 0.18
Proxy-NCA25.6625.5225.3725.3625.3325.2625.2225.0425.35 ± 0.18
PS + Proxy-NCA26.7726.6326.5026.4226.3726.3126.2526.1226.42 ± 0.20

Performance Graph

Reference

Citation

If you find Proxy Synthesis useful in your research, please consider to cite the following paper.

@inproceedings{gu2020proxy,
    title={Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning},
    author={Geonmo Gu, Byungsoo Ko, and Han-Gyu Kim},
    booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
    year={2021}
}

License

Copyright 2021-present NAVER Corp.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.