Awesome

Self-Supervised Learning of Image Scale and Orientation Estimation (BMVC 2021)

This is the official implementation of the paper "Self-Supervised Learning of Image Scale and Orientation Estimation" by Jongmin Lee [Google Scholar], Yoonwoo Jeong [Google Scholar], and Minsh Cho [Google Scholar]. We introduce a self-supervised framework for learning patch pose. Given a rescaled/rotated pair of image patches, we feed them to the patch pose estimation networks that output scale/orientation histograms for each. We compare the output histogram vectors by the histogram alignment technique and compute the loss.

Requirements

Ubuntu 18.04
python 3.8
pytorch 1.8.1
torchvision 0.9.1
wandb 0.10.28

Environment

Clone the Git repository

git clone https://github.com/bluedream1121/SelfScaOri.git

Install dependency

Run the script to install all the dependencies. You need to provide the conda install path (e.g. ~/anaconda3) and the name for the created conda environment.

bash install.sh conda_install_path self-sca-ori

Dataset preparation

You can download the training/test dataset using the following scripts:

cd datasets
bash download.sh

If the upper link does not work, please use this link patchPose, hpatchesPose. (password: self)

If you want to regenerate the patchPose datasets, please run the following script:

cd datasets/patchpose_dataset_generation
bash generation_script.sh

Trained models

cd trained_models
bash download_ori_model.sh
bash download_sca_model.sh

Test on the patchPose and the HPatches

After download the datasets and the pre-trained models, you can evaluate the patch pose estimation results using the following scripts:

python test.py --load trained_models/_*branchori/best_model.pt  --dataset_type ppa_ppb
python test.py --load trained_models/_*branchsca/best_model.pt  --dataset_type ppa_ppb

python test.py --load trained_models/_*branchori/best_model.pt  --dataset_type hpa
python test.py --load trained_models/_*branchsca/best_model.pt  --dataset_type hpa

Training

You can train the networks for patch scale estimation and orientation estimation using the proposed histogram alignment loss as follows:

python train.py --branch ori --output_ori 36

python train.py --branch sca --output_sca 13

Citation

If you find our code or paper useful to your research work, please consider citing our work using the following bibtex:

@inproceedings{lee2021self,
  title={Self-Supervised Learning of Image Scale and Orientation},
  author={Lee, Jongmin and Jeong, Yoonwoo and Cho, Minsu },
  booktitle     = {31st British Machine Vision Conference 2021, {BMVC} 2021, Virtual Event, UK},
  publisher = {{BMVA} Press},
  year      = {2021},
  url       = {https://www.bmvc2021-virtualconference.com/programme/accepted-papers/},
}

Contact

Jongmin Lee (ljm1121@postech.ac.kr)

Questions can also be left as issues in the repository.