Awesome
This repository contains code of our CVPR 2021 paper - "Learning Camera Localization via Dense Scene Matching" by Shitao Tang, Chengzhou Tang, Rui Huang, Siyu Zhu and Ping Tan.
This paper presents a new method for scene agnostic camera localization using dense scene matching (DSM), where a cost volume is constructed between a query image and a scene. The cost volume and the corresponding coordinates are processed by a CNN to predict dense coordinates. Camera poses can then be solved by PnP algorithms.
If you find this project useful, please cite:
@inproceedings{Tang2021Learning,
title={Learning Camera Localization via Dense Scene Matching},
author={Shitao Tang, Chengzhou Tang, Rui Huang, Siyu Zhu and Ping Tan},
booktitle={Computer Vision and Pattern Recognition (CVPR)},
year={2021}
}
Usage
Environment
- The codes are tested along with
- pytorch=1.4.0
- lmdb (optional)
- yaml
- skimage
- opencv
- numpy=1.17
- tensorboard
Installation
- Build PyTorch operations
cd libs/model/ops python setup.py install
- Build PnP algorithm
cd libs/utils/lm_pnp mkdir build cd build cmake .. make all
Train and Test
-
Download
You can download the trained models and label files for 7scenes, Cambridge, Scannet.
For 7scenes, you can use the prepared data in the following.
Chess Fire Heads Office Pumpkin Kitchen Stairs For Cambridge landmarks, you can download image files here, and depths here.
-
Test
Please refer to configs/7scenes.yaml for detailed explaination of how to set label file path and image file path.
- 7scenes
python tools/video_test.py --config configs/7scenes.yaml
- Camrbrige
python tools/video_test.py --config configs/cambridge.yaml
- 7scenes
-
Train
We use ResNet-FPN pretrained model.
python tools/train_net.py
License
The MIT License (MIT)
Copyright (c) 2022 Shitao Tang
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.