Home

Awesome

VisSat Satellite Stereo

Note: this repo is not actively maintained; I've been trying to build a new full-stack satellite stereo pipelines in computer vision style: SatelliteSfM.

Introduction

This is the python interface for VISion-based SATellite stereo (VisSat) that is backed by our adapted COLMAP. You can run both SfM and MVS on a set of satellite images.

Project page: https://kai-46.github.io/VisSat/

Installation

pip3 install -r requirements.txt

Quick Start

python3 stereo_pipeline.py --config_file aoi_config/MVS3DM_Explorer.json

For Hackers

General Program Logic

We use a specific directory structure to help organize the program logic. The base directory is called {work_dir}. To help understand how the system works, let me try to point out what directory or files one should pay attention to at each stage of the program.

SfM stage

You need to enable {“clean_data”, “crop_image”, “derive_approx”, “choose_subset”, “colmap_sfm_perspective”} in the configuration. Then note the following files.

  1. (.ntf, .tar) pairs inside {dataset_dir}
  2. (.ntf, .xml) pairs inside {work_dir}/cleaned_data
  3. {work_dir}/aoi.json
  4. .png inside {work_dir}/images, and .json inside {work_dir}/metas
  5. .json inside {work_dir}/approx_camera, especially perspective_enu.json
  6. {work_dir}/colmap/subset_for_sfm/{images, perspective_dict.json}
  7. {work_dir}/colmap/sfm_perspective/init_ba_camera_dict.json

Step [1-4] transform the (.ntf, .tar) data into more accessible conventional formats. Step 5 approximates the RPC cameras with perspective cameras. Step [6-7] selects a subset of images (by default, all the images), performs bundle adjustment, and writes bundle-adjusted camera parameters to {work_dir}/colmap/sfm_perspective/init_ba_camera_dict.json. For perspective cameras in the .json files mentioned in Step [5-7], the camera parameters are organized as:

w, h, f_x, f_y, c_x, c_y, s, q_w, q_x, q_y, q_z, t_x, t_y, t_z

, where (w,h) is image size, (f_{x,y}, c_{x,y}, s) are camera intrinsics, q_{w,x,y,z} is the quaternion representation of the rotation matrix, and t_{x,y,z} is the translation vector.

Coordinate system

Our perspective cameras use the local ENU coordinate system instead of the global (lat, lon, alt) or (utm east, utm north, alt).

For conversion between (lat, lon, alt) and local ENU, please refer to: coordinate_system.py and latlonalt_enu_converter.py

For conversion between (lat, lon) and (utm east, utm north), please refer to: lib/latlon_utm_converter.py

MVS stage

To run MVS after the SfM stage is done, you need to enable {“reparam_depth”, “colmap_mvs”, “aggregate_3d”} or {“reparam_depth”, “colmap_mvs”, “aggregate_2p5d”}.

If you enable "aggregate_2p5d", you will be able to see the per-view DSM in {work_dir}/colmap/mvs/dsm.

Cite Our Work

@inproceedings{VisSat-2019,
 title={Leveraging Vision Reconstruction Pipelines for Satellite Imagery},
 author={Zhang, Kai and Sun, Jin and Snavely, Noah},
 booktitle={IEEE International Conference on Computer Vision Workshops},
 year={2019}
}

License

This software uses the 3-clause BSD license.