Awesome
Image Lens Reprojection Tool
What is lens-reproject?
lens-reproject is a tool to reproject images taken with a known lens to a new lens. The process unprojects every pixel coordinate back to the light ray in spherical coordinates -- with the origin set to the center of projection -- after which it projects the light ray back onto the sensor using the new lens parameters. A variety of lenses is supported:
- Rectilinear (The default perspective projection lenses).
- Equisolid (Fisheye)
- Equidistant (Fisheye)
Prerequisites
On Linux systems, install libjpeg:
sudo apt install libjpeg-turbo8-dev
Build
You will need a C++ compiler and CMake. All other dependencies are included as submodules.
git clone https://github.com/IDLabMedia/image-lens-reproject.git
cd image-lens-reproject
git submodule update --init --recursive
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
cmake --build . --config Release
You will find the exectable files with the required library files next to in in a bin/ subdirectory.
Updating your local copy from Git
git pull origin main
git submodule update
CLI interface
This program has two ways of being used:
- through Blender-style JSON configuration files.
- by manually specifying the lens characteristics on the command line.
Originally, this tool was made to convert images generated by Blender, from one
lens type to another. However, as this tool had potential to be used in other
contexts, where no configuration files from Blender were available, simpliy
specifying which lens-type the input image has directly on the command-line
was very desirable. To specify the input image lens-type directly on the
command-line, use the --no-configs argument along with --i-... flags under
the Input optics section. When using equirectangular panoraminc input
images, the --rotation flag might prove useful.
Usage:
./bin/reproject [OPTION...]
Color processing options:
--exposure EV Exposure compensation in stops (EV) to brigthen or
darken the pictures. (default: 0.0)
--reinhard max Use reinhard tonemapping with given maximum value
(after exposure processing) on the output images.
(default: 1.0)
Filter files options:
--filter-prefix prefix Only include files starting with (default:
"")
--filter-suffix suffix Only include files ending with (default: "")
Input optics.
These are usually inferred by the config JSONs. When specifying
--no-configs, lens information needs to be passed through these
command line options options:
--i-rectilinear focal_length,sensor_width
Input rectilinear images with given
focal_length,sensor_width tuple.
--i-equisolid focal_length,sensor_width,fov
Input equisolid images with given
focal_length,sensor_width,fov tuple.
--i-equidistant fov Input equidistant images with given fov
value.
--i-equirectangular long_min,long_max,lat_min,lat_max (radians)
Input equirectangular images with given
longitude min,max and latitude min,max
value or 'full'.
Input/output options:
--input-cfg json-file Input JSON file containing lens and camera
settings of the input images.
--output-cfg json-file Output JSON file containing lens and camera
settings of the output images.
--no-configs width,height
Work without reading and writing config
files. Requires you to specify the input
lens through the input-optics flags
(staring with -i-...) and the expected
resolution of the input images here.
-i, --input-dir file Input directory containing images to
reproject.
--single file A single input file to convert.
-o, --output-dir file Output directory to put the reprojected
images.
--exr Output EXR files. Color and depth.
--png Output PNG files. Color only.
Output optics options:
--no-reproject Do not reproject at all.
--rectilinear focal_length,sensor_width
Output rectilinear images with given
focal_length,sensor_width tuple.
--equisolid focal_length,sensor_width,fov
Output equisolid images with given
focal_length,sensor_width,fov tuple.
--equidistant fov Output equidistant images with given fov
value.
--equirectangular longitude_min,longitude_max,latitude_min,latitude_max
Output equirectangular images with given
longitude min,max and latitude min,max
value or 'full'.
--rotation pan, pitch, roll (degrees)
Specify a rotation (default: 0.0)
Runtime options:
--skip-if-exists Skip if the output file already exists.
-j, --parallel threads Number of parallel images to process. (default:
1)
--dry-run Do not actually reproject images. Only produce
config.
-h, --help Show help
Sampling options:
-s, --samples number Number of samples per dimension for
interpolating (default: 1)
--nn Nearest neighbor interpolation
--bl Bilinear interpolation
--bc Bicubic interpolation (default)
--scale percentage Output scale, as a fraction of the input
size. It is recommended to increase
--samples to prevent aliassing in case you
are downscaling. Eg: --scale 0.5 --samples
2 or --scale 0.33334 --samples 3 or --scale
0.25 --samples 4. Final dimensions are
rounded towards zero. (default: 1.0)
--output-resolution width,height
A fixed output resolution. Overwrites the
behavior of the 'scale' parameter.
Configuration JSON
The configuration JSON files required in the input of the CLI interface are
flexible. The application will read in the JSON, extract the lens information
from the "camera", "resolution", and "sensor_size" keys in the JSON root
object, rewrite the values of those keys to match the new settings, and
finally write out the updated JSON to a new file. This way, all custom
information is left untouched and it is more easy to integrate it in other
pipelines and systems.
The "camera" object in the root object follows the Blender camera settings
structure. This is not always the clearest, but we chose it as we work a lot
with Blender and it was straightforward to export the camera and lens
information straight from Blender. Next we show the different lens
configuration templates for the JSON file:
rectilinear
{
"camera": {
"focal_length": 36.0,
"lens_unit": "MILLIMETERS",
"projection_matrix": [
[ 2.0, 0.0, 0.0, 0.0 ],
[ 0.0, 2.0, 0.0, 0.0 ],
[ 0.0, 0.0, 0.0, 0.0 ],
[ 0.0, 0.0, 0.0, 1.0 ]
],
"type": "PERSP"
},
"resolution": [ 2048, 2048 ],
"sensor_size": [ 36.0, 36.0 ]
}
(Note that more elements may be present in the root object, as the tool just ignores them, but will copy them over to the output JSON.)
equidistant
{
"camera": {
"type": "PANO",
"panorama_type": "FISHEYE_EQUIDISTANT",
"fisheye_fov": 3.1415927410125732
},
"resolution": [ 2048, 2048 ],
"sensor_size": [ 36.0, 36.0 ]
}
Note: This lens is not expressed by a focal length. Instead the lens perfectly fits a circle of projection on the sensor. The equidistant lens is only specified by a field of view (fov).
equisolid
{
"camera": {
"type": "PANO",
"panorama_type": "FISHEYE_EQUISOLID",
"lens": 12.5,
"fisheye_fov": 3.1415927410125732
},
"resolution": [ 2048, 2048 ],
"sensor_size": [ 36.0, 36.0 ]
}
Note: lens is the focal length of the lens, expressed in the same unit as the
sensor_size element (typically millimeters). The naming is taken from Blender.
License
See the LICENSE file.
Credits
To cite this paper:
<!-- {% raw %} -->@inproceedings{courteaux2022silvr,
title = {{SILVR: A Synthetic Immersive Large-Volume Plenoptic Dataset}},
author = {Courteaux, Martijn and Artois, Julie and De Pauw, Stijn and Lambert, Peter and Van Wallendael, Glenn},
year = {2022},
doi = {10.1145/3524273.3532890},
publisher = {Association for Computing Machinery},
url = {https://doi.org/10.1145/3524273.3532890},
address = {New York, NY, USA},
month = {jun},
numpages = {6},
isbn = {978-1-4503-9283-9/22/06},
booktitle = {13th ACM Multimedia Systems Conference (MMSys '22)},
location = {Athlone, Ireland}
}