Awesome
EventEMin
Code for the following papers:
- Entropy Minimisation Framework for Event-based Vision Model Estimation
- Robust Event-based Vision Model Estimation by Dispersion Minimisation
The authors provide this code in the hope it will be useful for understanding the proposed method, as well as for reproducibility of the results.
For more information and more open-source software please visit the Personal Robotic Lab's website: https://www.imperial.ac.uk/personal-robotics/.
Requirements
This code was tested on Ubuntu 16.04, 18.04 and 20.04 distros.
Dependencies
- CMake: https://cmake.org/download/
sudo apt install cmake cmake-curses-gui
-
OpenCV: https://docs.opencv.org/trunk/d7/d9f/tutorial_linux_install.html
-
Eigen3: http://eigen.tuxfamily.org/index.php?title=Main_Page#Download
git clone https://gitlab.com/libeigen/eigen.git
cd eigen
mkdir build && cd build
cmake ..
sudo make install
or
sudo apt install libeigen3-dev
This repo was originally developed using Eigen 3.3.7. Currently, there are some compilation issues between Eigen 3.4.0 and the AutoDiff module. So, you should compile this repo with at most Eigen 3.3.9.
- GSL - GNU (only used for the batch mode): https://www.gnu.org/software/gsl/
git clone https://github.com/ampl/gsl.git
cd gsl
./configure
sudo make install
- OpenMP (optional): https://www.openmp.org/
sudo apt install libomp-dev
General
git clone https://github.com/ImperialCollegeLondon/EventEMin.git
cd EventEMin
mkdir build && cd build
cmake .. <cmake arguments>
The cmake arguments can be set as follows:
-DEventEMin_BATCH_MODE=ON/OFF
Build batch mode.
(default: OFF)
-DEventEMin_INCREMENTAL_MODE=ON/OFF
Build incremental mode.
(default: OFF)
-DEventEMin_FAST_EXP=ON/OFF
Use fast exponentiation.
(default: ON)
-DEventEMin_USE_OPENMP=ON/OFF
Uses the OpenMP library for parallelization.
(default: ON)
Lastly, ensure all environment path variables are well set, and compile everything:
make
ROS
A ROS package for real-time incremental estimation can be found on the following repository: https://github.com/ImperialCollegeLondon/event_emin_ros.
Datasets
We provide samples in the dataset folder, corresponding to some results obtained in the paper. Most of the examples provided work with these samples, so you are not required to download any dataset.
Examples
The source files are located in the test directory and the binary files will be located in the bin directory. We provide estimation examples per model, and the dispersion measure to be used can be chosen on the corresponding source file. Please note that the exact entropy-based measures have quadratic complexity with the number of events and the respective examples are expected to take longer (especially if you do not use OpenMP).
Batch Mode
For each example, two images should be seen, corresponding to the original events and the transformed events, according to the estimated parameters, accumulated on the image plane. The status of the optimisation procedure should be displayed in the following format:
iteration, restart iteration: score, gradient magnitude
gradient
parameters
In the end, the estimated parameters are also displayed.
2D Translation Estimation
To run the example, on a terminal type:
./example_translation2d
Rotation Estimation
To run the example, on a terminal type:
./example_rotation
Motion Estimation in Planar Scenes
To run the example, on a terminal type:
./example_homography
6-DOF in 3D
To run the example, on a terminal type:
./example_6dof
Incremental Mode
For each example, two images should be seen, corresponding to the original events and the transformed events, according to the estimated parameters, accumulated on the image plane. The timestamp and corresponding estimates should be displayed in the following format:
ts: timestamp, v: motion parameter estimates
In the end, the estimated parameters are also displayed.
2D Translation Estimation
To run the example, on a terminal type:
./example_incremental_translation2d
Rotation Estimation
To run the example, on a terminal type:
./example_incremental_rotation
6-DOF in 3D
To run the example, on a terminal type:
./example_incremental_6dof
Test Sequences
To run the sequences test, you need to download at least one sequence of the dataset provided in http://rpg.ifi.uzh.ch/davis_data.html.
Batch Mode
2D
To run the test, on a terminal type:
./example_test_sequence <path-to-events-dir> <batch-size> <path-to-estimates-saving-dir> <estimates-file-name>
The executable arguments are as follows:
- path-to-events-dir:
Path to the events' directory, following the format proposed in http://rpg.ifi.uzh.ch/davis_data.html, e.g.
/poster_rotation
. The events' directory must contain two files, namely,events.txt
(list of events) andcalib.txt
(camera parameters). Please check the folders under dataset for examples. - batch-size:
Number of events of each batch, e.g.
20000
. - path-to-estimates-saving-dir:
Path to where the estimates are to be stored, e.g.
/poster_rotation/estimates
. - estimates-file-name:
File name of the estimates, e.g.
approx_tsallis_2
.
For example, if you downloaded the poster_rotation
sequence and stored it under foo
directory, by running
./example_test_sequence /foo/poster_rotation 20000 /foo/poster_rotation/estimates approx_tsallis_2
a file containig the estimates using the Approx. Tsallis measure should be created under the /foo/poster_rotation/estimates
directory (/estimates
directory should be created before running the command).
3D
To run the test, on a terminal type:
./example_test_sequence_3d <path-to-events-dir> <batch-size> <minimum-depth> <maximum-depth> <path-to-estimates-saving-dir> <estimates-file-name>
The additional executable arguments are as follows:
- minimum-depth: Minimum depth of the depth-augmented events.
- maximum-depth: Maximum depth of the depth-augmented events.
The rest of the arguments are the same as previously. The events should already be undistorted and augmented with depth. See indoor_flying1 for an example.
Incremental Mode
2D
To run the test, on a terminal type:
./example_incremental_test_sequence <path-to-events-dir> <number-events> <path-to-estimates-saving-dir> <estimates-file-name>
The executable arguments are as follows:
- path-to-events-dir:
Path to the events' directory, following the format proposed in http://rpg.ifi.uzh.ch/davis_data.html, e.g.
/poster_rotation
. The events' directory must contain two files, namely,events.txt
(list of events) andcalib.txt
(camera parameters). Please check the folders under dataset for examples. - number-events:
Number of the most recent events to maintain, e.g.
10000
. - path-to-estimates-saving-dir:
Path to where the estimates are to be stored, e.g.
/poster_rotation/estimates
. - estimates-file-name:
File name of the estimates, e.g.
incremental_potential
.
For example, if you downloaded the poster_rotation
sequence and stored it under foo
directory, by running
./example_incremental_test_sequence /foo/poster_rotation 10000 /foo/poster_rotation/estimates incremental_potential
a file containig the estimates using the Incremental Potential measure should be created under the /foo/poster_rotation/estimates
directory (/estimates
directory should be created before running the command).
3D
To run the test, on a terminal type:
./example_incremental_test_sequence_3d <path-to-events-dir> <number-events> <minimum-depth> <depth-scale> <path-to-estimates-saving-dir> <estimates-file-name>
The additional executable arguments are as follows:
- minimum-depth: Minimum depth of the depth-augmented events.
- depth-scale: Depth scaling factor.
The rest of the arguments are the same as previously. The events should already be undistorted and augmented with depth. See indoor_flying1 for an example.
Compute Errors
To compute the errors for rotational motion estimation, run the MATLAB script sequence_error.m.
License
The EventEMin code is licensed under CC BY-NC-SA 4.0. Commercial usage is not permitted.