Home

Awesome

Build Status

EDS: Event-aided Direct Sparse Odometry

Direct Sparse Odometry

This is the code for the paper Event-aided Direct Sparse Odometry (CVPR 2022 oral) by
Javier Hidalgo-Carrió, Guillermo Callego, and Davide Scaramuzza.

If you use this work in your research, please cite it as follows:

@@InProceedings{Hidalgo2022cvpr,
  author = {Javier Hidalgo-Carri{\'{o}} and Guillermo Gallego and Davide Scaramuzza},
  title = {Event-aided Direct Sparse odometry},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2022}
}

Project Page and Dataset

General Page: https://rpg.ifi.uzh.ch/eds.html

EDS Dataset and Calibration Details

Beamsplitter Design

License

This source code is GPLv3 license. See the LICENSE file for further details.

Installation

  1. Make sure that the Ruby interpreter is installed on your machine. Rock requires ruby 2.3 or higher, which is provided on Debian and Ubuntu by the ruby2.3 package. This installation process is tested with Ruby 2.7 on Ubuntu 20.04.
docker@javi$ ruby --version
ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x86_64-linux-gnu]
docker@javi$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.4 LTS
Release:	20.04
Codename:	focal
  1. Create and “cd” into the directory in which you want to install the toolchain.
docker@javiS mkdir rock && cd rock && mkdir dev && cd dev
docker@dev$ pwd
/home/javi/rock/dev
  1. To build EDS, use this bootstrap.sh script. Save it in the folder you just created.
docker@dev:$ wget https://raw.githubusercontent.com/uzh-rpg/eds-buildconf/master/bootstrap.sh
  1. In a console run
docker@dev:$ sh bootstrap.sh
  1. Follow the installation guide and answer the questions. In case you hesitate choose the answer by default.
Connecting to www.rock-robotics.org (www.rock-robotics.org)|185.199.111.153|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 30078 (29K) [application/octet-stream]
Saving to: 'autoproj_bootstrap'

autoproj_bootstrap       100%[=================================>]  29.37K  --.-KB/s    in 0.007s  

2022-04-13 15:58:06 (3.97 MB/s) - 'autoproj_bootstrap' saved [30078/30078]

Which protocol do you want to use to access uzh-rpg/eds-buildconf.git on github.com?
[git|ssh|http] (default: http) 


So, what do you want ? (all, none or a comma-separated list of: os gem pip) [all] 
Would you like autoproj to keep apt packages up-to-date? [yes] 

The current directory is not empty, continue bootstrapping anyway ? [yes] 


How should I interact with github.com (git, http, ssh)
If you give one value, it's going to be the method used for all access
If you give multiple values, comma-separated, the first one will be
used for pulling and the second one for pushing. An optional third value
will be used to pull from private repositories (the same than pushing is
used by default) [http,ssh]

Which flavor of Rock do you want to use ?
Stay with the default ('master') if you want to use Rock on the most recent
distributions (Ubuntu 16.04 and later). Use 'stable' only for 
now officially unsupported distributions (Ubuntu 14.04) [master]
  1. Answer here 'yes' in case you want to activate python. You need a python installation in your system for this.
Do you want to activate python? [no] yes
Select the path to the python executable [/usr/bin/python3]
  1. Select 'master' in case you want to build EDS with DSO backend or type 'ceres' otherwise
Which flavor of EDS do you want to use ?
Stay with the default ('master') if you want to use EDS with DSO backend
otherwise select ('ceres') [master]
  1. Choose the answers by default in the questions regarding RTT/Orocos. Select 'gnulinux' for Linux based systems
Do you need compatibility with OCL ? (yes or no)
New Rock users that don't need backward compatibility with legacy Orocos components
probably want to say 'no'. Otherwise, say 'yes'.
Saying 'yes' will significantly impact compilation time and the size of the resulting binaries
Please answer 'yes' or 'no' [no]

the target operating system for Orocos/RTT (gnulinux, xenomai, or macosx) [gnulinux]
  1. In case no problems occurred the installation shoudl finish "successfully"
updated environment
Command finished successfully at 2022-04-13 16:47:58 +0200 (took 44 mins 20 secs)
  1. Don't forget to source the environmental variables
docker@dev:$ source env.sh

Dockerfile and Image

All the steps described in the Installation are also written in a Dockerfile. Run the following command line to build the EDS docker image

wget https://raw.githubusercontent.com/uzh-rpg/eds-buildconf/master/Dockerfile 
docker build -t <image_name> -f Dockerfile .

There is also a built docker image in here, that you can just download and run your docker container with EDS installed. You might pull the latest EDS docker image

docker pull jhidalgocarrio/eds:20.04

Execution

All the executing scripts are in bundles/eds directory of your installation.

docker@javi $ cd rock/dev
docker@dev $ source env.sh
docker@dev $ cd bundles/eds

Download one dataset in log format, for example the atrium dataset:

docker:eds (master) $ wget https://download.ifi.uzh.ch/rpg/eds/atrium/atrium.log -O /tmp/atrium.log

Run the run_eds_log.rb script in one terminal

docker@eds (master) $ ruby scripts/slam/eds/run_eds_logs.rb --dataset=atrium --log_type=davis /tmp/atrium.log

Open another terminal and visualize the progress with the eds_visualizaton.rb script.

docker@javi $ cd rock/dev
docker@dev $ source env.sh
docker@dev $ cd bundles/eds
docker@eds (master) $ ruby scripts/gui/eds_visualization.rb

It does not matter the order you run the scripts, visualization or running. The visualization will always wait until the running gives data. You should see something like this:

<p align="left"> <a href="https://rpg.ifi.uzh.ch/eds.html"> <img src="./doc/img/atrium_eds.png" alt="EDS" width="640"/> </a> </p>

Log Files and Conversion to Rosbag

The log files are created in the /tmp folder by default (You can change the folder here). The log files are named in a folder as /tmp/date, e.g.: /tmp/20220401-1015 EDS is not real time. You can convert the log file to the correct time by running the following command in your log folder

docker@javi $ cd rock/dev
docker@dev $ source env.sh
docker@dev $ cd /tmp/20220401-1015
docker@20220401-1015 $ rock-convert --use_sample-time eds.0.log

The resulting eds.0.log file should be in the newly created updated folder in /tmp/20220401-1015. You can replay the log files and visualize the results with eds_visualization.rb by doing

docker@javi $ cd rock/dev
docker@dev $ source env.sh
docker@dev $ cd bundles/eds
docker@eds (master) $ ruby scripts/gui/eds_visualization.rb --log /tmp/20220401-1015/updated/eds.0.log

Pocolog files (i.e.: eds.0.log) conversion to Rosbag is possible by running the pocolog2rosbag.py. It is explained in the e2calib repository.

Troubleshooting

In case you see CORBA related errors when runing the run_eds_log.rb. Restart the omniorb deamon

docker@dev $ sudo /etc/init.d/omniorb4-nameserver stop
docker@dev $ sudo rm -f /var/lib/omniorb/*
docker@dev $ sudo /etc/init.d/omniorb4-nameserver start

The Event-to-Image Tracker: Source Code

EDS source code is structure as follows:

EDS combines Events with classical frame-based direct methods. You can have a look at this zip file, in case you are already familiar with direct methods and you just want to see the core part of the event-based tracker implemented in EDS. The code in the zip file contains comments to the equations in the paper.

RPG Beamsplitter Design

Describing the mechanical and electronic setup for combined Event Camera and RGB Camera with simultaneous trigger

Beamsplitter Design

Cameras:

Mirror:

IR Filter (optional)

Case Design:

Acknowledgements

The Authors would like to thank Simon Klenk from TUM for the nice discussions about the potential of direct methods. We also thank our collaboration with Prophesee and Huawei.

Additional Resources on Event Cameras