Home

Awesome

Event-based Agile Object Catching with a Quadrupedal Robot

<p align="center"> <a href="https://youtu.be/FpsVB8EO54M"> <img src="media/anymal_catch_thumbnail_yt_button.png" alt="Dynamic Ball Catching with Anymal" width="600"/> </a> </p>

This repository contains code that implements dynamic obstacle tracking with a DVS sensor, as described in Forrai et. al ICRA '23 (see paper here) and Falanga et. al SciRob '20 (see paper here).

Citation

If you use this code in an academic context, please cite the following works:

Benedek Forrai*, Takahiro Miki*, Daniel Gehrig*, Marco Hutter, Davide Scaramuzza, "Event-based Agile Object Catching with a Quadrupedal Robot", ICRA, 2023. PDF (* denotes equal contribution)

@InProceedings{Forrai_23_ICRA,
  author = {Benedek Forrai and Takahiro Miki and Daniel Gehrig and Marco Hutter and Davide Scaramuzza},
  title = {Event-based Agile Object Catching with a Quadrupedal Robot},
  booktitle = {{IEEE} International Conference on Robotics and Automation (ICRA)},
  month = {June},
  year = {2023}
}

Davide Falanga, Kevin Kleber, Davide Scaramuzza, "Dynamic obstacle avoidance for quadrotors with event cameras", Science Robotics, 2020

@article{Falanga_20_SciRob,
  title={Dynamic obstacle avoidance for quadrotors with event cameras},
  author={Davide Falanga and Kevin Kleber and Davide Scaramuzza},
  journal={Science Robotics},
  volume={5},
  number={40},
  pages={eaaz9712},
  year={2020},
  publisher={American Association for the Advancement of Science}
}

Installation

Environment

This code was used on the following setup: Ubuntu 20.04 LTS (Focal) + ROS Noetic + LibOpenCV 4.2. Although you might get away with using 22.04, ROS Kinetic or later versions of LibOpenCV, the package was not tested under such circumstances.

Install dependencies

Our repository depends on the following non-standard packages:

To install these in your workspace, follow the instructions below.

First, you should install and test rpg_dvs_ros. To do this, follow the instructions of the README.

Then, you can create a ~/git folder to which we pull all the other repos:

mkdir -p ~/git
cd ~/git

git clone git@github.com:uzh-rpg/rpg_quadrotor_common.git
git clone git@github.com:ethz-asl/eigen_catkin.git
git clone git@github.com:ethz-asl/gflags_catkin.git
git clone git@github.com:ethz-asl/glog_catkin.git

You should then symlink these packages to your catkin workspace (change the first line according to your workspace path)

catkin_workspace=~/catkin_ws
ln -s ~/git/rpg_quadrotor_common $catkin_workspace/src/.
ln -s ~/git/eigen_catkin $catkin_workspace/src/.
ln -s ~/git/gflags_catkin $catkin_workspace/src/.
ln -s ~/git/glog_catkin $catkin_workspace/src/.

Clone

Clone the repo

mkdir -p ~/git
cd ~/git
git clone git@github.com:fbenedek/rpg_dynamic_obstacle_detection.git

Symlink it to your ROS workspace similarly to the dependencies (change the first line according to your workspace path):

catkin_workspace=~/catkin_ws
ln -s ~/git/rpg_dynamic_obstacle_detection $catkin_workspace/src/.

Build the packages

Having cloned the packages, you can build them using catkin build:

cd $catkin_workspace
source devel/setup.bash
catkin build rpg_dynamic_obstacle_detection rpg_ransac_parabola

Don't forget to source your environment again before using the packages:

cd $catkin_workspace
source devel/setup.bash

Testing

After building and sourcing your environment, you can test your installation by running detection on a supplied .bag file. To do so, you only need to launch the following .launch file:

cd $catkin_workspace
roslaunch rpg_dynamic_obstacle_detection test_detection_and_fitting.launch

This should pop up an RVIZ window showing the detections, the mean timestamp images and a visualization of the clustering similar to the following: Test window