Home

Awesome

LSD-SLAM: Large-Scale Direct Monocular SLAM

LSD-SLAM is a novel approach to real-time monocular SLAM. It is fully direct (i.e. does not use keypoints / features) and creates large-scale, semi-dense maps in real-time on a laptop. For more information see http://vision.in.tum.de/lsdslam where you can also find the corresponding publications and Youtube videos, as well as some example-input datasets, and the generated output as rosbag or .ply point cloud.

Related Papers

1. Quickstart / Minimal Setup

First, install LSD-SLAM following 2.1 or 2.2, depending on your Ubuntu / ROS version. You don't need openFabMap for now.

Download the Room Example Sequence and extract it.

Launch the lsd_slam viewer:

	rosrun lsd_slam_viewer viewer

Launch the lsd_slam main ros node:

	rosrun lsd_slam_core live_slam image:=/image_raw camera_info:=/camera_info

Play the sequence:

	rosbag play ~/LSD_room.bag

You should see one window showing the current keyframe with color-coded depth (from live_slam), and one window showing the 3D map (from viewer). If for some reason the initialization fails (i.e., after ~5s the depth map still looks wrong), focus the depth map and hit 'r' to re-initialize.

2. Installation

We tested LSD-SLAM on two different system configurations, using Ubuntu 12.04 (Precise) and ROS fuerte, or Ubuntu 14.04 (trusty) and ROS indigo. Note that building without ROS is not supported, however ROS is only used for input and output, facilitating easy portability to other platforms.

2.1 ROS fuerte + Ubuntu 12.04

Install system dependencies:

sudo apt-get install ros-fuerte-libg2o liblapack-dev libblas-dev freeglut3-dev libqglviewer-qt4-dev libsuitesparse-dev libx11-dev

In your ROS package path, clone the repository:

git clone https://github.com/tum-vision/lsd_slam.git lsd_slam

Compile the two package by typing:

rosmake lsd_slam

2.2 ROS indigo + Ubuntu 14.04

We do not use catkin, however fortunately old-fashioned CMake-builds are still possible with ROS indigo. For this you need to create a rosbuild workspace (if you don't have one yet), using:

sudo apt-get install python-rosinstall
mkdir ~/rosbuild_ws
cd ~/rosbuild_ws
rosws init . /opt/ros/indigo
mkdir package_dir
rosws set ~/rosbuild_ws/package_dir -t .
echo "source ~/rosbuild_ws/setup.bash" >> ~/.bashrc
bash
cd package_dir

Install system dependencies:

sudo apt-get install ros-indigo-libg2o ros-indigo-cv-bridge liblapack-dev libblas-dev freeglut3-dev libqglviewer-dev libsuitesparse-dev libx11-dev

In your ROS package path, clone the repository:

git clone https://github.com/tum-vision/lsd_slam.git lsd_slam

Compile the two package by typing:

rosmake lsd_slam

2.3 openFabMap for large loop-closure detection [optional]

If you want to use openFABMAP for large loop closure detection, uncomment the following lines in lsd_slam_core/CMakeLists.txt :

#add_subdirectory(${PROJECT_SOURCE_DIR}/thirdparty/openFabMap)
#include_directories(${PROJECT_SOURCE_DIR}/thirdparty/openFabMap/include)
#add_definitions("-DHAVE_FABMAP")
#set(FABMAP_LIB openFABMAP )

Note for Ubuntu 14.04: The packaged OpenCV for Ubuntu 14.04 does not include the nonfree module, which is required for openFabMap (which requires SURF features). You need to get a full version of OpenCV with nonfree module, which is easiest by compiling your own version. We suggest to use the 2.4.8 version, to assure compatibility with the current indigo open-cv package.

3 Usage

LSD-SLAM is split into two ROS packages, lsd_slam_core and lsd_slam_viewer. lsd_slam_core contains the full SLAM system, whereas lsd_slam_viewer is optionally used for 3D visualization. Please also read General Notes for good results below.

3.1 lsd_slam_core

We provide two different usage modes, one meant for live-operation (live_slam) using ROS input/output, and one dataset_slam to use on datasets in the form of image files.

3.1.1 Using live_slam

If you want to directly use a camera.

rosrun lsd_slam_core live_slam /image:=<yourstreamtopic> /camera_info:=<yourcamera_infotopic>

When using ROS camera_info, only the image dimensions and the K matrix from the camera info messages will be used - hence the video has to be rectified.

Alternatively, you can specify a calibration file using

rosrun lsd_slam_core live_slam /image:=<yourstreamtopic> _calib:=<calibration_file>

In this case, the camera_info topic is ignored, and images may also be radially distorted. See the Camera Calibration section for details on the calibration file format.

3.1.2 Using dataset_slam

rosrun lsd_slam_core dataset_slam _files:=<files> _hz:=<hz> _calib:=<calibration_file>

Here, <files> can either be a folder containing image files (which will be sorted alphabetically), or a text file containing one image file per line. <hz> is the framerate at which the images are processed, and <calibration_file> the camera calibration file.

Specify _hz:=0 to enable sequential tracking and mapping, i.e. make sure that every frame is mapped properly. Note that while this typically will give best results, it can be much slower than real-time operation.

3.1.3 Camera Calibration

LSD-SLAM operates on a pinhole camera model, however we give the option to undistort images before they are being used. You can find some sample calib files in lsd_slam_core/calib.

Calibration File for FOV camera model:

fx/width fy/height cx/width cy/height d
in_width in_height
"crop" / "full" / "none" / "e1 e2 e3 e4 0"
out_width out_height

Here, the values in the first line are the camera intrinsics and radial distortion parameter as given by the PTAM cameracalibrator, in_width and in_height is the input image size, and out_width out_height is the desired undistorted image size. The latter can be chosen freely, however 640x480 is recommended as explained in section 3.1.6. The third line specifies how the image is distorted, either by specifying a desired camera matrix in the same format as the first four intrinsic parameters, or by specifying "crop", which crops the image to maximal size while including only valid image pixels.

Calibration File for Pre-Rectified Images

This one is without radial distortion correction, as a special case of ATAN camera model but without the computational cost:

fx/width fy/height cx/width cy/height 0
width height
none
width height

Calibration File for OpenCV camera model:

fx fy cx cy k1 k2 p1 p2
inputWidth inputHeight
"crop" / "full" / "none" / "e1 e2 e3 e4 0"
outputWidth outputHeight

3.1.4 Useful Hotkeys

3.1.5 Parameters (Dynamic Reconfigure)

A number of things can be changed dynamically, using (for ROS fuerte)

rosrun dynamic_reconfigure reconfigure_gui 

or (for ROS indigo)

rosrun rqt_reconfigure rqt_reconfigure

Parameters are split into two parts, ones that enable / disable various sorts of debug output in /LSD_SLAM/Debug, and ones that affect the actual algorithm, in /LSD_SLAM. Note that debug output options from /LSD_SLAM/Debug only work if lsd_slam_core is built with debug info, e.g. with set(ROS_BUILD_TYPE RelWithDebInfo).

Useful for debug output are:

3.1.6 General Notes for Good Results

3.2 LSD-SLAM Viewer

The viewer is only for visualization. It can also be used to output a generated point cloud as .ply. For live operation, start it using

rosrun lsd_slam_viewer viewer

You can use rosbag to record and re-play the output generated by certain trajectories. Record & playback using

rosbag record /lsd_slam/graph /lsd_slam/keyframes /lsd_slam/liveframes -o file_pc.bag
rosbag play file_pc.bag

You should never have to restart the viewer node, it resets the graph automatically.

If you just want to lead a certain pointcloud from a .bag file into the viewer, you can directly do that using

rosrun lsd_slam_viewer viewer file_pc.bag

3.2.1 Useful Hotkeys

3.2.2 Parameters (Dynamic Reconfigure)

4 Datasets

For convenience we provide a number of datasets, including the video, lsd-slam's output and the generated point cloud as .ply. See http://vision.in.tum.de/lsdslam

5 License

LSD-SLAM is licensed under the GNU General Public License Version 3 (GPLv3), see http://www.gnu.org/licenses/gpl.html.

For commercial purposes, we also offer a professional version under different licencing terms.

6 Troubleshoot / FAQ

How can I get the live-pointcloud in ROS to use with RVIZ?

You cannot, at least not on-line and in real-time. The reason is the following:

In the background, LSD-SLAM continuously optimizes the pose-graph, i.e., the poses of all keyframes. Each time a keyframe's pose changes (which happens all the time, if only by a little bit), all points from this keyframe change their 3D position with it. Hence, you would have to continuously re-publish and re-compute the whole pointcloud (at 100k points per keyframe and up to 1000 keyframes for the longer sequences, that's 100 million points, i.e., ~1.6GB), which would crush real-time performance.

Instead, this is solved in LSD-SLAM by publishing keyframes and their poses separately:

Points are then always kept in their keyframe's coodinate system: That way, a keyframe's pose can be changed without even touching the points. In fact, in the viewer, the points in the keyframe's coodinate frame are moved to a GLBuffer immediately and never touched again - the only thing that changes is the pushed modelViewMatrix before rendering.

Note that "pose" always refers to a Sim3 pose (7DoF, including scale) - which ROS doesn't even have a message type for.

If you need some other way in which the map is published (e.g. publish the whole pointcloud as ROS standard message as a service), the easiest is to implement your own Output3DWrapper.

Tracking immediately diverges / I keep getting "TRACKING LOST for frame 34 (0.00% good Points, which is -nan% of available points, DIVERGED)!"