Home

Awesome

Awesome-LiDAR-Camera-Calibration

Awesome

A Collection of LiDAR-Camera-Calibration Papers, Toolboxes and Notes.

Outline

0. Introduction

For applications such as autonomous driving, robotics, navigation systems, and 3-D scene reconstruction, data of the same scene is often captured using both lidar and camera sensors. To accurately interpret the objects in a scene, it is necessary to fuse the lidar and the camera outputs together. Lidar camera calibration estimates a rigid transformation matrix (extrinsics, rotation+translation, 6 DoF) that establishes the correspondences between the points in the 3-D lidar plane and the pixels in the image plane.

Example

1. Target-based methods

PaperTargetFeatureOptimizationToolboxNote
Extrinsic Calibration of a Camera and Laser Range Finder (improves camera calibration), 2004checkerboardC:Plane (a), L: pts in plane (m)point-to-planeCamLaserCalibraToolCN
Fast Extrinsic Calibration of a Laser Rangefinder to a Camera, 2005checkerboardC: Plane (a), L: Plane (m)plane(n/d) correspondence, point-to-planeLCCT*
Extrinsic calibration of a 3D laser scanner and an omnidirectional camera, 2010checkerboardC: plane (a), L: pts in plane (m)point-to-planecam_lidar_calib*
LiDAR-Camera Calibration using 3D-3D Point correspondences, 2017cardboard + ArUcoC: 3D corners (a), L: 3D corners (m)ICPlidar_camera_calibration*
Reflectance Intensity Assisted Automatic and Accurate Extrinsic Calibration of 3D LiDAR and Panoramic Camera Using a Printed Chessboard, 2017checkerboardC: 2D corners (a), L: 3D corners (a)PnP, angle differenceILCC*
Extrinsic Calibration of Lidar and Camera with Polygon, 2018regular cardboardC: 2D edge, corners (a), L: 3D edge, pts in plane (a)point-to-line, point-inside-polygonram-lab/plycal*
Automatic Extrinsic Calibration of a Camera and a 3D LiDAR using Line and Plane Correspondences, 2018checkerboardC: 3D edge, plane(a), L: 3D edge, pts in plane (a)direcion/normal, point-to-line, point-to-planeMatlab LiDAR Toolbox*
Improvements to Target-Based 3D LiDAR to Camera Calibration, 2020cardboard with ArUcoC: 2d corners (a), L: 3D corners (a)PnP, IOUgithub*
ACSC: Automatic Calibration for Non-repetitive Scanning Solid-State LiDAR and Camera Systems, 2020checkerboardC: 2D corners (a), L: 3D corners (a)PnPACSC*
Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups, 2021cardboard with circle & ArucoC: 3D points (a), L: 3D points (a)ICPvelo2cam_ calibration*
Single-Shot is Enough: Panoramic Infrastructure Based Calibration of Multiple Cameras and 3D LiDARs,2021 IROSpanoramic infrastructureC: CCTag(a), L: corner points and vectors(a)PnP, ICPmultiple-cameras-and-3D-LiDARs-extrinsic-calibration*
Joint Camera Intrinsic and LiDAR-Camera Extrinsic Calibration, 2023, ICRAchessboard + circlesC: chessboard corners, circle centers, L: circle centersPnP, intrinsic constraints (corners, cirlce center)OpenCalib/JointCalibCN
LCE-Calib: Automatic LiDAR-Frame/Event Camera Extrinsic Calibration With a Globally Optimal Solutioncheckboard3D point-to-3D plan, 3D point-to-3D edgepoint-to-plane, point-to-edgeLCECalib*

C: camera, L: LiDAR, a: automaic, m: manual

2. Targetless methods

2.1. Motion-based methods

PaperFeatureOptimizationToolboxNote
LiDAR and Camera Calibration Using Motions Estimated by Sensor Fusion Odometry, 2018C: motion (ICP), L: motion (VO)hand-eye calibration**

2.2. Scene-based methods

2.2.1. Traditional methods

PaperFeatureOptimizationToolboxNote
Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information, 2012C:grayscale, L: reflectivitymutual information, BB steepest gradient ascentExtrinsic Calib*
Automatic Calibration of Lidar and Camera Images using Normalized Mutual Information, 2013C:grayscale, L: reflectivity, noramlnormalized MI, particle swarm**
Automatic Online Calibration of Cameras and Lasers, 2013C: Canny edge, L: depth-discontinuous edgecorrelation, grid search**
SOIC: Semantic Online Initialization and Calibration for LiDAR and Camera, 2020semantic centroidPnP**
A Low-cost and Accurate Lidar-assisted Visual SLAM System, 2021C: edge(grayscale), L: edge (reflectivity, depth projection)ICP, coordinate descent algorithmsCamVox*
Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments,2021C:Canny edge(grayscale), L: depth-continuous edgepoint-to-line, Gaussian-Newtonlivox_camera_calib*
CRLF: Automatic Calibration and Refinement based on Line Feature for LiDAR and Camera in Road Scenes, 2021C:straight line, L: straight lineperspective3-lines (P3L)*CN
Fast and Accurate Extrinsic Calibration for Multiple LiDARs and Cameras,2021C:Canny edge(grayscale), L: depth-continuous edgepoint-to-plane, point-to-edgemlcc*
General, Single-shot, Target-less, and Automatic LiDAR-Camera Extrinsic Calibration Toolbox,ICRA 2023C:keypoints, L: keypoints on intensity images (by SuperGlue)reprojection error minimization, RANSACdirect_visual_lidar_calibration*
Calib-Anything: Zero-training LiDAR-Camera Extrinsic Calibration Method Using Segment Anything, 2023C: Segmentation, L: normal estimation, plane fitting & euclidean cluster, intensity normalizationrandom search, grid searchOpenCalib*

2.2.2. Deep-learning methods

PaperToolboxNote
RegNet: Multimodal sensor registration using deep neural networks, 2017,IVregnet*
CalibNet: Geometrically supervised extrinsic calibration using 3d spatial transformer networks,2018,IROSCalibNet*
LCCNet: Lidar and Camera Self-Calibration using CostVolume Network,2021,CVPRWLCCNet*
Calib-Anything: Zero-training LiDAR-Camera Extrinsic Calibration Method Using Segment Anything,2023Calib-Anything*

3. Other toolboxes

ToolboxIntroductionNote
Apollo sensor calibration toolstargetless method, no source codeCN
Autoware camera lidar calibratorpick points mannually, PnP*
Autoware calibration camera lidarcheckerboard, similar to LCCTCN
livox_camera_lidar_calibrationpick points mannually, PnP*
OpenCalibtarget-based, target-less, mannualOpenCalib: A Multi-sensor Calibration Toolbox for Autonomous Driving
tier4/CalibrationToolstarget-based, mannual*