Camera-LiDAR Extrinsic Calibration

相机-激光雷达联合标定Advanced

Finds the rotation and translation between a lidar and a camera so a point cloud can be projected accurately onto the image.

Robots and self-driving cars often carry both a lidar and a camera: the lidar gives accurate range, the camera gives color and texture. Fusing the two first requires knowing the rigid-body transform — three rotation plus three translation parameters — from the lidar’s coordinate frame to the camera’s, i.e., the extrinsics. Target-based methods use a checkerboard, ChArUco, or Aprilgrid: corners are detected in the image, a matching plane is fit in the point cloud to find corresponding corners, and the transform is then solved. Target-free methods register directly against the structure and texture of the environment — for example, the tool Koide and colleagues at Japan’s AIST open-sourced at ICRA 2023, which also supports non-repetitive-scan lidars like Livox. The result is used to color a point cloud, project it onto an image to assist detection, or feed BEV fusion perception; a common sanity check is whether the edges of the projected point cloud line up with object edges in the image.

ExampleA quadruped robot carries a Livox Mid-360 and an RGB camera; after calibration, the point cloud is projected onto the image to check whether points along a step’s edge land on the step’s outline in the image, and the same extrinsics are then used to color the point cloud into a colored map.

Also called: LiDAR-Camera Calibration
Related: Camera Extrinsics · LiDAR · Multi-Sensor Fusion · Calibration Board · Camera Calibration · Bird’s-Eye View
Sources: What Is Lidar-Camera Calibration? - MATLAB & Simulink
koide3/direct_visual_lidar_calibration（GitHub）