After the DARPA Grand Challenge 2005, the autonomous driving system (ADS) captures the world's attention and has achieved impressive results in various subfields of study in the last several years. However, continuous fatal accidents told us the single type sensor ADS cannot handle countless complex cases. Every sensor can fail in certain scenarios since its drawback even the extremely expensive LiDAR. To overcome this, sensor fusion may become the only practical solution at present. In this thesis, we are aiming to utilize novel approaches to realize sensor fusion for the autonomous driving system with the LiDAR-Camera system. We will answer and discuss the following questions in this thesis: (1) How to solve the temporal domain problem which includes time synchronization and sensor da...[ Read more ]

After the DARPA Grand Challenge 2005, the autonomous driving system (ADS) captures the world's attention and has achieved impressive results in various subfields of study in the last several years. However, continuous fatal accidents told us the single type sensor ADS cannot handle countless complex cases. Every sensor can fail in certain scenarios since its drawback even the extremely expensive LiDAR. To overcome this, sensor fusion may become the only practical solution at present. In this thesis, we are aiming to utilize novel approaches to realize sensor fusion for the autonomous driving system with the LiDAR-Camera system. We will answer and discuss the following questions in this thesis: (1) How to solve the temporal domain problem which includes time synchronization and sensor data alignment? (2) How to solve spatial domain problems, i.e. homogeneous and heterogeneous sensor calibration?

For the first question, we study the sensor characteristics and propose a hardware-based solution for the synchronization and triggering of heterogeneous and multiple sensors. For the second question, we propose a novel automatic algorithm using independent planar surfaces for LiDAR-LiDAR extrinsic calibration and two effective and distinct calibration approaches for LiDAR-Camera based on 3D-2D line constraints which are extracted via polygon board or nature scene. The complete pipeline work has been successfully applied in our several generation autonomous driving vehicles and support tasks under various work conditions. After conducting a series of experiments in both simulation and real-world environments. We clearly demonstrate the effectiveness and robustness of the proposed sensor fusion pipeline for the autonomous driving systems.