It is believed that self-driving vehicles will play an essential role in reducing road accidents, alleviating traffic congestion, and reducing energy consumption in transportation, thus providing safety, efficiency, and convenience for everyone. This thesis focuses on building autonomous driving systems leveraging deep learning under complex contexts from normal-speed urban to high-speed drifting/racing scenarios.

The first part studies the frameworks of end-to-end driving in dynamic urban scenarios based on imitation learning (IL).We progressively consider all-day and all-weather driving conditions using a combination of multi-modal sensors, i.e., cameras, lidars, and radars. The uncertainty measures are also utilized to indicate dangerous cases and improve the interpretability of the...[ Read more ]

It is believed that self-driving vehicles will play an essential role in reducing road accidents, alleviating traffic congestion, and reducing energy consumption in transportation, thus providing safety, efficiency, and convenience for everyone. This thesis focuses on building autonomous driving systems leveraging deep learning under complex contexts from normal-speed urban to high-speed drifting/racing scenarios.

The first part studies the frameworks of end-to-end driving in dynamic urban scenarios based on imitation learning (IL).We progressively consider all-day and all-weather driving conditions using a combination of multi-modal sensors, i.e., cameras, lidars, and radars. The uncertainty measures are also utilized to indicate dangerous cases and improve the interpretability of the driving system.

In the second part, we focus on local motion planning using semantic state abstractions rather than learning the entire navigation stack to achieve more generalizable and strategic driving performance in interaction-intensive environments, e.g., unregulated intersections/ roundabouts. The interactions among different road users are modeled implicitly through graph attention networks (GAT), where the inter-vehicle influences are adaptively weighted according to specific contexts. By training the whole network through deep reinforcement learning (DRL), the proposed method performs better in balancing driving safety and efficiency than the baselines. The learned policy can also be transferred into a real-world dataset thanks to the decoupled perception-planning framework.

Furthermore, the third part highlights highly dynamic drifting/racing scenarios to facilitate agile autonomous driving near the handling limits under high-speed conditions. We first study the drift cornering control problems based on model-free DRL approaches, then combine IL and model-based DRL to solve a vision-based autonomous racing task. The proposed method is validated both in a realistic simulator and on a physical RC car. Related methodologies also support the 1st HKUST autonomous RC-car racing competition¹.