Semantic segmentation serves an important role in autonomous driving. The goal of vision-based semantic segmentation is to take images as input and generate the semantic estimation of each element in the output space, such as the image space or the bird’s-eye-view (BEV) space. With the development of artificial intelligence, data-driven approaches have achieved impressive performance for this task. However, existing approaches often only take color images as input, and the potential exploration of using other visual features for semantic segmentation in the autonomous driving system is still limited.

This thesis contributes to semantic segmentation with the assistance of visual features for autonomous driving. In the first part, I consider the computation of visual features, including d...[ Read more ]

Semantic segmentation serves an important role in autonomous driving. The goal of vision-based semantic segmentation is to take images as input and generate the semantic estimation of each element in the output space, such as the image space or the bird’s-eye-view (BEV) space. With the development of artificial intelligence, data-driven approaches have achieved impressive performance for this task. However, existing approaches often only take color images as input, and the potential exploration of using other visual features for semantic segmentation in the autonomous driving system is still limited.

This thesis contributes to semantic segmentation with the assistance of visual features for autonomous driving. In the first part, I consider the computation of visual features, including disparity images and optical flow. The former can provide useful depth information, while the latter can provide valuable motion information. The experimental results verify the effectiveness of the proposed unsupervised approaches for stereo matching and optical flow estimation.

In the second part, I focus on employing these visual features to improve the semantic segmentation performance for autonomous driving, including semantic segmentation in the image space and in the BEV space. Specifically, for semantic segmentation in the image space, I develop a surface normal estimator based on the disparity estimations to achieve accurate drivable area detection. I also adopt the disparity estimations to generate transformed disparity images and further improve the performance of semantic driving scene understanding. In addition, for semantic segmentation in the BEV space, I propose an optical flow distillation paradigm based on the optical flow estimations to further improve the performance of BEV semantic forecasting. The experimental results demonstrate the effectiveness of the proposed approaches for semantic segmentation. These useful visual features can effectively improve the performance of semantic segmentation for autonomous driving.