Compared to known and static environments, mobile robot navigation within dynamic, pedestrian-rich and unfamiliar environments is still challenging. To model these scenarios through traditional hand-crafted features is not efficient and effective enough. Deep learning has achieved state-of-the-art results in various fields of study, especially in complex task modelling, in the last several years. Sensorimotor learning has also shown great potential to solve many complicated manipulation tasks. However, the potential exploration of deep learning in robot sensorimotor policies, especially for mobile robot navigation is still limited. In this thesis, we are targeting to leverage fully differentiable structures to realize end-to-end sensorimotor learning for ground mobile robot navi...[ Read more ]

Compared to known and static environments, mobile robot navigation within dynamic, pedestrian-rich and unfamiliar environments is still challenging. To model these scenarios through traditional hand-crafted features is not efficient and effective enough. Deep learning has achieved state-of-the-art results in various fields of study, especially in complex task modelling, in the last several years. Sensorimotor learning has also shown great potential to solve many complicated manipulation tasks. However, the potential exploration of deep learning in robot sensorimotor policies, especially for mobile robot navigation is still limited. In this thesis, we are targeting to leverage fully differentiable structures to realize end-to-end sensorimotor learning for ground mobile robot navigation. We aim to answer the following questions: (1) How do we learn the sensorimotor policy for mobile robot navigation? And (2) how should we deploy it in the real world considering the reality gap and uncertainties? For question (1), we proposed two structures for sensorimotor learning through both deep reinforcement learning and inverse imitation learning. For the second question, we propose shift loss to constraint the sequence input domain adaptation and combine it with a generative adversarial network to translate the real-world image streams back to the synthetic domain during the deployment phase. The uncertainties of deep learning are also considered to learn a stochastic policy. By conducting a series of experiments, both in simulated and real-world environments, we show how these learned sensorimotor models can be successfully applied in both indoor and outdoor mobile robot navigation scenarios.