Real-time and accurate state estimation has become the backbone technology for many applications such as autonomous driving, UAV navigation and augmented reality. Over the past decade, numerous localization algorithms, either visual or visual-inertial based, have been developed and deployed for real-world systems. Despite the success achieved by those algorithms, the inconsistency caused by the odometry drifting is still inevitable, especially over long-term runs. This thesis aims to improve the global consistency of the state estimation algorithm for localization, as well as keep the merit of local smoothness inherited from visual(-inertial) based methods. To achieve the goal, the GNSS raw measurements are tightly fused with visual and inertial information under a unified probabilistic...[ Read more ]

Real-time and accurate state estimation has become the backbone technology for many applications such as autonomous driving, UAV navigation and augmented reality. Over the past decade, numerous localization algorithms, either visual or visual-inertial based, have been developed and deployed for real-world systems. Despite the success achieved by those algorithms, the inconsistency caused by the odometry drifting is still inevitable, especially over long-term runs. This thesis aims to improve the global consistency of the state estimation algorithm for localization, as well as keep the merit of local smoothness inherited from visual(-inertial) based methods. To achieve the goal, the GNSS raw measurements are tightly fused with visual and inertial information under a unified probabilistic framework. The proposed system is able to provide accurate global 6-DoF estimation under complex indoor-outdoor environments where GNSS signals may be intermittent or even inaccessible. To relate the global measurements with the local states, a coarse-to-fine initialization procedure is described to efficiently calibrate the transformation online and initialize GNSS states from only a short window of measurements. The GNSS code pseudorange, Doppler shift and carrier phase measurements are modelled in detail and are used to constrain the system states in a factor graph. The integer ambiguity problem caused by carrier cycle slip, which used to be the major concern for high-precision GNSS applications, can be simplified and steadily addressed with the aid of visual and inertial information. For complex and GNSS-unfriendly areas, the degenerate cases are discussed and carefully handled to ensure robustness. Thanks to the tightly-coupled multi-sensor approach and system design, the system can fully exploit the advantages of three types of sensors and is able to seamlessly cope with the transition between indoor and outdoor environments, where satellites are lost and reacquired. Extensive experiments are designed to test the performance of the system, and results show that the proposed method substantially eliminates the drift of VIO and preserves the local accuracy in spite of noisy GNSS measurements. The challenging indoor-outdoor and urban driving scenes are also considered to verify the availability and robustness of the proposed system in harsh environments. In addition, experiments show that our system can gain from even a single satellite, while conventional GNSS algorithms need four at least.