Accurate state estimation is a fundamental problem for autonomous robots. Robots usually adopt multi-sensor fusion to improve precision and robustness in practice. The first section focuses on a monocular visual-inertial system (VINS), which is the minimum sensor suite (in size, weight, and power) for metric six degrees-of-freedom (DOF) state estimation. The system starts with a robust procedure for estimator initialization. A tightly-coupled, nonlinear optimization-based method is used to obtain highly accurate visual-inertial odometry by fusing pre-integrated IMU measurements and feature observations. A loop detection module, in combination with tightly-coupled formulation, enables relocalization with minimum computation. Additionally, it performs four degrees-of-freedom pose...[ Read more ]

Accurate state estimation is a fundamental problem for autonomous robots. Robots usually adopt multi-sensor fusion to improve precision and robustness in practice. The first section focuses on a monocular visual-inertial system (VINS), which is the minimum sensor suite (in size, weight, and power) for metric six degrees-of-freedom (DOF) state estimation. The system starts with a robust procedure for estimator initialization. A tightly-coupled, nonlinear optimization-based method is used to obtain highly accurate visual-inertial odometry by fusing pre-integrated IMU measurements and feature observations. A loop detection module, in combination with tightly-coupled formulation, enables relocalization with minimum computation. Additionally, it performs four degrees-of-freedom pose graph optimization to enforce global consistency. Furthermore, the proposed system can reuse a map by saving and loading it in an efficient way. Current map and previous map can be merged together by the global pose graph optimization. The performance of this system was validated on public datasets and real-world experiments and compare against other state-of-the-art algorithms. Also, onboard closed-loop autonomous flight on the MAV platform was performed. The algorithm was ported the to an iOS device for demonstration. The proposed work is a reliable, complete, and versatile system that is applicable for different applications that require high accuracy in localization. Implementations for both PCs¹ and iOS mobile devices² were open-sourced.

The second section focuses on the generality of visual-inertial framework. Although many algorithms for state estimation had been proposed in the past, they were usually applied to a single sensor or a specific sensor suite. Also, few of them dealed with temporal misalignment (time offset) among different sensors, which severely impacts fusion performance. To this end, a visual-inertial sensor fusion framework was proposed, which supported multiple sensor combinations and achieves locally accurate and globally drift-free state estimation. Local sensors (camera, IMU) are fused in a tightly-coupled visual-inertial optimization, which produces precise local states (position, orientation, velocity, bias, and feature depth). Meanwhile, the temporal offset between visual and inertial sensors is calibrated online within optimization. Local estimation is further fused with global sensor (GPS) in a pose graph optimization to register into a global coordinate and eliminate accumulate drift. Furthermore, this optimization-based framework is robust to sensor failure inherently. When one camera or IMU fails, the system can work with other sensors. The performance of propoesed system was evaluated on public datasets and with real-world experiments. Results were compared against other state-of-the-art algorithms. The robustness of proposed system was demonstrated by simulating sensor failure cases in real devices. The implementation code was open-sourced³.

¹https://github.com/HKUST-Aerial-Robotics/VINS-Mono

²https://github.com/HKUST-Aerial-Robotics/VINS-Mobile

³https://github.com/HKUST-Aerial-Robotics/VINS-Fusion