Loop closure can effectively correct the accumulated error in robot localization, which plays a critical role in the long-term navigation of the robot. Traditional appearance-based methods rely on feature points and are prone to failure in ambiguous environments. On the other hand, object recognition can infer objects’ category, extent, and pose. We can use these objects as stable semantic landmarks, suitable for viewpoint-independent and non-ambiguous loop closures. However, there is a critical data association problem due to the lack of efficient and robust algorithms.

We introduce a novel object-level data association algorithm, which incorporates 2D intersection over union and instance-level embeddings, formulated as a linear assignment problem. After solving the data association p...[ Read more ]

Loop closure can effectively correct the accumulated error in robot localization, which plays a critical role in the long-term navigation of the robot. Traditional appearance-based methods rely on feature points and are prone to failure in ambiguous environments. On the other hand, object recognition can infer objects’ category, extent, and pose. We can use these objects as stable semantic landmarks, suitable for viewpoint-independent and non-ambiguous loop closures. However, there is a critical data association problem due to the lack of efficient and robust algorithms.

We introduce a novel object-level data association algorithm, which incorporates 2D intersection over union and instance-level embeddings, formulated as a linear assignment problem. After solving the data association problem, we model the objects in the environment as TSDF volumes and reconstruct the environment as a graph of 3D objects with semantics and topology.

Based on the 3D semantic graphs, we propose a robust semantic graph matching-based approach for loop closure recognition. Then, we correct the accumulated drift by aligning the matched objects between the local and global 3D semantic graphs. Finally, we jointly optimize object poses and camera trajectories in pose graph optimization.

Experimental results show that the proposed object-level data association method significantly outperforms the commonly used nearest neighbor method in accuracy. Compared with existing appearance-based loop closure methods, the semantic graph matching-based loop closure recognition method is more robust to environmental appearance changes.