With the emergence of consumer RGB-D camera, interdisciplinary research in computer vision, graphics and robotics experienced huge growth in recent years. Aiming at the intelligent human body 3D reconstruction and motion capture, we adopt the aerial robot that employed with RGB-D camera as the flying camera, and present two novel works in this thesis: iHuman3D and FlyFusion for automated, adaptive, and real-time human body 3D reconstruction and motion capture.

Specifically, for static human full body 3D reconstruction, a real-time and active view planning system iHuman3D is proposed based on a highly efficient ray casting algorithm in GPU and a novel information gain formulation directly in Truncated Signed Distance Function (TSDF). Human body reconstruction module revises the...[ Read more ]

With the emergence of consumer RGB-D camera, interdisciplinary research in computer vision, graphics and robotics experienced huge growth in recent years. Aiming at the intelligent human body 3D reconstruction and motion capture, we adopt the aerial robot that employed with RGB-D camera as the flying camera, and present two novel works in this thesis: iHuman3D and FlyFusion for automated, adaptive, and real-time human body 3D reconstruction and motion capture.

Specifically, for static human full body 3D reconstruction, a real-time and active view planning system iHuman3D is proposed based on a highly efficient ray casting algorithm in GPU and a novel information gain formulation directly in Truncated Signed Distance Function (TSDF). Human body reconstruction module revises the traditional volumetric fusion pipeline with a compactly-designed non-rigid deformation for slight motion of the human target. Both the active view planning and human body reconstruction are unified in the same TSDF volume-based representation. On the other hand, for dynamic human motion capture, following the active reconstruction clue, a Geometry And Motion Energy (GAME) metric for guiding the viewpoint optimization in the volumetric space, proposed FlyFusion succeeds to enable active viewpoint selection based on the immediate dynamic reconstruction geometry and predicted human motion. Quantitative and qualitative experiments are conducted to validate that the proposed systems effectively remove the constraints of fixed capture volume and extra manual labor, enabling real-time and intelligent human body 3D reconstruction and motion capture. Given above distinctiveness, we believe our work bridges the gap in robotics and human modelling, and will further promote both robotics, computer vision and graphics communities in interactive 3D human reconstruction.