THESIS
2022
1 online resource (xiii, 111 pages) : illustrations (chiefly color)
Abstract
3D modeling is an important task to preserve and visualize real-world scenes by computer.
The applications include but are not limited to heritage preserving, city-scale surveys
and AR/VR applications. Typical image-based 3D reconstruction methods include SfM, MVS,
meshing and texturing. This thesis aims at improving the last three steps with neural network
techniques so that the reconstruction pipeline is capable to produce a 3D model with high accuracy
and realistic appearance. First, we introduce visibility handling into learning-based stereo
matching systems to improve the quality of estimated depth maps. We propose to explicitly
detect the visibility and recover the erroneous pixels by neighborhood or other views. Second,
we adopt differentiable rendering in neural implicit surface...[
Read more ]
3D modeling is an important task to preserve and visualize real-world scenes by computer.
The applications include but are not limited to heritage preserving, city-scale surveys
and AR/VR applications. Typical image-based 3D reconstruction methods include SfM, MVS,
meshing and texturing. This thesis aims at improving the last three steps with neural network
techniques so that the reconstruction pipeline is capable to produce a 3D model with high accuracy
and realistic appearance. First, we introduce visibility handling into learning-based stereo
matching systems to improve the quality of estimated depth maps. We propose to explicitly
detect the visibility and recover the erroneous pixels by neighborhood or other views. Second,
we adopt differentiable rendering in neural implicit surface optimization to simultaneously obtain
accurate geometry and realistic appearance. We investigate effective geometric prior and
critical regularizations to improve the ability of generalization and robustness of the system.
Third, we decompose the appearance into environmental lighting and physical-based material
to support efficient rendering in arbitrary novel environments. We discuss possible techniques
to reduce the ambiguity between environment and material, and provide an approximated indirect
illumination handling method to improve the estimation quality in complex scenes. The
proposed modules are extensively evaluated on multiple datasets, including both synthetic and
real-world data.
Post a Comment