Surface reconstruction from images is an old, fundamental yet difficult problem in computer vision, which has been extensively investigated over the past three decades. Recent developments in camera calibration and multimedia computing have broadened the interests in the reconstruction problem, and the big proliferation of digital cameras and computers allow to take and process multiple images respectively. The emphasis of the nowaday applications has been shifted to generating more "appearance" views of the scene, and it requires highly detailed surfaces to be constructed within a tol-erable computational resources. Such a task, however, is difficult to accomplish due to the intrinsic ill-posedness of the reconstruction, interference from image noise, and insufficiency of scalability a...[ Read more ]

Surface reconstruction from images is an old, fundamental yet difficult problem in computer vision, which has been extensively investigated over the past three decades. Recent developments in camera calibration and multimedia computing have broadened the interests in the reconstruction problem, and the big proliferation of digital cameras and computers allow to take and process multiple images respectively. The emphasis of the nowaday applications has been shifted to generating more "appearance" views of the scene, and it requires highly detailed surfaces to be constructed within a tol-erable computational resources. Such a task, however, is difficult to accomplish due to the intrinsic ill-posedness of the reconstruction, interference from image noise, and insufficiency of scalability and flexibility.

Mathematically, scene objects can be represented as smooth surfaces with arbitrary topology, and surface reconstruction from images could be cast into various problems and formulations. In this dissertation, we focus on the mathematical descriptions and representations, and discuss the reconstruction approaches that use volumetric, graph-cut, and level-set optimization tools, and the objective functionals that use different image cues, such as silhouette, photometry, texture and shading. The dissertation is also a general introduction to the surface reconstruction background knowledge, and a general discussion and comparison among different kinds of reconstruction methods.

In despite of the successful mathematical solutions to the ill-posedness of the re-construction and the interference from image noise, the scalability and flexibility issues remain largely unaddressed. In order to overcome the limitations stemmed from these issues, we further propose the patchwork representation and reconstruction, which is a collection of small surface patches progressively and stitched together. This patchwork framework is capable of retrieving both complete shapes and open surfaces.

The final part of this dissertation contains a discussion beyond multi-view sur-face reconstruction. This is because, in some situations, when only a single image is provided or when the structural information is predominant, multi-view surface re-construction is not capable of modeling the scene. The main focuses are given to single-view surface reconstruction and multi-view space curve reconstruction. Mean-while, shape-from-shading is formulated with the classical graph theory of finding the shortest path on a network, and solved with the Fast Marching methods. Space curve is represented by an implicit distance function, and optimized using the variational framework.