3D reconstruction - inferring the 3D geometry and representation from a collection of 2D images - has been a long-standing task in computer vision for its complexity and wide possible applications especially for its capability to convert 2D images into a more immersive perspective.

From that regard, Neural Radiance Fields (NeRF) propose a neural rendering paradigm for representing a scene as a continuous function in 3D space and inspire many new ways of efficiently representing and effectively rendering a scene. However, most methods are tested on a controlled setting where it has a clean set of images along with its camera information. In a more general and practical setting, images captured by the user will contain various types of noises, including blur, defocus, and background noise...[ Read more ]

3D reconstruction - inferring the 3D geometry and representation from a collection of 2D images - has been a long-standing task in computer vision for its complexity and wide possible applications especially for its capability to convert 2D images into a more immersive perspective.

From that regard, Neural Radiance Fields (NeRF) propose a neural rendering paradigm for representing a scene as a continuous function in 3D space and inspire many new ways of efficiently representing and effectively rendering a scene. However, most methods are tested on a controlled setting where it has a clean set of images along with its camera information. In a more general and practical setting, images captured by the user will contain various types of noises, including blur, defocus, and background noise. Although NeRF-based model structures inherently serve as a scene prior for noise handling, there is no component in NeRF-based models accounting for the potential noise in the source images. Furthermore, existing noise-aware NeRF models typically design a rendering method based on the real physical image formulation process of one specific type of noise. This can limit the robustness of the model against multiple types of noises and also requires the prior knowledge of the type of noise existing in the source images.

With these motivations, this work designed a simple and model-agnostic module for noise-robust neural rendering in Generalizable NeRF (GNeRF) models. For noise robust rendering, this work devises reconstruction-based modules to handle multiple types of noise including burst noise and blur noise. The proposed module shows improvement not only in terms of visual quality but also in stable and consistent prediction over varying noise levels.