THESIS
2022
1 online resource (xiii, 95 pages) : illustrations (some color)
Abstract
Face reconstruction and reenactment have been extensive research for decades due to their
fundamental research status and wide applications. Previous reconstruction methods request
multi-view input or different illumination conditions. Current applications require less constrained
settings, even from a monocular image. These approaches rely on face priors, commonly
known as 3D Morphable Models (3DMMs). One target of this thesis aims to use the
analysis-by-synthesis idea to reconstruct 3DMMs from monocular input. We propose a self-supervised
training architecture considering the 3DMM as a face prior. We analyze the 3D face
by Deep Neural Network (DNN) and then synthesize the 2D facial landmark and rendering
results by projection and differentiable rendering. Then, we leverage the synthes...[
Read more ]
Face reconstruction and reenactment have been extensive research for decades due to their
fundamental research status and wide applications. Previous reconstruction methods request
multi-view input or different illumination conditions. Current applications require less constrained
settings, even from a monocular image. These approaches rely on face priors, commonly
known as 3D Morphable Models (3DMMs). One target of this thesis aims to use the
analysis-by-synthesis idea to reconstruct 3DMMs from monocular input. We propose a self-supervised
training architecture considering the 3DMM as a face prior. We analyze the 3D face
by Deep Neural Network (DNN) and then synthesize the 2D facial landmark and rendering
results by projection and differentiable rendering. Then, we leverage the synthesis results to
provide supervision for the DNN training. However, the 3DMM is a statistic model learned
from datasets of 3D scans, which contains limited expressions and loses face detail. Hence,
another goal of this thesis is to construct a digital avatar construction pipeline, which provides
avatars with vivid face shapes and expressions to build a face model. Besides, it can also offer
face details. The last goal of this thesis is 3D-awareness face reenactment, which synthesizes
images combining the source image identity and the facial expression and pose of driving video
frames under 3D consistency. Recent methods regard 2D landmarks as face shape, expression,
and pose representations, which are affected by the entangling of face shape, expression and
pose. Hence, we push the state-of-the-art in high-quality face reenactment in a synthesis-by-analysis
strategy. In this way, we synthesize images under the analysis of the 3D face, which is a
more robust representation than facial landmarks. Hence, this thesis’s key idea is to analyze and
synthesize human faces. Finally, we construct a facial expression recognition network, where face analysis and synthesis are applied to improve the recognition results.
Post a Comment