This work explores the reference-based talking face video super-resolution for video conferencing even under low bandwidth network conditions. Our objective is to reconstruct high-quality talking face video with given low-resolution video and sparsely given high-resolution frames for every ten frames. To this end, our method utilizes the pretrained GANs as priors knowledge to reconstruct photo-realistic face images. Using GANs pretrained on a large dataset is much helpful to generate plausible face images even with the low-resolution images; however, it shows low fidelity. It means that the person's face identity between original and reconstructed ones is quite different. To tackle this problem, our method is designed to exploit the high-resolution feature which can help generate high-f...[ Read more ]

This work explores the reference-based talking face video super-resolution for video conferencing even under low bandwidth network conditions. Our objective is to reconstruct high-quality talking face video with given low-resolution video and sparsely given high-resolution frames for every ten frames. To this end, our method utilizes the pretrained GANs as priors knowledge to reconstruct photo-realistic face images. Using GANs pretrained on a large dataset is much helpful to generate plausible face images even with the low-resolution images; however, it shows low fidelity. It means that the person's face identity between original and reconstructed ones is quite different. To tackle this problem, our method is designed to exploit the high-resolution feature which can help generate high-fidelity face images. The proposed method exploits the recent development of reference-based super-resolution techniques and generative priors. We aggregate these two approaches into a single framework. Experimental results show that the proposed method can generate high-fidelity talking face video when more reference frames are given.