The objective of this research is to develop graph signal processing techniques to address challenges in compression and restoration of piecewise smooth (PWS) images, which have wide applications to 3D video coding, depth-image-based rendering, image segmentation, etc. While generic image processing tools are applicable to these problems, PWS images (e.g., depth maps or animation images) possess unique signal characteristics, such as sharp object boundaries and slowly-varying interior surfaces, which we exploit for better performance. Specifically, leveraging on recent advances in graph signal processing, we essentially propose compact representations of PWS images in the graph transform domain or in the graph spatial domain. The effective representations of PWS images contribu...[ Read more ]

The objective of this research is to develop graph signal processing techniques to address challenges in compression and restoration of piecewise smooth (PWS) images, which have wide applications to 3D video coding, depth-image-based rendering, image segmentation, etc. While generic image processing tools are applicable to these problems, PWS images (e.g., depth maps or animation images) possess unique signal characteristics, such as sharp object boundaries and slowly-varying interior surfaces, which we exploit for better performance. Specifically, leveraging on recent advances in graph signal processing, we essentially propose compact representations of PWS images in the graph transform domain or in the graph spatial domain. The effective representations of PWS images contribute to improved compression and restoration performance.

In the transform domain, Graph Fourier Transform (GFT) representation provides approximately optimal energy compaction of PWS images when the graph is appropriately constructed, which is useful for compact compression. We first propose a multi-resolution GFT scheme for compression of PWS images. In particular, we propose an optimality criterion of the GFT, which minimizes the total signal representation cost of each pixel block, considering both the sparsity of the signal's transform coefficients and the compactness of transform description. Efficient algorithms are developed to search for optimal GFTs, leveraging on graph optimization techniques. Further, we introduce two techniques to reduce computation complexity of the GFT. Experimental results show that we outperform a representative compression codec H.264 intra by 6.8 dB on average in PSNR at the same bit rate.

Secondly, we extend the first work to transform coding of intra-prediction residues. Conventional intra-prediction schemes copy directly from known pixels across block boundaries as prediction. Instead, we propose a cluster-based intra-prediction to take into account discontinuities occurring at block boundaries. Then we develop a generalized G FT optimized for the signal modeling of the intra-prediction residues, which proves to optimally decorrelate the signal. Using PWS and natural images as examples, we outperform combinations of previous intra-prediction and Asymmetric Discrete Sine Transform coding by 2.5dB in PSNR on average.

Thirdly, we take advantage of efficient GFT representations for one restoration problem of PWS images--denoising from Additive White Gaussian Noise. Instead of constructing GFTs from a local block as done in previous works, we derive the GFT from a group of similar non-local patches so as to represent the common structure underlying this group. This non-local GFT representation jointly exploits local smoothness and non-local self-similarity of a PWS image. We outperform state-of-the-art image denoising methods such as BM3D by up to 2.37 dB in PSNR.

Finally, we represent PWS images in graph spatial domain, in the form of a graph smoothness prior. Based on this prior, we address the other restoration problem--dequantization. For PWS signals distorted due to quantization, we derive the maximum a posteriori (MAP) estimation of the latent signal, based on the graph smoothness prior of the latent signal and two priors of the corresponding graph. This MAP estimation then translates to a joint optimization of the latent signal and the corresponding graph. Further, an efficient algorithm is developed to alternately optimize the two variables. Experimental results show that the proposed approach outperforms one state-of-the-art image dequantization method by 1dB in PSNR.