THESIS
2019
xiii, 113 pages : illustrations ; 30 cm
Abstract
Learning sparse representations by sparse coding has been used in many applications for
decades. Recently, convolutional sparse coding (CSC) improves sparse coding by learning
a shift-invariant dictionary and convolutional sparse representations from the data. It
has been successfully extracting local patterns from various data types, such as trajectories,
images, audios, videos, multi-spectral and light field images, and biomedical data.
However, most existing CSC algorithms operate in the batch mode and are computationally
expensive. This lack of scalability restricts the use of CSC on large-scale data. Apart
from that, existing CSC works mainly assume that the noise in the data is from Gaussian
distribution, which can be restrictive and does not suit many real-world problems....[
Read more ]
Learning sparse representations by sparse coding has been used in many applications for
decades. Recently, convolutional sparse coding (CSC) improves sparse coding by learning
a shift-invariant dictionary and convolutional sparse representations from the data. It
has been successfully extracting local patterns from various data types, such as trajectories,
images, audios, videos, multi-spectral and light field images, and biomedical data.
However, most existing CSC algorithms operate in the batch mode and are computationally
expensive. This lack of scalability restricts the use of CSC on large-scale data. Apart
from that, existing CSC works mainly assume that the noise in the data is from Gaussian
distribution, which can be restrictive and does not suit many real-world problems. In this
thesis, we first propose a scalable online CSC algorithm called OCSC for data sets of large
quantity. The key is a reformulation of the CSC objective so that convolution can be handled
easily in the frequency domain, and much smaller space is needed. Empirical results
validate that OCSC is more scalable, has faster convergence and better reconstruction performance.
Further, instead of convolving with a dictionary shared by all samples, we propose
the use of a sample-dependent dictionary in which each filter is a linear combination
of a small set of base filters learned from data. This added flexibility allows a large number
of sample-dependent patterns to be captured, which is especially useful in the handling
of large or high-dimensional data sets. Computationally, the resultant model can be efficiently
learned by online learning. Finally, we propose a general CSC model capable of
learning convolutional filters and representations from data with complicated unknown
noise. The noise is now modeled by Gaussian mixture model, which can approximate
any continuous probability density function. We use the expectation-maximization algorithm
to solve the problem and design an efficient method for the weighted CSC problem
in the maximization step. The crux is to speed up the convolution in the frequency domain
while keeping the other computation involving weight matrix in the spatial domain.
We show that this method obtains comparable time and space complexity compared with
existing CSC methods, models noise effectively and obtains high-quality filters and representation.
In sum, we propose a series of works to make CSC scalable to deal with large
data, capable of extracting a large number of local patterns, and free of contamination of
complicated noises. Therefore, better representations and dictionary can be obtained.
Post a Comment