Learning sparse representations by sparse coding has been used in many applications for decades. Recently, convolutional sparse coding (CSC) improves sparse coding by learning a shift-invariant dictionary and convolutional sparse representations from the data. It has been successfully extracting local patterns from various data types, such as trajectories, images, audios, videos, multi-spectral and light field images, and biomedical data. However, most existing CSC algorithms operate in the batch mode and are computationally expensive. This lack of scalability restricts the use of CSC on large-scale data. Apart from that, existing CSC works mainly assume that the noise in the data is from Gaussian distribution, which can be restrictive and does not suit many real-world problems....[ Read more ]

Learning sparse representations by sparse coding has been used in many applications for decades. Recently, convolutional sparse coding (CSC) improves sparse coding by learning a shift-invariant dictionary and convolutional sparse representations from the data. It has been successfully extracting local patterns from various data types, such as trajectories, images, audios, videos, multi-spectral and light field images, and biomedical data. However, most existing CSC algorithms operate in the batch mode and are computationally expensive. This lack of scalability restricts the use of CSC on large-scale data. Apart from that, existing CSC works mainly assume that the noise in the data is from Gaussian distribution, which can be restrictive and does not suit many real-world problems. In this thesis, we first propose a scalable online CSC algorithm called OCSC for data sets of large quantity. The key is a reformulation of the CSC objective so that convolution can be handled easily in the frequency domain, and much smaller space is needed. Empirical results validate that OCSC is more scalable, has faster convergence and better reconstruction performance. Further, instead of convolving with a dictionary shared by all samples, we propose the use of a sample-dependent dictionary in which each filter is a linear combination of a small set of base filters learned from data. This added flexibility allows a large number of sample-dependent patterns to be captured, which is especially useful in the handling of large or high-dimensional data sets. Computationally, the resultant model can be efficiently learned by online learning. Finally, we propose a general CSC model capable of learning convolutional filters and representations from data with complicated unknown noise. The noise is now modeled by Gaussian mixture model, which can approximate any continuous probability density function. We use the expectation-maximization algorithm to solve the problem and design an efficient method for the weighted CSC problem in the maximization step. The crux is to speed up the convolution in the frequency domain while keeping the other computation involving weight matrix in the spatial domain. We show that this method obtains comparable time and space complexity compared with existing CSC methods, models noise effectively and obtains high-quality filters and representation. In sum, we propose a series of works to make CSC scalable to deal with large data, capable of extracting a large number of local patterns, and free of contamination of complicated noises. Therefore, better representations and dictionary can be obtained.