Modern data arising from various domains, such as audio, image and text, are often high-dimensional and contain spurious features with various structures. In most cases, a simple model is at a more favorable side than complicated ones, since it often provides better generalization performance, together with intuitive interpretation.

The structured-sparsity-inducing regularizers are highly desirable in this case. In this thesis, several extensions of existing sparse models are proposed, which can be used to encourage feature selection in 1. groups; 2. hierarchy 3. graph; and 4. clusters.

As the resulted objectives are often challenging to optimize, due to the nonsmoothness or even nonconvexity of the regularizers. To overcome this obstacle, I developed a set of first-order algo...[ Read more ]

Modern data arising from various domains, such as audio, image and text, are often high-dimensional and contain spurious features with various structures. In most cases, a simple model is at a more favorable side than complicated ones, since it often provides better generalization performance, together with intuitive interpretation.

The structured-sparsity-inducing regularizers are highly desirable in this case. In this thesis, several extensions of existing sparse models are proposed, which can be used to encourage feature selection in 1. groups; 2. hierarchy 3. graph; and 4. clusters.

As the resulted objectives are often challenging to optimize, due to the nonsmoothness or even nonconvexity of the regularizers. To overcome this obstacle, I developed a set of first-order algorithms based on the proximal method for a wide range of structured regularization problems, including convex and nonconvex objectives in both deterministic or stochastic settings. Theoretically, these algorithms enjoy low per-iteration complexity with convergence (rate) guarantee.

Experiments on a number of synthetic and real data sets demonstrate the advantage of proposed models and the efficiency of developed algorithms.