THESIS
2023
1 online resource (xvii, 203 pages) : illustrations (chiefly color)
Abstract
This PhD thesis delves into two distinct yet interconnected research areas within tensor
estimation, both utilizing the Riemannian gradient descent (RGrad) algorithm. In
the first part, we propose a generalized framework for estimating a latent low-rank plus
sparse tensor. This framework captures the multi-way principal components in the low-rank
tensor, while the sparse tensor accommodates potential model mis-specifications
and heterogeneous signals beyond the explanatory power of the low-rank component.
Our flexible framework encompasses both linear and generalized linear models and seamlessly
handles continuous and categorical variables. To efficiently estimate the tensors,
we develop a fast algorithm by integrating the Riemannian gradient descent with a novel
gradient pruning proced...[
Read more ]
This PhD thesis delves into two distinct yet interconnected research areas within tensor
estimation, both utilizing the Riemannian gradient descent (RGrad) algorithm. In
the first part, we propose a generalized framework for estimating a latent low-rank plus
sparse tensor. This framework captures the multi-way principal components in the low-rank
tensor, while the sparse tensor accommodates potential model mis-specifications
and heterogeneous signals beyond the explanatory power of the low-rank component.
Our flexible framework encompasses both linear and generalized linear models and seamlessly
handles continuous and categorical variables. To efficiently estimate the tensors,
we develop a fast algorithm by integrating the Riemannian gradient descent with a novel
gradient pruning procedure. Under suitable conditions, the algorithm exhibits linear convergence
and enables simultaneous estimation of both the low-rank and sparse tensors.
We establish statistical error bounds for the final estimates based on the gradient of the
loss function. Notably, the error bounds are sharp under specific statistical models, such
as the sub-Gaussian robust PCA and Bernoulli tensor model. Moreover, our method
achieves non-trivial error bounds for heavy-tailed tensor PCA when the noise has a finite
2 + ε moment. We apply our proposed method to analyze the international trade
flow dataset and the statistician hypergraph co-authorship network, unveiling new and
intriguing findings. In the second part, we focus on tensor completion within the tensor train (TT) format, which offers significant advantages in handling high-order tensors
with complex structures. Despite the wide range of applications for TT-format tensors in
various disciplines, theoretical guarantees for tensor completion algorithms specifically designed
for this format are largely missing or sub-optimal. This is partly attributed to the
intricate and recursive algebraic operations involved in TT-format decomposition, which
differ substantially from algorithms designed for other tensor formats such as Tucker
and CP. In this thesis, we provide the first theoretical guarantees for the convergence
of the Riemannian gradient descent (RGrad) algorithm in TT-format tensor completion
under a nearly optimal sample size condition. The RGrad algorithm achieves linear
convergence with a constant contraction rate that remains independent of the tensor’s
condition number, eliminating the need for re-conditioning. Additionally, we introduce
a novel approach called the sequential second-order moment method, enabling warm initialization
with similar sample size requirements. Furthermore, our results significantly
refine prior investigations of the RGrad algorithm for matrix completion. Finally, we
derive statistically optimal (or near-optimal) rates for the RGrad algorithm when the
observed tensor entries are corrupted by random sub-Gaussian noise. Numerical experiments
validate our theoretical discoveries and demonstrate the computational speedup
gained through TT-format decomposition.
Post a Comment