This PhD thesis delves into two distinct yet interconnected research areas within tensor estimation, both utilizing the Riemannian gradient descent (RGrad) algorithm. In the first part, we propose a generalized framework for estimating a latent low-rank plus sparse tensor. This framework captures the multi-way principal components in the low-rank tensor, while the sparse tensor accommodates potential model mis-specifications and heterogeneous signals beyond the explanatory power of the low-rank component. Our flexible framework encompasses both linear and generalized linear models and seamlessly handles continuous and categorical variables. To efficiently estimate the tensors, we develop a fast algorithm by integrating the Riemannian gradient descent with a novel gradient pruning proced...[ Read more ]

This PhD thesis delves into two distinct yet interconnected research areas within tensor estimation, both utilizing the Riemannian gradient descent (RGrad) algorithm. In the first part, we propose a generalized framework for estimating a latent low-rank plus sparse tensor. This framework captures the multi-way principal components in the low-rank tensor, while the sparse tensor accommodates potential model mis-specifications and heterogeneous signals beyond the explanatory power of the low-rank component. Our flexible framework encompasses both linear and generalized linear models and seamlessly handles continuous and categorical variables. To efficiently estimate the tensors, we develop a fast algorithm by integrating the Riemannian gradient descent with a novel gradient pruning procedure. Under suitable conditions, the algorithm exhibits linear convergence and enables simultaneous estimation of both the low-rank and sparse tensors. We establish statistical error bounds for the final estimates based on the gradient of the loss function. Notably, the error bounds are sharp under specific statistical models, such as the sub-Gaussian robust PCA and Bernoulli tensor model. Moreover, our method achieves non-trivial error bounds for heavy-tailed tensor PCA when the noise has a finite 2 + ε moment. We apply our proposed method to analyze the international trade flow dataset and the statistician hypergraph co-authorship network, unveiling new and intriguing findings. In the second part, we focus on tensor completion within the tensor train (TT) format, which offers significant advantages in handling high-order tensors with complex structures. Despite the wide range of applications for TT-format tensors in various disciplines, theoretical guarantees for tensor completion algorithms specifically designed for this format are largely missing or sub-optimal. This is partly attributed to the intricate and recursive algebraic operations involved in TT-format decomposition, which differ substantially from algorithms designed for other tensor formats such as Tucker and CP. In this thesis, we provide the first theoretical guarantees for the convergence of the Riemannian gradient descent (RGrad) algorithm in TT-format tensor completion under a nearly optimal sample size condition. The RGrad algorithm achieves linear convergence with a constant contraction rate that remains independent of the tensor’s condition number, eliminating the need for re-conditioning. Additionally, we introduce a novel approach called the sequential second-order moment method, enabling warm initialization with similar sample size requirements. Furthermore, our results significantly refine prior investigations of the RGrad algorithm for matrix completion. Finally, we derive statistically optimal (or near-optimal) rates for the RGrad algorithm when the observed tensor entries are corrupted by random sub-Gaussian noise. Numerical experiments validate our theoretical discoveries and demonstrate the computational speedup gained through TT-format decomposition.