The main theme of this thesis is the design and analysis of high-dimensional covariance matrix estimators using random matrix theory (RMT). Classical estimators, such as the sample covariance matrix, as well as the robust covariance estimators dealing with heavy-tailed distributed data are known to yield poor performance in modern data-limited or high-dimensional scenarios when the sample size is small compared to the number of variables. In the thesis, two types of high-dimensional covariance matrix estimators are proposed with the application to portfolio optimization in financial engineering and adaptive beamforming in array processing.

We first study the design of a covariance matrix estimator of the portfolio asset returns, aiming to improve the performance of the global m...[ Read more ]

The main theme of this thesis is the design and analysis of high-dimensional covariance matrix estimators using random matrix theory (RMT). Classical estimators, such as the sample covariance matrix, as well as the robust covariance estimators dealing with heavy-tailed distributed data are known to yield poor performance in modern data-limited or high-dimensional scenarios when the sample size is small compared to the number of variables. In the thesis, two types of high-dimensional covariance matrix estimators are proposed with the application to portfolio optimization in financial engineering and adaptive beamforming in array processing.

We first study the design of a covariance matrix estimator of the portfolio asset returns, aiming to improve the performance of the global minimum variance portfolio (GMVP). For large portfolios, the number of available market returns is often of similar order to the number of assets. Additionally, the return observations often exhibit impulsiveness and local loss of stationarity. We address these issues by studying the performance of a hybrid covariance matrix estimator based on Tyler’s robust M-estimator and on Ledoit-Wolf’s shrinkage estimator while assuming samples with heavy-tailed distribution. Employing recent results from RMT, we develop a consistent estimator of (a scaled version of) the realized portfolio risk, which is minimized by optimizing online the shrinkage intensity. Our portfolio optimization method with the proposed covariance matrix estimator is shown via simulations to outperform existing methods both for synthetic and real market data.

The second part of the thesis studies the problem of covariance matrix estimation in minimum variance distortionless response (MVDR) beamforming. We consider high-dimensional settings with large arrays. The problem formulation is similar to that of the GMVP optimization in the first work, and the beamformer’s performance relies on the estimation accuracy of the covariance matrix of the received signals. Due to the structural feature of the covariance matrix, we propose a new covariance estimator based on the recent results on the so-called “spiked models” and a different set of the RMT tools is used. By the design of the covariance matrix estimator with eigenvalue clipping and shrinkage functions that are tailored to the MVDR application, the resulting MVDR solution is shown to outperform classical approaches, as well as more robust solutions, such as methods based on the diagonal loading.