The main part of this thesis is devoted to the framework of covariance matrix estimation in the high-dimensional setting. Two problems are studied under this framework: precision matrix (i.e., the inverse covariance matrix) estimation and linear filter optimization. These problems have wide applications in statistical signal processing and related fields, such as array beamforming, financial engineering, and functional genomics. Motivated by the ubiquity of high-dimensional data in these fields, we focus on both problems in the practical high-dimensional setting, where the observation dimension and the number of samples are comparable in magnitude. In this setting, the traditional methods that intrinsically rely on the availability of a large number of observation samples perform...[ Read more ]

The main part of this thesis is devoted to the framework of covariance matrix estimation in the high-dimensional setting. Two problems are studied under this framework: precision matrix (i.e., the inverse covariance matrix) estimation and linear filter optimization. These problems have wide applications in statistical signal processing and related fields, such as array beamforming, financial engineering, and functional genomics. Motivated by the ubiquity of high-dimensional data in these fields, we focus on both problems in the practical high-dimensional setting, where the observation dimension and the number of samples are comparable in magnitude. In this setting, the traditional methods that intrinsically rely on the availability of a large number of observation samples perform poorly.

High-dimensional precision matrix estimation with limited sample size is first presented. In order to obtain well-behaved estimators in high-dimensional settings, a general class of precision matrix estimators based on weighted sampling and linear shrinkage is proposed. The estimation error is measured in terms of both quadratic loss and Stein’s loss. The performance of the structured estimator with arbitrary weighting parameters is analyzed in an asymptotic setting, and then these parameters are calibrated under specific loss functions. Theoretical results show the asymptotic optimality of uniform weighting in precision matrix estimators, and provide efficient ways to obtain the optimal shrinkage weights. Both synthetic data and real financial market data are used to test the proposed estimators and the advantages are shown with Monte Carlo simulations.

Finite-sample linear filter optimizations in wireless communications and financial systems are then investigated. Under the independent and identical distributed (i.i.d.) assumptions of noise in communications or risky components of financial asset returns, the ideal linear filters are constructed with the inverse covariance matrix and the steering vector or the mean return. Previous results on precision matrix estimation could be applied to mitigate the finite-sample effect, but for this particular problem, more accurate estimation could be achieved. The estimations of the steering vector and the noise covariance (or the mean and covariance of assets in finance) are handled together, and optimal diagonal loading are provided under several widely-used criteria in the fields of wireless communications and finance. All the results are derived in an asymptotic regime where the observation dimension is comparable in magnitude to the number of samples, and Monte Carlo simulations with both synthetic data and real data justify our theories.

The forementioned estimation and optimization techniques are based on the batch mode, where one has to wait until the collection of all the data. In practice, the observation samples can be sequential in time, and online algorithms are of great importance due to the efficiency and memory requirements. In the other part of the thesis, online estimation of sparse signals is studied. In contrast to the benchmark algorithms that update one variable as the arrival of a new observation, the proposed parallel algorithm updates all the variables simultaneously. With a regularization term on each update, the estimated signal using the proposed parallel algorithm is shown to converge to the true sparse signal under mild technical assumptions. Monte Carlo simulations also show that the proposed algorithm has faster convergence speed and higher estimation accuracy compared to the competitors.