Given the 1000x capacity increase requirement for the next-generation cellular networks, i.e., 5G networks, cell densification has been proposed as an important enabling technique. By deploying various types of small cells coexisting with conventional macro base stations (BSs), the network is evolving into a heterogeneous network (HetNet). Consequently, it brings formidable design challenges especially due to the irregular BS locations. Meanwhile, multiple-input multiple-output (MIMO) techniques have proven to be a powerful tool to boost the performance of wireless communication systems, and it will continue to play a pivotal role in 5G networks. The combination of MIMO techniques and HetNets, while promising, brings additional difficulties in system analysis and design. The mai...[ Read more ]

Given the 1000x capacity increase requirement for the next-generation cellular networks, i.e., 5G networks, cell densification has been proposed as an important enabling technique. By deploying various types of small cells coexisting with conventional macro base stations (BSs), the network is evolving into a heterogeneous network (HetNet). Consequently, it brings formidable design challenges especially due to the irregular BS locations. Meanwhile, multiple-input multiple-output (MIMO) techniques have proven to be a powerful tool to boost the performance of wireless communication systems, and it will continue to play a pivotal role in 5G networks. The combination of MIMO techniques and HetNets, while promising, brings additional difficulties in system analysis and design. The main theme of this thesis is to analyze the performance of MIMO HetNets, based on which effective techniques will then be proposed to design and improve the network performance.

To evaluate the performance of such complex networks, the random spatial model is employed, with stochastic geometry as a powerful tool. Compared with existing works normally providing results involving multiple integrals, throughout this thesis, a novel Toeplitz matrix representation is developed to provide highly tractable closed-form expressions. Based on this analytical framework, the thesis explores two fundamental questions: one is how the network densification will affect various performance metrics in HetNets, while the other is how MIMO techniques should be utilized to improve network performance.

The results show that in single-tier networks, cell densification can improve success probability and area spectral efficiency (ASE), while it can increase energy efficiency only if the non-transmission power consumption is less than a certain threshold. In multi-tier MIMO HetNets, cell densification may decrease the success probability. Meanwhile, it is proved that the maximum success probability is achieved by activating only one tier of BSs, while the maximum ASE is achieved by activating all the BSs. This reveals a unique tradeoff between the ASE and link reliability in MIMO HetNets, which has not been identified in previous works.

Although increasing the number of BS antennas can increase success probability with single-user beamforming, the network throughput will be fundamentally limited by intercell interference. Thus we propose a novel interference nulling scheme to improve the success probability, which demonstrates significant performance gains (about 35%-40%) compared with the non-coordination case. Moreover, it is shown that the proposed scheme outperforms other interference nulling methods. Furthermore, a joint jamming and interference nulling scheme is proposed to enhance the network secrecy. Numerical results show that the proposed scheme achieves significant performance gains over the one based only on jamming, which implies the important role of interference management in jamming assisted networks.