The next generation wireless networks are expected to support ubiquitous artificial intelligence (AI) services. These services will include training, inference and deployment of the AI tasks at the edge of wireless networks, which calls for a paradigm shift from connected things to connected intelligence. This stimulates the design of communication systems to meet new performance metrics that have never been considered, e.g., machine learning (ML) model utility instead of bit error rate. In this thesis, we will present an algorithm design for communication-efficient Federated Learning (FL) systems. FL has emerged as a promising collaborative training framework which addresses both the data privacy and the increasing volume of decentralized data at the network edge. Communication-reducti...[ Read more ]

The next generation wireless networks are expected to support ubiquitous artificial intelligence (AI) services. These services will include training, inference and deployment of the AI tasks at the edge of wireless networks, which calls for a paradigm shift from connected things to connected intelligence. This stimulates the design of communication systems to meet new performance metrics that have never been considered, e.g., machine learning (ML) model utility instead of bit error rate. In this thesis, we will present an algorithm design for communication-efficient Federated Learning (FL) systems. FL has emerged as a promising collaborative training framework which addresses both the data privacy and the increasing volume of decentralized data at the network edge. Communication-reduction techniques often introduce trade-offs of the model utility and the overall communication cost. A careful design is essential to balance the trade-offs. A critical step is to theoretically characterize the relationship between the communication cost and the training performance. This thesis enhances the performance of communication-efficient FL systems from the following three different perspectives.

Firstly, we propose and investigate a client-edge-cloud hierarchical FL system to leverage the cloud server’s access to abundant clients’ data, as well as the efficient device-edge communication links. An accurate convergence analysis with respect to the key system parameters is derived, which leads to practical guidelines for important design problems such as the client-edge aggregation and edge-client association strategies. Based on the obtained analytical results, we optimize the two levels of aggregation intervals to minimize the total training delay.

We then investigate an FL system utilizing a different optimization algorithm, namely federated distillation (FD). In this approach, logits rather than model weights are communicated, and knowledge distillation is employed for model aggregation. This method enhances communication efficiency and accommodates model heterogeneity. We present both performance and communication efficiency analyses for the FD algorithm. Based on these analytical results, we propose an active data sampling strategy to further enhance the system’s efficiency and effectiveness.

Lastly, we consider a quantized FL system with differential privacy (DP). We will investigate three DP mechanisms for communication-efficient FL systems, i.e., quantized FL with a binomial mechanism, quantized FL with a quantized gaussian mechanism and binary FL with a random response mechanism. We shall also develop a privacy analysis for the three DP mechanisms. Specifically, we will co-design the quantization and noise level to maximize the model utility for the binomial mechanism.