Recent years have witnessed a plethora of learning-based solutions, especially ones adopting deep reinforcement learning (DRL), for congestion control, which show outstanding performance improvement compared to traditional TCP schemes. However, several challenges still remain when incorporating deep reinforcement learning into the classic control task in networking. Some of them are intrinsic and have not been solved by the current DRL-based CC schemes, e.g., the fairness issue; Some are introduced by learning-based algorithms adopting deep neural networks, e.g., the overhead issue and the generalization issue; Furthermore, new demands arise to extend capability and flexibility of CC schemes, e.g., multiple objectives. These problems hinder network transport designers and operators from...[ Read more ]

Recent years have witnessed a plethora of learning-based solutions, especially ones adopting deep reinforcement learning (DRL), for congestion control, which show outstanding performance improvement compared to traditional TCP schemes. However, several challenges still remain when incorporating deep reinforcement learning into the classic control task in networking. Some of them are intrinsic and have not been solved by the current DRL-based CC schemes, e.g., the fairness issue; Some are introduced by learning-based algorithms adopting deep neural networks, e.g., the overhead issue and the generalization issue; Furthermore, new demands arise to extend capability and flexibility of CC schemes, e.g., multiple objectives. These problems hinder network transport designers and operators from putting DRL-based solutions into practice in the real world.

This thesis presents our effort in solving the above problems. For the fairness issue, we propose Astraea, a novel learning-based solution based on multi-agent reinforcement learning that ensures fast convergence to fairness with stability; For the overhead issue, we propose Spine, a hierarchical congestion control algorithm that fully utilizes the performance gain from DRL but with ultra-low overhead; For the multi-objective requirement from applications, we propose MOCC, a congestion control scheme based on multi-objective reinforcement learning that fits various performance objectives in one single model. For the generalization issue, we propose a transfer learning-based DRL CC that aligns state features from various network conditions. In the future, we will continue to work towards providing a practical, efficient, and flexible DRL-based congestion control scheme with consistently high performance across various network conditions.