There is a surge of interest in efficient resource control for future wireless communication and control systems. Different applications, such as video streaming, content-centric caching, online gaming, pose different levels of challenges to the stochastic resource control problems. The existing literatures on improving the physical layer performance cannot be extended to solve the above-mentioned problems that are related to the application level performance metrics in different application scenarios. Even though Markov decision process (MDP) is a useful tool to formulate the stochastic optimization problems, the conventional solution such as value iteration and policy evaluation algorithms, which suffer from slow convergence and a lack of design insights. In this talk, we focus on obt...[ Read more ]

There is a surge of interest in efficient resource control for future wireless communication and control systems. Different applications, such as video streaming, content-centric caching, online gaming, pose different levels of challenges to the stochastic resource control problems. The existing literatures on improving the physical layer performance cannot be extended to solve the above-mentioned problems that are related to the application level performance metrics in different application scenarios. Even though Markov decision process (MDP) is a useful tool to formulate the stochastic optimization problems, the conventional solution such as value iteration and policy evaluation algorithms, which suffer from slow convergence and a lack of design insights. In this talk, we focus on obtaining low complexity stochastic control solutions to the following specific problems: i) online trajectory and power management for cache-enabled single-UAV wireless systems; ii) online trajectory and radio resource management for cache-enabled multi-UAV wireless systems. We formulate the associated stochastic optimization problems as infinite horizon average cost MDP problems. For each of the above problems, we apply our data-driven learning based approaches, including policy gradient, reinforcement learning and PDE learning, to obtain low complexity and insightful solutions. Note that even though our proposed approaches offer a framework for obtaining low complexity approximate solutions to the original MDP, it still is a case-by-case problem for actually solving the associated optimality conditions (e.g., a multi-dimensional partial differential equation). In addition, the proposed solution for each problem is compared with some state-of-the-art baselines and it is shown through simulations that significant performance gain can be achieved.