Online resource allocation problems have attracted considerable attention in today’s data-driven world, where decision-making processes often rely on real-time information and samples. Effective algorithms must strike a delicate balance between exploration and exploitation. In this thesis, we contribute decision-making algorithms for two online resource allocation problems. In the first problem, online service platform manages heterogeneous servers to sequentially serve online customers with unknown reward. We integrate multi-armed bandit algorithms into resource allocation to explore and exploit the customer reward information under service resource constraint. Based on the new algorithms, we propose a three-stage architecture for the platform’s long-run operation. In the second proble...[ Read more ]

Online resource allocation problems have attracted considerable attention in today’s data-driven world, where decision-making processes often rely on real-time information and samples. Effective algorithms must strike a delicate balance between exploration and exploitation. In this thesis, we contribute decision-making algorithms for two online resource allocation problems. In the first problem, online service platform manages heterogeneous servers to sequentially serve online customers with unknown reward. We integrate multi-armed bandit algorithms into resource allocation to explore and exploit the customer reward information under service resource constraint. Based on the new algorithms, we propose a three-stage architecture for the platform’s long-run operation. In the second problem, an agent sequentially allocates measurement efforts to select the best set of arms with the highest means in a multi-armed bandit experiment. We characterize the necessary and sufficient conditions for the optimal allocation problem. We reveal the connection between the Karush–Kuhn–Tucker conditions and top-two algorithm design principle, initially proposed for best-arm-identification. We propose a simple and effective selection rule dubbed information-directed selection (IDS) that selects one of the top-two candidates based on a measure of information gain. Extensive numerical experiments show the superior performance of the proposed top-two algorithms with IDS.