In recent years, wireless networking is booming all around the world and has attracted many research attentions. In the current wireless networks, most protocols are mainly adopted from their wired counterparts, which are designed on wired environment assumptions and use static strategies. However, wireless environments are characterized by error prone and time-varying wireless channels, user mobility, etc. These dynamic features make those static strategies not achieve good performance in wireless environments. Therefore, it is quite desirable to develop an on-line strategy to optimize the system performance by adapting to the time varying wireless environments....[ Read more ]

In recent years, wireless networking is booming all around the world and has attracted many research attentions. In the current wireless networks, most protocols are mainly adopted from their wired counterparts, which are designed on wired environment assumptions and use static strategies. However, wireless environments are characterized by error prone and time-varying wireless channels, user mobility, etc. These dynamic features make those static strategies not achieve good performance in wireless environments. Therefore, it is quite desirable to develop an on-line strategy to optimize the system performance by adapting to the time varying wireless environments.

As a fundamental tool in optimization, Markov Decision Process (MDP) theory is widely used in system control problems, such as routing and scheduling in communication networks. A recently proposed performance potential based solution to MDP problems can significantly reduce the huge computation of the standard solution, thus applicable for on-line optimization in communication systems.

In this research, we formulate two applications in wireless systems (CDMA call admission control and adaptive error control) as MDP problems, and propose single sample path based on-line optimization algorithms as the solutions. The algorithm is proved to be effective by simulations, and its low computational complexity and flexibility shows its potential in communication applications, especially in wireless networks.