基于马尔可夫决策模型的异构无线网络切换选择算法∗

摘要

针对异构环境下不同业务类型用户对于接入网络的不同服务质量(quality of service, QoS)需求,该文提出了一种基于马尔可夫决策模型的切换选择算法。建立基于软件定义网络(software defined network, SDN)的异构无线网络架构,以实现对异构网络的通透控制。利用马尔可夫过程预测下一时刻的网络状态以得到采取动作后的一次回报,依据网络的不同状态属性针对实时用户和非实时用户分别构建立即回报函数,并采用层次分析法确定属性权重；基于状态动作对构建期望回报函数,采用逐次逼近的迭代方式得到使长期期望回报最大的切换策略。仿真结果表明,该方法针对不同业务类型用户均能选取最优切换策略,同时降低阻塞率,提高了用户的QoS和无线网络的资源利用率。%Coexistence of multiple wireless access technologies will be an indicator of next-generation wireless network, and the integration of heterogeneous wireless networks will meet the needs of high-performance services for mobile users. According to unique quality of service (QoS) requirements of different service type users in heterogeneous environment, the Markov decision model based handoff selection algorithm is proposed in this paper. A heterogeneous wireless network architecture based on the software defined network (SDN) is established to realize the transparency control of heterogeneous networks. Network state information of heterogeneous wireless networks is mastered by SDN controller. It is responsible for scheduling network resources dynamically according to the performance characteristics of each network. If the network state information in equal interval is sampled, the next moment state of network is only related to the current network state and action, but it is not related to the historical state. The problem of handoff selection for heterogeneous wireless networks is modeled as a Markov process with discrete time and continuous state. To predict the next moment state of network by Markov process to obtain a reward, when the reward is positive, it represents the income; when it is negative, it represents the cost. An immediate reward function is constructed for real-time service and non real-time service users respectively according to their different state attributes of the network. Considering five state attributes of wireless network as follows: delay, delay jitter, bandwidth, error rate and network load, the immediate reward function is constructed with weighted summation. Due to the difference in attribute weight distribution among different service type users, the attribute weights are determined by the analytic hierarchy process. In the long term, the objective function which consists of immediate reward function sequence is used to measure future long-term rewards. Then expected reward function based on the state action pair is constructed to obtain the handoff strategy of the maximum expected return by the iterative method of successive approximation. The proposed Markov decision model based handoff selection algorithm is used in simulation of the Matlab platform. The simulation results show that the proposed method can select the optimal handoff strategy for different service type users and reduce the blocking rate, thereby improving the QoS of users and resource utilization of wireless networks.