A multi-state Q-learning based CSMA MAC protocol for wireless networks

摘要

Carrier sense multiple access (CSMA) is a very common media access control scheme across various networking topologies and architectures. Whenever multiple agents require the use of a shared transmission medium to send data, some form of orchestration of access is required. Broadly generalized, the medium can either be partitioned or the attached agents can take turns transmitting. CSMA falls under the latter grouping of schemes. Agents first check if another agent is transmitting; if no other device is, then transmission can begin. P-persistent CSMA is one of three variants that deal with the problem of multiple agents all starting transmission as soon as the medium is clear. Under this scheme, each agent will transmit with a probability p and defer to the next time the channel is available with probability 1-p. It should come as no surprise that the optimal value of p depends on the number of agents and how frequently they need to transmit data. In any network where either of these two parameters changes over time, it makes sense that each agent is able to adapt its value of p to the changing circumstances.