Development and evaluation of an arterial adaptive traffic signal control system using reinforcement learning

摘要

This dissertation develops and evaluates a new adaptive traffic signal controlsystem for arterials. This control system is based on reinforcement learning, which is animportant research area in distributed artificial intelligence and has been extensivelyused in many applications including real-time control.In this dissertation, a systematic comparison between the reinforcement learningcontrol methods and existing adaptive traffic control methods is first presented from thetheoretical perspective. This comparison shows both the connections between them andthe benefits of using reinforcement learning. A Neural-Fuzzy Actor-CriticReinforcement Learning (NFACRL) method is then introduced for traffic signal control.NFACRL integrates fuzzy logic and neural networks into reinforcement learning and canbetter handle the curse of dimensionality and generalization problems associated withordinary reinforcement learning methods.This NFACRL method is first applied to isolated intersection control. Twodifferent implementation schemes are considered. The first scheme uses a fixed phase sequence and variable cycle length, while the second one optimizes phase sequence inreal time and is not constrained to the concept of cycle. Both schemes are furtherextended for arterial control, with each intersection being controlled by one NFACRLcontroller. Different strategies used for coordinating reinforcement learning controllersare reviewed, and a simple but robust method is adopted for coordinating traffic signalsalong the arterial.The proposed NFACRL control system is tested at both isolated intersection andarterial levels based on VISSIM simulation. The testing is conducted under differenttraffic volume scenarios using real-world traffic data collected during morning, noon,and afternoon peak periods. The performance of the NFACRL control system iscompared with that of the optimized pre-timed and actuated control.Testing results based on VISSIM simulation show that the proposed NFACRLcontrol has very promising performance. It outperforms optimized pre-timed andactuated control in most cases for both isolated intersection and arterial control. At theend of this dissertation, issues on how to further improve the NFACRL method andimplement it in real world are discussed.