An MDP decomposition approach for traffic control at isolated signalized intersections

This article presents a novel approach for the dynamic control of a signalized intersection.
At the intersection, there is a number of arrival flows of cars, each having
a single queue (lane). The set of all flows is partitioned into disjoint combinations
of nonconflicting flows that will receive green together. The dynamic control of the
traffic lights is based on the numbers of cars waiting in the queues. The problem
concerning when to switch (and which combination to serve next) is modeled as a
Markovian decision process in discrete time. For large intersections (i.e., intersections
with a large number of flows), the number of states becomes tremendously
large, prohibiting straightforward optimization using value iteration or policy iteration.
Starting from an optimal (or nearly optimal) fixed-cycle strategy, a one-step
policy improvement is proposed that is easy to compute and is shown to give a close
to optimal strategy for the dynamic problem.