عنوان انگلیسی مقاله:
Evaluating semi-cooperative Nash/Stackelberg Q-learning for traffic routes plan in a single intersection
ترجمه فارسی عنوان مقاله:
ارزیابی شبه شرکتی Q / Nash / Stackelberg برای طرح مسیرهای ترافیکی در یک تقاطع واحد
Sciencedirect - Elsevier - Control Engineering Practice, 102 (2020) 104525. doi:10.1016/j.conengprac.2020.104525
Jian Guo a,∗, Istvan Harmati b
As traffic congestion grows tremendous and frequent in the urban transportation system, many efficient models
with reinforcement learning (RL) methods have already been proposed to optimize this situation. A multi-agent
reinforcement learning (MARL) system can be constructed from the traffic problem, where the incoming links
(i.e., sections) are regarded as agents and the actions made by the agents are for controlling signal lights.
A semi-cooperative Nash Q-learning approach on the basis of single-agent Q-learning and Nash equilibrium
is proposed and presented in this paper, in which the agents agree on the process of action selection by
Nash equilibrium, but strive finally for a common goal with cooperative behaviour when more than one Nash
equilibriums exist. Then an extended version called semi-cooperative Stackelberg Q-learning is designed to
make a comparison, where Nash equilibrium is replaced by Stackelberg equilibrium in the Q-learning process.
Specifically, the agent who has the largest queues will be promoted as a leader and the others are followers who
react to the leader’s decision. Instead of adjusting the plan of green light timing published in other research,
this paper is contributing to finding the best multi-routes plan for passing most vehicles in a single traffic
intersection, with combining game theory and RL in decision-making in the multi-agent framework. These
two multi-agent Q-learning methods are implemented and compared with the constant strategy (i.e., the time
intervals of green or red lights are fixed and periodical). The simulated result shows that the performance of
semi-cooperative Stackelberg Q-learning is better.
Keywords: Traffic routes plan | Multi-agent system | Semi-cooperation | Reinforcement learning | Game theory