Please use this identifier to cite or link to this item:
Title: An evolutionary dynamical analysis of multi-agent learning in iterated games
Authors: TUYLS, Karl 
't Hoen, P.
Vanschoenwinkel, B.
Issue Date: 2006
Publisher: Springer
Abstract: In this paper, we investigate Reinforcement learning (RL) in multi-agent systems (MAS) from an evolutionary dynamical perspective. Typical for a MAS is that the environment is not stationary and the Markov property is not valid. This requires agents to be adaptive. RL is a natural approach to model the learning of individual agents. These Learning algorithms are however known to be sensitive to the correct choice of parameter settings for single agent systems. This issue is more prevalent in the MAS case due to the changing interactions amongst the agents. It is largely an open question for a developer of MAS of how to design the individual agents such that, through learning, the agents as a collective arrive at good solutions. We will show that modeling RL in MAS, by taking an evolutionary game theoretic point of view, is a new and potentially successful way to guide learning agents to the most suitable solution for their task at hand. We show how evolutionary dynamics (ED) from Evolutionary Game Theory can help the developer of a MAS in good choices of parameter settings of the used RL algorithms. The ED essentially predict the equilibriums outcomes of the MAS where the agents use individual RL algorithms. More specifically, we show how the ED predict the learning trajectories of Q-Learners for iterated games. Moreover, we apply our results to (an extension of) the COllective INtelligence framework (COIN). COIN is a proved engineering approach for learning of cooperative tasks in MASs. The utilities of the agents are re-engineered to contribute to the global utility. We show how the improved results for MAS RL in COIN, and a developed extension, are predicted by the ED.
Keywords: COllective INtelligence, Evolutionary Game Theory, iterated games, multi-agent systems, reinforcement learning
Document URI:
ISSN: 1387-2532
e-ISSN: 1573-7454
DOI: 10.1007/s10458-005-3783-9
ISI #: 000234922800004
Category: A1
Type: Journal Contribution
Appears in Collections:Research publications

Show full item record


checked on Sep 2, 2020


checked on May 21, 2022

Page view(s)

checked on May 20, 2022

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.