The main focus of my research is on studying the dynamics of (strategic) interactions between multiple self-interested learning agents.

My research interests include:

  • Multi-Agent Systems
  • Reinforcement Learning
  • (Evolutionary) Game Theory
  • Robotics

Below is an overview of my ongoing projects.

Framework for multi-agent reinforcement learning

The aim of this project is to extend the existing framework, based on evolutionary game theory (EGT), for multi-agent reinforcement learning. Concepts from EGT, in particular the replicator dynamics, have been linked to reinforcement learning algorithms, making it possible to study and compare the learning behaviour of such algorithms analytically.

Key publications related to this project are:

  • [PDF] [DOI] D. Bloembergen, K. Tuyls, D. Hennes, and M. Kaisers, “Evolutionary Dynamics of Multi-Agent Learning: A Survey,” Journal of Artificial Intelligence Research, vol. 53, p. 659–697, 2015.
    [Bibtex] 
    @article{Bloembergen2015jair,
title={Evolutionary Dynamics of Multi-Agent Learning: A Survey},
author={Bloembergen, Daan and Tuyls, Karl and Hennes, Daniel and Kaisers, Michael},
journal={Journal of Artificial Intelligence Research},
volume={53},
pages={659--697},
year={2015},
month = {8},
doi = {10.1613/jair.4818},
pdf={http://jair.org/papers/paper4818.html},
}

Evolutionary analysis of complex interactions

Many multi-agent interactions are too complex to analyse theoretically in a direct way. However, by taking a high-level view and focussing on so-called heuristic meta-strategies we can still use the tools from evolutionary game theory (see above) to get meaningful insights. We use the framework of empirical game theory to construct heuristic pay-off functions that capture the comparative strength of the high level meta-strategies. These pay-off functions provide insight into the dynamics of the multi-agent system: which meta-strategies can survive under evolutionary pressure. We have so far applied these methods to trading in stock markets, and the space debris removal dilemma.

Key publications related to this project are:

  • [PDF] R. Klima, D. Bloembergen, R. Savani, K. Tuyls, D. Hennes, and D. Izzo, “Space Debris Removal: A Game Theoretic Analysis,” Games, vol. 7, iss. 3, p. 20, 2016.
    [Bibtex] 
    @article{Klima2016games,
title={Space Debris Removal: A Game Theoretic Analysis},
author={Klima, Richard and Bloembergen, Daan and Savani, Rahul and Tuyls, Karl and Hennes, Daniel and Izzo, Dario},
journal={Games},
volume={7},
number={3},
pages={20},
year={2016},
month = {8},
pdf={http://www.mdpi.com/2073-4336/7/3/20}
}
  • [PDF] D. Bloembergen, D. Hennes, P. McBurney, and K. Tuyls, “Trading in markets with noisy information: An evolutionary analysis,” Connection Science, vol. 27, p. 253–268, 2015.
    [Bibtex] 
    @article{Bloembergen2015cs,
author = {Daan Bloembergen and Daniel Hennes and Peter McBurney and Karl Tuyls},
title = {Trading in markets with noisy information: An evolutionary analysis},
journal = {Connection Science},
volume = {27},
issue = {3},
pages = {253--268},
year = {2015},
month = {4},
pdf = {http://www.tandfonline.com/eprint/NEI4DtUyFgHDia8IpBay/full},
}

The evolution of cooperation in complex social networks

The question how cooperative behaviour can be sustained in societies of self-interested individuals has recently received a lot of interest. In this project, we develop a mathematical model for the evolution of cooperation in networked societies, where individuals interact only with their direct neighbours. The model provides insight into how structural properties of the network determine the resulting network-wide behaviour.

Key publications related to this project are:

  • [PDF] D. Bloembergen, I. Caliskanelli, and K. Tuyls, “Learning in Networked Interactions: A Replicator Dynamics Approach,” in Artificial Life and Intelligent Agents, C. J. Headleand, W. J. Teahan, and L. Ap Cenydd, Eds., Springer International Publishing, 2015, vol. 519, pp. 44-58.
    [Bibtex] 
    @incollection{Bloembergen2014alia,
year = {2015},
month = {6},
booktitle = {Artificial Life and Intelligent Agents},
volume = {519},
series = {Communications in Computer and Information Science},
editor = {Headleand, Christopher J. and Teahan, William J. and Ap Cenydd, Llyr},
title = {Learning in Networked Interactions: A Replicator Dynamics Approach},
pdf = {http://www.flowermountains.nl/pub/Bloembergen2014alia.pdf},
publisher = {Springer International Publishing},
author = {Bloembergen, Daan and Caliskanelli, Ipek and Tuyls, Karl},
pages = {44-58},
}
  • [PDF] D. Bloembergen, B. Ranjbar-Sahraei, H. Bou Ammar, K. Tuyls, and G. Weiss, “Influencing Social Networks: An Optimal Control Study,” in Proc. of the 21st Europ. Conf. on Artificial Intelligence (ECAI), 2014, p. 105–110.
    [Bibtex] 
    @inproceedings{Bloembergen2014ecai,
author = {Daan Bloembergen and Bijan {Ranjbar-Sahraei} and Haitham {Bou Ammar} and Karl Tuyls and Gerhard Weiss},
title = {Influencing Social Networks: An Optimal Control Study},
booktitle = {Proc. of the 21st Europ. Conf. on Artificial Intelligence (ECAI)},
editor = {Schaub and Friedrich and O{'}Sullivan},
pages = {105--110},
year = {2014},
month = {8},
pdf = {http://ebooks.iospress.nl/volumearticle/36924}
}
  • [PDF] B. Ranjbar-Sahraei, H. Bou Ammar, D. Bloembergen, K. Tuyls, and G. Weiss, “Evolution of Cooperation in Arbitrary Complex Networks,” in Proc. of 13th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS), 2014, p. 677–684.
    [Bibtex] 
    @inproceedings{Ranjbar2014aamas,
author = {Bijan {Ranjbar-Sahraei} and Haitham {Bou Ammar} and Daan Bloembergen and Karl Tuyls and Gerhard Weiss},
title = {Evolution of Cooperation in Arbitrary Complex Networks},
booktitle = {Proc. of 13th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS)},
pages = {677--684},
publisher = {International Foundation for AAMAS},
editor = {Lomuscio and Scerri and Bazzan and Huhns},
year = {2014},
month = {5},
pdf = {http://www.flowermountains.nl/pub/Ranjbar2014aamas.pdf}
}
  • [PDF] B. Ranjbar-Sahraei, H. Bou Ammar, D. Bloembergen, K. Tuyls, and G. Weiss, “Theory of Cooperation in Complex Social Networks,” in Proc. of the 25th AAAI Conf. on Artificial Intelligence (AAAI), 2014, p. 1471–1477.
    [Bibtex] 
    @inproceedings{Ranjbar2014aaai,
author = {Bijan {Ranjbar-Sahraei} and Haitham {Bou Ammar} and Daan Bloembergen and Karl Tuyls and Gerhard Weiss},
title = {Theory of Cooperation in Complex Social Networks},
booktitle = {Proc. of the 25th AAAI Conf. on Artificial Intelligence (AAAI)},
pages = {1471--1477},
year = {2014},
month = {7},
pdf = {http://www.flowermountains.nl/pub/Ranjbar2014aaai.pdf}
}