Incentive Grant
On digital marketplaces, auctions or trading platforms, autonomous algorithms interact, independently trying to maximize their individual profits. This is not science fiction. This is the reality that deep reinforcement learning (DRL) occasions. Combining reinforcement learning (RL) with deep learning (DL), enables machines to approach optimal actions through trial and error, guided by feedback in the form of rewards. However, implementation of (DRL) in interactive environments as opposed to single-agent environments not only holds promise, but harbours several dangers.
Consider the unsettling possibility that, in their individualistic pursuit, these algorithms engage in collusion, distorting markets and causing substantial harm to social welfare. A legitimate concern according to the OECD. This scenario highlights just one of the dangers of autonomous DRL in multiagent systems (MDRL). How can we effectively regulate MDRL and design reliable, scalable algorithms? Addressing these challenges is daunting, especially without theoretical guarantees and a deeper understanding of MDRL methodology.
This is where our project begins. We aim to set the foundations for developing an all-encompassing framework for MDRL in the context of algorithmic collusion to:
(O1) Establish theoretical and computational convergence guarantees.
(O2) Guarantee algorithmic robustness to environmental changes.
(O3) Evaluate average and worst-case performance.
Initially we restrict to DRL for simple pricing environments using two timescale learning for interaction versus adaption.
We will approach these objectives with our interdisciplinary team, featuring Janusz’s expertise on RL in multiagent systems, Sophie’s statistical understanding of DL and Moritz’s proficiency in formal methods, a technique par excellence for improving reliability and developing accompanying tool-support.
For us, this collaboration and the resulting follow-up projects extend beyond a mere academic endeavour; it serves as a foundation for understanding and designing MDRL that is reliable and explainable. Our vision is to contribute to shaping a digital society where autonomous algorithms play a pivotal role, ensuring transparency and accountability in their decision-making process.