Fictitious play in Markov games with single controller

Date
2022-07-12
Advisor
Supervisor
Co-Advisor
Co-Supervisor
Instructor
Source Title
Proceedings of the 23rd ACM Conference on Economics and Computation
Print ISSN
Electronic ISSN
Publisher
Association for Computing Machinery, Inc
Volume
Issue
Pages
919 - 936
Language
English
Type
Conference Paper
Journal Title
Journal ISSN
Volume Title
Series
Abstract

Certain but important classes of strategic-form games, including zero-sum and identical-interest games, have thefictitious-play-property (FPP), i.e., beliefs formed in fictitious play dynamics always converge to a Nash equilibrium (NE) in the repeated play of these games. Such convergence results are seen as a (behavioral) justification for the game-theoretical equilibrium analysis. Markov games (MGs), also known as stochastic games, generalize the repeated play of strategic-form games to dynamic multi-state settings with Markovian state transitions. In particular, MGs are standard models for multi-agent reinforcement learning - a reviving research area in learning and games, and their game-theoretical equilibrium analyses have also been conducted extensively. However, whether certain classes of MGs have the FPP or not (i.e., whether there is a behavioral justification for equilibrium analysis or not) remains largely elusive. In this paper, we study a new variant of fictitious play dynamics for MGs and show its convergence to an NE in n-player identical-interest MGs in which a single player controls the state transitions. Such games are of interest in communications, control, and economics applications. Our result together with the recent results in [42] establishes the FPP of two-player zero-sum MGs and n-player identical-interest MGs with a single controller (standing at two different ends of the MG spectrum from fully competitive to fully cooperative). © 2022 Owner/Author.

Course
Other identifiers
Book Title
Keywords
Fictitious play, Identical-interest games, Markov games, Zero-sum games
Citation
Published Version (Please cite this version)