Q-learning in regularized mean-field games

Date
2022-05-23
Advisor
Supervisor
Co-Advisor
Co-Supervisor
Instructor
Source Title
Dynamic Games and Applications
Print ISSN
2153-0785
Electronic ISSN
2153-0793
Publisher
Birkhaeuser Science
Volume
Issue
Pages
1 - 29
Language
English
Type
Article
Journal Title
Journal ISSN
Volume Title
Series
Abstract

In this paper, we introduce a regularized mean-field game and study learning of this game under an infinite-horizon discounted reward function. Regularization is introduced by adding a strongly concave regularization function to the one-stage reward function in the classical mean-field game model. We establish a value iteration based learning algorithm to this regularized mean-field game using fitted Q-learning. The regularization term in general makes reinforcement learning algorithm more robust to the system components. Moreover, it enables us to establish error analysis of the learning algorithm without imposing restrictive convexity assumptions on the system components, which are needed in the absence of a regularization term.

Course
Other identifiers
Book Title
Keywords
Mean-field games, Q-learning, Regularized Markov decision processes, Discounted reward
Citation
Published Version (Please cite this version)