Browsing by Subject "Zero-sum games"
Now showing 1 - 2 of 2
Results Per Page
Sort Options
Item Open Access Fictitious play in Markov games with single controller(Association for Computing Machinery, Inc, 2022-07-12) Sayın, Muhammed Ömer; Zhang, Kaiqing; Ozdaglar, AsumanCertain but important classes of strategic-form games, including zero-sum and identical-interest games, have thefictitious-play-property (FPP), i.e., beliefs formed in fictitious play dynamics always converge to a Nash equilibrium (NE) in the repeated play of these games. Such convergence results are seen as a (behavioral) justification for the game-theoretical equilibrium analysis. Markov games (MGs), also known as stochastic games, generalize the repeated play of strategic-form games to dynamic multi-state settings with Markovian state transitions. In particular, MGs are standard models for multi-agent reinforcement learning - a reviving research area in learning and games, and their game-theoretical equilibrium analyses have also been conducted extensively. However, whether certain classes of MGs have the FPP or not (i.e., whether there is a behavioral justification for equilibrium analysis or not) remains largely elusive. In this paper, we study a new variant of fictitious play dynamics for MGs and show its convergence to an NE in n-player identical-interest MGs in which a single player controls the state transitions. Such games are of interest in communications, control, and economics applications. Our result together with the recent results in [42] establishes the FPP of two-player zero-sum MGs and n-player identical-interest MGs with a single controller (standing at two different ends of the MG spectrum from fully competitive to fully cooperative). © 2022 Owner/Author.Item Open Access Zero-sum Markov games with impulse controls(Society for Industrial and Applied Mathematics, 2020) Basu, Arnab; Stettner, L.In this paper we consider a zero-sum Markov stopping game on a general state space with impulse strategies and infinite time horizon discounted payoff where the state dynamics is a weak Feller--Markov process. One of the key contributions is our analysis of this problem based on “shifted” strategies, thereby proving that the original game can be practically restricted to a sequence of Dynkin's stopping games without affecting the optimalty of the saddle-point equilibria and hence completely solving some open problems in the existing literature. Under two quite general (weak) assumptions, we show the existence of the value of the game and the form of saddle-point (optimal) equilibria in the set of shifted strategies. Moreover, our methodology is different from the previous techniques used in the existing literature and is based on purely probabilistic arguments. In the process, we establish an interesting property of the underlying Feller--Markov process and the impossibility of infinite number of impulses in finite time under saddle-point strategies which is crucial for the verification result of the corresponding Isaacs--Bellman equations.