Fictitious play in zero-sum stochastic games

Simple item page
buir.contributor.authorSayin, Muhammed O.
buir.contributor.orcidSayin, Muhammed O. | 0000-0003-2661-0918
dc.citation.epage2114en_US
dc.citation.issueNumber4en_US
dc.citation.spage2095en_US
dc.citation.volumeNumber60en_US
dc.contributor.authorSayin, Muhammed O.
dc.contributor.authorParise, Francesca
dc.contributor.authorOzdaglar, Asuman
dc.date.accessioned2023-02-23T08:56:52Z
dc.date.available2023-02-23T08:56:52Z
dc.date.issued2022
dc.departmentDepartment of Electrical and Electronics Engineeringen_US
dc.description.abstractWe present a novel variant of fictitious play dynamics combining classical fictitiousplay with Q-learning for stochastic games and analyze its convergence properties in two-player zero-sum stochastic games. Our dynamics involves players forming beliefs on the opponent strategyand their own continuation payoff (Q-function), and playing a greedy best response by using theestimated continuation payoffs. Players update their beliefs from observations of opponent actions.A key property of the learning dynamics is that update of the beliefs onQ-functions occurs at aslower timescale than update of the beliefs on strategies. We show that in both the model-based andmodel-free cases (without knowledge of player payoff functions and state transition probabilities),the beliefs on strategies converge to a stationary mixed Nash equilibrium of the zero-sum stochasticgame.en_US
dc.identifier.doi10.1137/21M1426675en_US
dc.identifier.eissn1095-7138
dc.identifier.issn0363-0129
dc.identifier.urihttp://hdl.handle.net/11693/111622
dc.language.isoEnglishen_US
dc.publisherSociety for Industrial and Applied Mathematicsen_US
dc.relation.isversionofhttps://www.doi.org/10.1137/21M1426675en_US
dc.source.titleSIAM Journal on Control and Optimizationen_US
dc.subjectStochastic gamesen_US
dc.subjectfictitious playen_US
dc.subjectQ-learningen_US
dc.subjecttwo-timescale learningen_US
dc.titleFictitious play in zero-sum stochastic gamesen_US
dc.typeArticleen_US

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Fictitious_play_in_zero-sum_stochastic_games.pdf
Size:
475.52 KB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.69 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Department of Electrical and Electronics Engineering