Martin Zinkevich, Michael Johanson, Michael Bowling, and Carmelo Piccione. Regret Minimization in Games with Incomplete Information. In Advances in Neural Information Processing Systems 20 (NIPS), pp. 905–912, 2008. A longer version is available as a University of Alberta Technical Report, TR07-14.
Extensive games are a powerful model of multiagent decision-making scenarios with incomplete information. Finding a Nash equilibrium for very large instances of these games has received a great deal of recent attention. In this paper, we describe a new technique for solving large games based on regret minimization. In particular, we introduce the notion of counterfactual regret, which exploits the degree of incomplete information in an extensive game. We show how minimizing counterfactual regret minimizes overall regret, and therefore in self-play can be used to compute a Nash equilibrium. We demonstrate this technique in the domain of poker, showing we can solve abstractions of limit Texas Hold'em with as many as 10^12 states, two orders of magnitude larger than previous methods.
@InProceedings(07nips-regretpoker-w-tr, Title = "Regret Minimization in Games with Incomplete Information", Author = "Martin Zinkevich and Michael Johanson and Michael Bowling and Carmelo Piccione", Booktitle = "Advances in Neural Information Processing Systems 20 (NIPS)", Year = "2008", Pages = "905--912", Note = "A longer version is available as a University of Alberta Technical Report, TR07-14", AcceptRate = "22\%", AcceptNumbers = "217 of 975" )