Conclusions and future work

Using probability triples as Loki-2's betting strategy and as the reweighting factor in its opponent modeling module represents a significant improvement in Loki-2's play against previous versions of Loki in self-play experiments and against human opponents on IRC. Selective sampling simulations show impressive results in self-play experiments. Against human opponents on IRC, the best results were obtained when all three enhancements were used. In self-play experiments, the playing style of the computer players certainly matches the opponents' actions generated inside the simulations. Thus, the simulation-based betting strategy successfully exploits all the weaknesses in the computer opponents' play. In the more realistic environment on IRC, the less predictable approach of the simulation-based Loki-2 paid dividends by making it more difficult for regular opponents to form a correct model of Loki-2's play.

Developing Loki is an iterative process. The work concentrates on improving an aspect of the program until it becomes apparent that another aspect is the main performance bottleneck. That problem is then addressed until it is no longer the limiting factor, and new weaknesses in the program's play are revealed. Loki-1's deterministic betting strategy was its limiting factor. This bottleneck was overcome in two ways. Probability triples provide as a probabilistic representation of betting decisions to increase unpredictability. Simulations add dynamic functionality to static betting strategies. The PT-generation function also supports better use of the information available to the Opponent Modeler, and is more tolerant of the uncertainty in the opponents' actions. However, the opponent modeling still needs to be refined. In fact, it seems that further performance gains will depend on perfecting the opponent modeling module together with improvements to the simulation-based betting strategy.

This thesis presents the first steps in using a simulation-based betting strategy and improving the reweighting process in the Opponent Modeler. These are the initial steps and there are still many to take. Some avenues to explore in Loki-2's future development are:

1.

The opponent modeling information can be used to improve the simulations. Currently, the opponent modeling data is used to select the most likely opponents' hands; however, it can also be used to simulate the most likely opponents' actions.

2.

Simulations can also improve the opponent modeling. For example, after doing a simulation, the expected reaction for each opponent can be recorded. If their actions frequently differ from what is predicted, then Loki-2 can adjust its opponent model.

3.

Loki-2 can easily collect lots of data about the opponent while playing. The problem is filtering and utilizing this data. If these problems are not solved, Loki-2's opponent modeling will be too slow to react or its betting strategy will base its decisions on irrelevant information.

4.

Other metrics that may be better predictors of an opponent's style and future behavior have to be considered. For example, measuring the amount of money that a player invests per game may be a good predictor of loose/tight play.

5.

Using showdown information to re-play a hand and obtain clues about how an opponent perceived each decision during the hand may help to adaptively measure important characteristics like aggressiveness, bluffing frequency, predictability, affinity for draws and so forth.

6.

The employment of learning algorithms in Loki-2's simulation-based strategy and in its Opponent Modeler may help to make inferences based on limited data.

7.

Loki-2's preflop behavior can be improved by using a preflop PT-based betting strategy.

As experimental results point out, Loki-2 wins more money (plays better) than last-year's Loki. However, does the program play world-class level poker? It is not there yet, but many improvements are being made to its performance and there are still lots of ideas to try.