7.1.2 Action Frequencies

When an opponent's action is used to adjust their weight array, it is important to take into account what type of opponent it is. As a first approximation we can assume all opponents play the same ( e.g. raise with the top 15% of hands, and so on). In this case, all that matters when adjusting the weight array is the action taken, and not the type of player. This is called generic opponent modeling because the re-weighting system is identical for each player. However, initial tests against human opponents revealed that this was an incorrect assumption. A loose and aggressive opponent might bet with almost anything and therefore a bet should not be taken too seriously. On the other hand, a tight and passive opponent will usually only bet with a very good hand.

To account for this we introduce specific opponent modeling. Statistics for each opponent are tabulated and used to calculate betting frequencies, and these frequencies adjust how the re-weighting system works. For example, if we observe that a certain opponent bets 80% of the time when acting first on the flop, then we will take this into account when adjusting the weight array for a bet observed by that opponent in that same situation.

This introduces a new problem. Statistics can be retained for a large number of categories. Consider, for example, that we index each frequency by the number of active opponents (1, 2, 3+; 3+ is the `3 or more' category), total raises this round (0, 1, 2+), bets to call (0, 1, 2+), and game round (pre-flop, flop, turn or river). The `n+' classification is useful for putting together similar cases. In this example, each situation fits into one of 3 * 3 * 3 * 4 = 108 categories. It would take a large number of observed games before we had sufficient data for some categories. In fact, this categorization could easily be made more complex by also taking into account other important variables such as table position, remaining callers, and previous action taken (to catch advanced strategies like check-raising).

The finer the granularity of the data collected, the harder it is to get meaningful information (more observations are required). There are two ways to make it more manageable. The first way is to distinguish many different categories for the data collection phase (a fine level of granularity) and then combine classes that have similar contexts in the data usage phase. The other, less sophisticated, approach is to simply have few different categories for the data collection phase. This allows the frequency calculations of the data usage phase to be quick and simple. This is what we have done.

Data is categorized into 12 categories, indexed by (0, 1, 2+) bets to call and the 4 different game rounds. This is a simple first approximation but is sufficient to determine if this level of opponent modeling can be beneficial. It also has the advantage of enabling the program to learn `quickly', although less accurately. It is easy to adjust the definition of the context for the purpose of gathering statistics. The learning system (the re-weighting system that uses this information) is only interested in the frequency of an observed action and not how that figure was calculated.

For each opponent p, we determine the category c of the observed action (the context - amount to call and game round) and record the action a taken (fold, check/call, bet/raise) by incrementing T_p[c][a]. When we need to know the frequency that a player has taken a certain action in a certain situation, f_p[c][a], we simply compute the ratio of T_p[c][a] versus S_p[c] (the total number of actions recorded for that context):

$$S_{p}[c] = \sum_{i=0}^{2}T_{p}[c][i],\textnormal{ and}$$ (7.1)

$$f_{p}[c][a] = \frac{T_{p}[c][a]} {S_{p}[c]}.$$ (7.2)