Online Limit After Ten Duplicate Matches

Although I title these results "preliminary", I do not believe that there will be enough time to actually run more experiments.

At present, the rankings are:
  1. Hyperborean07 Online 2 (University of Alberta)
  2. Hyperborean07 Online 1 (University of Alberta)
  3. GS3 (Carnegie Mellon University)
  4. IanBot (Ian Fellows, UCSD)
  5. Quick (Brad Quick)
  6. Gomel 2 (Igor Korschunov, Gomel State University)
  7. PokeMinn 1 (University of Minnesota)
  8. Sequel 1 (Sequel, Inria Futurs)
  9. Sequel 2 (Sequel, Inria Futurs)
  10. Le Renard (Bruno Lucas)
  11. Dumbo Online 2 (Xavier Bitot)
  12. PokeMinn 2 (University of Minnesota)
  13. Dumbo Online 1 (Xavier Bitot)
  14. Gomel 1 (Igor Korschunov, Gomel State University)
  15. UNCC (University of North Carolina, Charlotte)
  16. Monash-BPP (Monash University)
  17. Milano Online

In order to determine how significant even the top three are, I have to look at which bots were likely to be amongst the top 11. Since the bots between 11th and 14th were very close, this will take some time. Below are the detailed statistics.

All Bots

Hyperborean07LimitOnline2 Hyperborean07LimitOnline1 GS3Limit1 IanBotLimit1 QuickLimit1 GomelLimit2 PokeMinnLimit1 SequelLimit1 SequelLimit2 LeRenardLimit1 DumboLimitOnline2 PokeMinnLimit2 DumboLimitOnline1 GomelLimit1 UNCCLimit1 MonashBPPLimit1 MilanoLimitOnline1 Average
Hyperborean07LimitOnline2 -0.0368 ± 0.0083 -0.0272 ± 0.01 -0.0367 ± 0.0145 0.1378 ± 0.0072 0.1551 ± 0.0104 0.1715 ± 0.0186 0.1657 ± 0.0092 0.1779 ± 0.008 0.1697 ± 0.0139 0.259 ± 0.0113 0.2465 ± 0.0137 0.2797 ± 0.0096 0.2086 ± 0.01 0.5303 ± 0.0089 0.631 ± 0.008 0.6991 ± 0.0261 0.2332 ± 0.0063
Hyperborean07LimitOnline1 0.0368 ± 0.0083 0.0214 ± 0.0072 0.0274 ± 0.0071 0.1162 ± 0.0096 0.1083 ± 0.0113 0.1409 ± 0.0112 0.1534 ± 0.0094 0.1754 ± 0.0069 0.1316 ± 0.0139 0.2071 ± 0.0073 0.1357 ± 0.0132 0.1895 ± 0.0086 0.1103 ± 0.0139 0.4658 ± 0.0087 0.4321 ± 0.0213 0.4495 ± 0.0133 0.1813 ± 0.0043
GS3Limit1 0.0272 ± 0.01 -0.0214 ± 0.0072 0.0064 ± 0.0082 0.0729 ± 0.0069 0.1124 ± 0.0111 0.1499 ± 0.0127 0.1396 ± 0.0072 0.148 ± 0.012 0.1418 ± 0.0127 0.1987 ± 0.0082 0.154 ± 0.0112 0.1829 ± 0.0092 0.1065 ± 0.0104 0.467 ± 0.0091 0.4119 ± 0.0178 0.5089 ± 0.0129 0.1754 ± 0.0043
IanBotLimit1 0.0367 ± 0.0145 -0.0274 ± 0.0071 -0.0064 ± 0.0082 0.0991 ± 0.0078 0.085 ± 0.0123 0.1295 ± 0.0147 0.1307 ± 0.0084 0.14 ± 0.0119 0.1297 ± 0.0082 0.1567 ± 0.0085 0.1421 ± 0.0103 0.1206 ± 0.0096 0.0875 ± 0.0117 0.4716 ± 0.0112 0.4078 ± 0.0199 0.4217 ± 0.0137 0.1578 ± 0.004
QuickLimit1 -0.1378 ± 0.0072 -0.1162 ± 0.0096 -0.0729 ± 0.0069 -0.0991 ± 0.0078 0.0194 ± 0.0115 -0.0398 ± 0.0142 0.1249 ± 0.0098 0.1205 ± 0.0119 0.0145 ± 0.0103 0.1292 ± 0.0137 0.1343 ± 0.0126 0.2436 ± 0.0175 0.1494 ± 0.0108 0.2979 ± 0.0079 0.5643 ± 0.0165 0.6104 ± 0.0173 0.1214 ± 0.0056
GomelLimit2 -0.1551 ± 0.0104 -0.1083 ± 0.0113 -0.1124 ± 0.0111 -0.085 ± 0.0123 -0.0194 ± 0.0115 -0.1439 ± 0.0176 0.1352 ± 0.0142 0.1496 ± 0.013 0.1364 ± 0.0149 0.1229 ± 0.0123 0.0162 ± 0.0232 0.2165 ± 0.0161 0.2324 ± 0.0071 0.2748 ± 0.0106 0.8021 ± 0.0231 1.1736 ± 0.0435 0.1647 ± 0.0123
PokeMinnLimit1 -0.1715 ± 0.0186 -0.1409 ± 0.0112 -0.1499 ± 0.0127 -0.1295 ± 0.0147 0.0398 ± 0.0142 0.1439 ± 0.0176 -0.0331 ± 0.0138 -0.0217 ± 0.0162 0.1272 ± 0.009 -0.0152 ± 0.0185 -0.024 ± 0.0181 0.0748 ± 0.0154 0.2648 ± 0.0075 0.3729 ± 0.0118 0.6268 ± 0.0231 0.4058 ± 0.0331 0.0856 ± 0.0061
SequelLimit1 -0.1657 ± 0.0092 -0.1534 ± 0.0094 -0.1396 ± 0.0072 -0.1307 ± 0.0084 -0.1249 ± 0.0098 -0.1352 ± 0.0142 0.0331 ± 0.0138 0.0187 ± 0.0098 0.092 ± 0.015 -0.001 ± 0.0112 0.1304 ± 0.0118 -0.0519 ± 0.0109 -0.017 ± 0.0192 0.1668 ± 0.0101 0.5555 ± 0.0159 0.6484 ± 0.0116 0.0453 ± 0.0044
SequelLimit2 -0.1779 ± 0.008 -0.1754 ± 0.0069 -0.148 ± 0.012 -0.14 ± 0.0119 -0.1205 ± 0.0119 -0.1496 ± 0.013 0.0217 ± 0.0162 -0.0187 ± 0.0098 0.0739 ± 0.0123 0.0174 ± 0.0122 0.1248 ± 0.0164 -0.0477 ± 0.0082 -0.0037 ± 0.0154 0.1739 ± 0.0115 0.5832 ± 0.019 0.6021 ± 0.0116 0.0385 ± 0.0035
LeRenardLimit1 -0.1697 ± 0.0139 -0.1316 ± 0.0139 -0.1418 ± 0.0127 -0.1297 ± 0.0082 -0.0145 ± 0.0103 -0.1364 ± 0.0149 -0.1272 ± 0.009 -0.092 ± 0.015 -0.0739 ± 0.0123 0.0206 ± 0.0063 -0.0596 ± 0.0111 0.0083 ± 0.0117 -0.0959 ± 0.0159 0.2808 ± 0.0123 0.478 ± 0.0216 0.4439 ± 0.0284 0.0037 ± 0.004
DumboLimitOnline2 -0.259 ± 0.0113 -0.2071 ± 0.0073 -0.1987 ± 0.0082 -0.1567 ± 0.0085 -0.1292 ± 0.0137 -0.1229 ± 0.0123 0.0152 ± 0.0185 0.001 ± 0.0112 -0.0174 ± 0.0122 -0.0206 ± 0.0063 0.0708 ± 0.0094 0.1217 ± 0.012 0.1463 ± 0.0162 0.1963 ± 0.0195 0.7073 ± 0.015 0.8978 ± 0.0235 0.0653 ± 0.004
PokeMinnLimit2 -0.2465 ± 0.0137 -0.1357 ± 0.0132 -0.154 ± 0.0112 -0.1421 ± 0.0103 -0.1343 ± 0.0126 -0.0162 ± 0.0232 0.024 ± 0.0181 -0.1304 ± 0.0118 -0.1248 ± 0.0164 0.0596 ± 0.0111 -0.0708 ± 0.0094 -0.0486 ± 0.0127 0.1226 ± 0.0098 0.0955 ± 0.0155 0.7697 ± 0.0207 0.9709 ± 0.0257 0.0524 ± 0.0033
DumboLimitOnline1 -0.2797 ± 0.0096 -0.1895 ± 0.0086 -0.1829 ± 0.0092 -0.1206 ± 0.0096 -0.2436 ± 0.0175 -0.2165 ± 0.0161 -0.0748 ± 0.0154 0.0519 ± 0.0109 0.0477 ± 0.0082 -0.0083 ± 0.0117 -0.1217 ± 0.012 0.0486 ± 0.0127 0.2852 ± 0.0169 -0.0038 ± 0.0179 0.7697 ± 0.0114 1.104 ± 0.0347 0.0541 ± 0.006
GomelLimit1 -0.2086 ± 0.01 -0.1103 ± 0.0139 -0.1065 ± 0.0104 -0.0875 ± 0.0117 -0.1494 ± 0.0108 -0.2324 ± 0.0071 -0.2648 ± 0.0075 0.017 ± 0.0192 0.0037 ± 0.0154 0.0959 ± 0.0159 -0.1463 ± 0.0162 -0.1226 ± 0.0098 -0.2852 ± 0.0169 -0.0952 ± 0.0108 0.7789 ± 0.029 1.4425 ± 0.0217 0.0331 ± 0.0102
UNCCLimit1 -0.5303 ± 0.0089 -0.4658 ± 0.0087 -0.467 ± 0.0091 -0.4716 ± 0.0112 -0.2979 ± 0.0079 -0.2748 ± 0.0106 -0.3729 ± 0.0118 -0.1668 ± 0.0101 -0.1739 ± 0.0115 -0.2808 ± 0.0123 -0.1963 ± 0.0195 -0.0955 ± 0.0155 0.0038 ± 0.0179 0.0952 ± 0.0108 0.5527 ± 0.0211 1.1294 ± 0.0288 -0.1258 ± 0.0081
MonashBPPLimit1 -0.631 ± 0.008 -0.4321 ± 0.0213 -0.4119 ± 0.0178 -0.4078 ± 0.0199 -0.5643 ± 0.0165 -0.8021 ± 0.0231 -0.6268 ± 0.0231 -0.5555 ± 0.0159 -0.5832 ± 0.019 -0.478 ± 0.0216 -0.7073 ± 0.015 -0.7697 ± 0.0207 -0.7697 ± 0.0114 -0.7789 ± 0.029 -0.5527 ± 0.0211 0.7624 ± 0.0461 -0.5193 ± 0.0124
MilanoLimitOnline1 -0.6991 ± 0.0261 -0.4495 ± 0.0133 -0.5089 ± 0.0129 -0.4217 ± 0.0137 -0.6104 ± 0.0173 -1.1736 ± 0.0435 -0.4058 ± 0.0331 -0.6484 ± 0.0116 -0.6021 ± 0.0116 -0.4439 ± 0.0284 -0.8978 ± 0.0235 -0.9709 ± 0.0257 -1.104 ± 0.0347 -1.4425 ± 0.0217 -1.1294 ± 0.0288 -0.7624 ± 0.0461 -0.7669 ± 0.014

Note that the ordering of the bots is nearly identical to their total bankroll, with the exception of Gomel 2 and Dumbo 2, which were mid-range bots that did very well against much weaker bots.

Hyperborean07 Online 2 did very well against the bottom 13 bots: however, it lost money to all of the others in the top 4. Hyperborean07 Online 1 was more successful against the top bots, but did not exploit the lower ranked bots as effectively. Next, we proceed to the final 11.

The Top 11 Bots

Hyperborean07LimitOnline2 Hyperborean07LimitOnline1 GS3Limit1 IanBotLimit1 QuickLimit1 GomelLimit2 PokeMinnLimit1 SequelLimit1 SequelLimit2 LeRenardLimit1 DumboLimitOnline2 Average
Hyperborean07LimitOnline2 -0.0368 ± 0.0083 -0.0272 ± 0.01 -0.0367 ± 0.0145 0.1378 ± 0.0072 0.1551 ± 0.0104 0.1715 ± 0.0186 0.1657 ± 0.0092 0.1779 ± 0.008 0.1697 ± 0.0139 0.259 ± 0.0113 0.1136 ± 0.0063
Hyperborean07LimitOnline1 0.0368 ± 0.0083 0.0214 ± 0.0072 0.0274 ± 0.0071 0.1162 ± 0.0096 0.1083 ± 0.0113 0.1409 ± 0.0112 0.1534 ± 0.0094 0.1754 ± 0.0069 0.1316 ± 0.0139 0.2071 ± 0.0073 0.1119 ± 0.0045
GS3Limit1 0.0272 ± 0.01 -0.0214 ± 0.0072 0.0064 ± 0.0082 0.0729 ± 0.0069 0.1124 ± 0.0111 0.1499 ± 0.0127 0.1396 ± 0.0072 0.148 ± 0.012 0.1418 ± 0.0127 0.1987 ± 0.0082 0.0975 ± 0.0051
IanBotLimit1 0.0367 ± 0.0145 -0.0274 ± 0.0071 -0.0064 ± 0.0082 0.0991 ± 0.0078 0.085 ± 0.0123 0.1295 ± 0.0147 0.1307 ± 0.0084 0.14 ± 0.0119 0.1297 ± 0.0082 0.1567 ± 0.0085 0.0874 ± 0.0031
QuickLimit1 -0.1378 ± 0.0072 -0.1162 ± 0.0096 -0.0729 ± 0.0069 -0.0991 ± 0.0078 0.0194 ± 0.0115 -0.0398 ± 0.0142 0.1249 ± 0.0098 0.1205 ± 0.0119 0.0145 ± 0.0103 0.1292 ± 0.0137 -0.0057 ± 0.0053
GomelLimit2 -0.1551 ± 0.0104 -0.1083 ± 0.0113 -0.1124 ± 0.0111 -0.085 ± 0.0123 -0.0194 ± 0.0115 -0.1439 ± 0.0176 0.1352 ± 0.0142 0.1496 ± 0.013 0.1364 ± 0.0149 0.1229 ± 0.0123 -0.008 ± 0.0101
PokeMinnLimit1 -0.1715 ± 0.0186 -0.1409 ± 0.0112 -0.1499 ± 0.0127 -0.1295 ± 0.0147 0.0398 ± 0.0142 0.1439 ± 0.0176 -0.0331 ± 0.0138 -0.0217 ± 0.0162 0.1272 ± 0.009 -0.0152 ± 0.0185 -0.0351 ± 0.0047
SequelLimit1 -0.1657 ± 0.0092 -0.1534 ± 0.0094 -0.1396 ± 0.0072 -0.1307 ± 0.0084 -0.1249 ± 0.0098 -0.1352 ± 0.0142 0.0331 ± 0.0138 0.0187 ± 0.0098 0.092 ± 0.015 -0.001 ± 0.0112 -0.0707 ± 0.005
SequelLimit2 -0.1779 ± 0.008 -0.1754 ± 0.0069 -0.148 ± 0.012 -0.14 ± 0.0119 -0.1205 ± 0.0119 -0.1496 ± 0.013 0.0217 ± 0.0162 -0.0187 ± 0.0098 0.0739 ± 0.0123 0.0174 ± 0.0122 -0.0817 ± 0.0036
LeRenardLimit1 -0.1697 ± 0.0139 -0.1316 ± 0.0139 -0.1418 ± 0.0127 -0.1297 ± 0.0082 -0.0145 ± 0.0103 -0.1364 ± 0.0149 -0.1272 ± 0.009 -0.092 ± 0.015 -0.0739 ± 0.0123 0.0206 ± 0.0063 -0.0996 ± 0.0046
DumboLimitOnline2 -0.259 ± 0.0113 -0.2071 ± 0.0073 -0.1987 ± 0.0082 -0.1567 ± 0.0085 -0.1292 ± 0.0137 -0.1229 ± 0.0123 0.0152 ± 0.0185 0.001 ± 0.0112 -0.0174 ± 0.0122 -0.0206 ± 0.0063 -0.1095 ± 0.0045

Amongst the top 11, the gap between Hyperborean07 Online 2 and the other bots is dramatically reduced, to the point where there is less than 0.002 small bets per hand between Hyperborean07 Online 1 and Hyperborean07 Online 2. If you eliminate one more bot, then Online 2's performance amongst the top 4 drops it to second place. Looking at the top 9 bots pushes Hyperborean07 Online 1 distinctively above Hyperborean07 Online 2.

However, the more bots you add to 11, the more Hyperborean07 Online 2's performance increases. This brings up two points:

  1. The number of bots considered is critical. If you consider too many bots, the match will be decided almost completely by how well you do against the weakest bots. If you consider too few, then the competition turns into an equilibrium competition.
  2. The reason there was this changeover was due to the fact that Hyperborean07 Online 1 was an approximate equilibrium and Hyperborean07 Online 2 was an aggressive exploiter. It may very well have been the case that a mixture of the two would have dominated both bots.