Computer Go Test Collection

The popularity of test collections such as the Bratko-Kopec test in computer chess [Kopec/Bratko 82] indicates the importance of a widely accepted set of test positions. Our Computer Go Test Collection is a first step to develop such a set for computer Go.

The Computer Go Test Collection is designed to test Go programs in all aspects of the game. Details of the collection are described in the Appendix of my Ph.D. thesis.

This is Version 1.0 of the Computer Go Test Collection. We encourage all Go programmers to try their program against the problems, publish their results, and contribute tests to future releases of the collection.

Designing Test Positions

There are many ways to set up test positions:

• Specialized problem sets for Life&Death, capturing stones, the endgame etc.
• Find-the-best-move problems: Positions where a single best move has been recommended by expert players.
• Computer games: Positions in which a Go program has gone wrong before.
• Anti-Computer traps: situations where the obvious-looking move is wrong (e.g. a worthless 'urgent' pattern move)

Because the designer of a test suite has complete freedom on the set of positions included, this testing method is very powerful. With carefully selected test sets one can get independent measures of performance in different stages of a game, leading to a detailed profile of program behavior.

While setting up a test suite certainly takes a lot of work, it is easy to run it over and over again automatically.

The collections

• General purpose collections
• [obsolete: Specialized collections by position type]
• Endgame problems by Berlekamp and others