The popularity of test collections such as the Bratko-Kopec test in computer chess [Kopec/Bratko 82] indicates the importance of a widely accepted set of test positions. Our Computer Go Test Collection is a first step to develop such a set for computer Go.
The Computer Go Test Collection is designed to test Go programs in all aspects of the game. Details of the collection are described in the Appendix of my Ph.D. thesis.
This is Version 1.0 of the Computer Go Test Collection. We encourage all Go programmers to try their program against the problems, publish their results, and contribute tests to future releases of the collection.
There are many ways to set up test positions:
Because the designer of a test suite has complete freedom on the set of positions included, this testing method is very powerful. With carefully selected test sets one can get independent measures of performance in different stages of a game, leading to a detailed profile of program behavior.
While setting up a test suite certainly takes a lot of work, it is easy to run it over and over again automatically.