Computer Go Test Collection

The original Computer Go Test Collection Version 1.0 was developed by Martin Müller in 1995. The collection was described in an appendix of his PhD thesis. The current collection is based on the specialized collections subset of the CGTC 1.0. It has been greatly improved in content and restructured to take advantage of current regression testing tools based on the Go Text Protocol (GTP). Test cases are now split into individual files. Testing uses a .tst script containing GTP commands.

CGTC contains realistic full-board test problems taken from computer Go games. Some problems have been edited by hand to make them more clear. A problem typically consists of a test position in sgf format and one or more good answers specified in a .tst file using the GTP command reg_genmove.

Example (from life_death.tst): 

  loadsgf ./sgf/life_death/life_death.3.sgf 1
  60 reg_genmove white
  #? [E18|K18]
This sequence of GTP commands loads an SGF file life_death.3.sgf, goes to the position before move 1, and calls the command reg_genmove white to generate a move for white. The number 60 is used to identify the test within the .tst file in case of a failure. The result of reg_genmove returned by the engine is a move to play. This move is compared with the correct answer(s) in the line below. In this case there are two possible correct answers, E18 or K18. If the program returns one of these two moves, it passes the test. If the program returns any other move, it fails the test.

Running the Test Collection

The test collection can easily be run using the regression test tool gogui-regress from GoGui.
gogui-regress additionally uses the GTP extension command cputime if supported by the Go program.
Here is how to run it (using gnugo --mode gtp as an example for the Go program):

  1. Download the GoGui package from http://gogui.sf.net and unpack it. The regression test tool is in lib/gogui-regress.jar
  2. Change into the directory of the test collection.
  3. Run: java -jar gogui-regress.jar "gnugo --mode gtp" *.tst
  4. The results will be formatted as HTML files. Open index.html for the summary.

Go engines that do not support the loadsgf command can use the tool gogui-adapter from GoGui, which translates loadsgf commands into a sequence of play commands.

Here is the example output for GNU Go 3.4.

Download

Current release: Older releases:

Feedback

We welcome feedback on the test set: modifications and additions to the test cases, alternative good (or bad) moves, and comments about which types of tests are most useful.

Martin Müller, Email (in domain cs.ualberta.ca): mmueller
Markus Enzenberger, Email (in domain cs.ualberta.ca): emarkus