The use of stochastic simulation to solve problems where exhaustive methods take too long is not new to the Artificial Intelligence community. Specifically, stochastic methods have been developed for performing inference in belief networks and for decision-making in imperfect information or non-deterministic games.
There are two important advantages of stochastic simulation algorithms. They have an ``anytime'' property in the sense that they can be stopped at any time and will give the best answer available. Given more time, their estimates will improve. This ``anytime'' property is especially appropriate for real-time domains like poker. Also, simulation algorithms are trivial to perform in parallel.
In this chapter, simulation algorithms used in belief networks and games will be reviewed. The algorithms discussed perform selective sampling on the search space by using available information about the state of the world to bias the sample selection. The aim of selective sampling methods is to converge faster than approaches using uniform sampling.