Curriculum Vitae

Last updated March 20, 2017: [PDF]



Last udpated March 25, 2013


AI Competitions


Significant Research Contributions

  1. Computational Game Theory and Poker
    • Algorithms for computing game theoretic solutions to extremely large extensive games.

      Over two years, increased the state of the art by four orders of magnitude (from 10^8 to 10^12 game states).

      • Range of Skill: AAAI, 2007. Solves games with 10^10 states.
      • Counterfactual Regret: NIPS, 2008. Solves games with 10^12 states and only requires memory and time linear in the number of information sets (typically the square root of the number of game states). This was the basis for our success in the man versus machine competition in 2008.
      • Monte Carlo CFR: NIPS, 2009. Extends our poker-specific optimizations to form a general family of extensive game solvers that can efficiently solve a wide rante of zero-sum imperfect information games.
    • Algorithms for building and exploiting opponent models.
      • Robust opponent modelling: NIPS, 2008; AISTATS, 2009.

        These papers involve a very general idea for trading off prior beliefs with worst-case analyses, and has already been applied to ideas outside of poker and games.

      • Machinery for practical Bayesian modelling in poker: UAI, 2005.
      • Modelling dynamic opponents: AAAI, 2007.
    • Unbiased estimates of agent performance from very small sample sizes

      This work has been key for our progress toward defeating top human poker players, and both breakthroughs below were critical parts of our man versus machine victory in 2008.

      • Unbiased, low variance estimates of skill (DIVAT): AAAI, 2006.
      • Unbiased, low variance off-policy estimates of skill: ICML, 2008.
      • Learning custom variance reducing estimators from data (MIVAT): IJCAI, 2009.
  2. Subjective Mapping
    • Building a map of an environment with (almost) no knowledge of the agent's sensors or effectors

      The amazing thing about this research is that any headway can be made at all in what is a very challenging problem. For a summary see the AAAI/NECTAR paper (2006).

      • Map building as dimensionality reduction with ARE (Action Respecting Embedding): ICML, 2005.
      • Planning in ARE maps: IJCAI, 2005.
      • Localizing in ARE maps: ISRR, 2005.
      • Scaling up ARE: ISAIM, 2008.
    • Other similar work:
      • Automatically Calibrating Sensor and Motion Models: AAAI, 2006.
      • The first online algorithm for discovery/learning of PSRs: NIPS, 2006.
      • Non-blind estimators for learning PSR models: ICML, 2006.
  3. Fundamental Reinforcement Learning
    • Incremental techniques for making data-efficient least-squares techniques computationally tractable: AAAI, 2006; NIPS, 2007.
    • Techniques for combining models with linear function approximation: AAMAS, 2008; UAI, 2008.
    • Dual analysis of RL with function approximation: NIPS, 2008; ADPRL, 2007.
    • Approximate planning in POMDPs using quadratic programming: AAAI, 2006.
  4. Older Research (Before 2004)
    • Cooperating and Competing Teams of Robots

      Between 1998 and 2003, I was a member (and often leader) of the CMUnited and CMDragons robot soccer teams. The team was world champions in the RoboCup Small-Size League in 1998 and won the American Open in 2003, as well as being consistently among the top teams throughout these years.

      These teams have made advancements in motion control and navigation (CIRA, 1999), object tracking and prediction (ICRA, 2002), adapting team strategy (IJCAI, 2003), coordination in impromptu teams (AAAI, 2005), as well as integration of these components (Advanced Robotics; ICRA, 2003; J of Sys & Control Eng., 2005). Many of these techniques have become the league standard and remain a central part of the current CMDragons team (world champions in 2006 and 2007) long after I left CMU.

    • Multiagent Learning and Planning

      I developed WoLF ("Win or Learn Fast"), a technique for reinforcement learning in multiagent (possibly adversarial) environments: AIJ, 2002. Developments include both theoretical work (ICML, 2001, NIPS, 2004) and practical work (IJCAI, 2001; IJCAI, 2003), including a demonstration of adversarial learning on real robots.

      This combined body of work has been cited over 600 times according to Google Scholar.

Academic Service