Fall 2003

Department of Computing Science

University of Alberta

**Instructor**: Dale Schuurmans, Ath409, x2-4806, dale@cs.ualberta.ca

**Room**: CSC B43

**Time**: TR 11:00-12:30

**Office hours**: TR 12:30-1:15 (or by appointment)

In the past decade

This course will cover the fundamentals of graphical probability models, focusing on the key representations, algorithms and theories that have facilitated much recent progress in artificial intelligence research.

There are no formal prerequisites for this course---all that is required is a basic programming capability and a rudimentary knowledge of probability and statistics. It would be advantageous (but not essential) to have some prior exposure to optimization methods, algorithms and complexity, and a previous course on artificial intelligence.

**Textbook**:
*An Introduction to Probabilistic Graphical Models*,

by Michael Jordan

(Chapters from his preliminary draft will
be distributed by the instructor.)

**Supplementary text**:
*Bayesian Networks and Beyond: Probabilistic Models for
Reasoning and Learning*,
by Daphne Koller and Nir Friedman

(Chapters from their preliminary draft will be distributed by the instructor as needed.)

**Course work**:

2 Assignments | 25% each | Assignment 1 | Assignment 2 | Assignment 2a | (Data files: walkingby2.mat, walkingby1.mat, walkingby.mat, face.mat, facesnow.mat) | ||||||

Project | 50% | Project handout |

Lecture 1 | Introduction | Thur Sep 4 | ||

Part 1 | Representation | |||
---|---|---|---|---|

Lecture 2 | Joint distributions, random variables | Tues Sep 9 | ||

Lecture 3 | Graphical model representations | Thur Sep 11 | ||

Part 2 | Inference | |||

Lecture 4 | Basic inference algorithm | Tues Sep 16 | ||

Lecture 5 | Independence properties | Thur Sep 18 | ||

Lecture 6 | Efficient tree-based inference | Tues Sep 23 | Normalization property of junction trees | |

Part 3 | Famous examples | |||

Lecture 7 | Multivariate Gaussian | Tues Sep 30 | ||

Lecture 8 | Naive Bayes, mixtures, hierarchies | Thur Oct 2 | ||

Lecture 9 | Hidden Markov models | Tues Oct 7 | ||

Lecture 10 | Kalman filters | Thur Oct 9 | ||

Lecture 11 | Exponential family | Tues Oct 14 | ||

Lecture 12 | Stochatic context free grammars | Thur Oct 16 | ||

Part 4 | Estimation | |||

Lecture 13 | Types of estimation/learning problems | Tues Oct 21 | ||

Lecture 14 | Maximum likelihood | Thur Oct 23 | ||

Lecture 15 | Maximum conditional likelihood | Tues Oct 28 | ||

Lecture 16 | Bayesian estimation | Thur Oct 30 | ||

Lecture 17 | Expectation-maximization | Tues Nov 4 | ||

Lecture 18 | Factor and principle component analysis | Thur Nov 6 | ||

Lecture 19 | Estimating hidden Markov models | Thur Nov 13 | ||

Lecture 20 | Learning structure | Tues Nov 18 | ||

Part 5 | Approximation | |||

Lecture 21 | Sampling | Thur Nov 20 | ||

Lecture 22 | Variational, loopy approximation | Tues Nov 25 | ||

Part 6 | Decision making | |||

Lecture 23 | Decision theory | Thur Nov 27 | ||

Lecture 24 | Markov decision processes | Tues Dec 2 |

Thanks for a great term!

A special thanks to our wonderful guest lecturers, Pascal Poupart and Gal Elidan!