Updated version is at GoogleDoc

CMPUT 466/551
Schedule, Slides, Readings
[Always a DRAFT!]


Below is the tentative schedule for CMPUT 466/551.
Column2 lists the topics covered, with links to the slides that will be used.
    The symbol means the material is basically ready to print.
Column3 points to some relevant readings.
    Simple numbers in [x] brackets refer to chapters of the (HTF) textbook (2nd edition).

WARNING: This material is still being tweaked!

#0 (1)

Administration
Introduction: What is ML? 		[1]
Auxiliary
Foundations
#1 (2)
 Probability 101
  • Foundations: Bayes Rule, Independence, Dutchbook, Moments
  • Estimation: MLE, Bayesian + Gaussian
    		• [2]
    Auxiliary
    Learning Linear Models
    #2a (1.5) Learning Linear Regression Models (Revised 18/Sept/09) [3, 7]
    Auxiliary (Java)
    #2b (1)
    		 General Issues for Predictors
    (Evaluating Predictors, Model Selection: Bias-Variance, Regularization, Bayesian)     		[2.8, 2.9, 3.4]
    Auxiliary
    #2c (1) Linear Classifiers (Revised 25/Sept/09)
    (Perceptron, MeanSquaredError, Logistic Regression, Linear Discriminant Analysis, FisherLinearDiscriminant, [Newton-Raphson])     		[4]
    Auxiliary
    #2d (1.5) (Linear) Support Vector Machines (Revised 10/Oct/09)
    (+ Foundations: Duality, Legrange, Quadratic Programming, ...)     		[4.5.2, 12]
    Auxiliary
    #2e (0)
    		(not covered, 2009) Framework: Supervised Learning, Issues
    (not covered, 2008, 2009) Hypothesis Testing for Evaluating Predictors     		[2]
    Auxiliary
    Learning Non-Linear Models
    #3a (4)
    [BP]     		Kernel Foundations (Updated 13/Oct/09) [5.8], [12] (esp [12.3])
    Auxiliary
    Learning Kernel Classifiers
    (Google Book)
    #3b (1)
    [BP]     		SVM (Kernel) [6]
    Auxiliary
    #3c (1)
    [BP]     		Gaussian Processes (Kernel) Gaussian Processes for Machine Learning
    Auxiliary
    #3d (2) Artificial Neural Nets
    (BackPropagation, Repr'n, Line Search, Conjugate Gradient, Topics)     		[11]
    Auxiliary
    #3e (2)
    		 Decision Trees
    (Entropy, Pruning, Topics)     		[9.2]
    Auxiliary
    Ensemble Methods
    #4 (1)
    		 Ensemble Methods
    (Bagging, Boosting, ...)     		[8.7, 10, 16]
    Auxiliary
    Computational Learning Theory
    #5 (1)
  • PAC Learning
    Framework, Finite Realizable Case;
    (Not covered, 2009): Unrealizable, VCdimension, Variable size, Data-dependent
  • (Not covered, 2009) Other topics: Different Protocols, Occam Alg, Compression, Mistake Bound
    		• [7.1.5]
    Auxiliary
    Graphical Models
    #6a (3)
    		Directed Models (Belief Nets)
  • Intro to Belief Nets
  • Parameter Estimation
  • (not covered, 2008) Dynamic Bayesian Net (HMM, ...)
  • Learning Structure
    		• [8]
    Auxiliary
    #6b(0)

    		 (not covered, 2008) Undirected Models (Markov Random Fields) Auxiliary
    Unsupervised Learning
    #7 (2)
    [BP]
  • Principle Components Analysis
  • Independent Component Analysis, and friends
  • (Not covered, 2008) Other: (K-means clustering, Hierarchical clustering, ...)
    		• [12]
    Auxiliary
    Reinforcement Learning
    #8 (0)
    (Not covered 2009)     		Reinforcement Learning: An Introduction (download)
    [Chapter 1], [Chapter 2 - 2.2], [Chapter 3 (excluding 3.4, 3.5, 3.9)], [Chapter 6 (excluding 6.6, 6.7, 6.8)], [Chapter 10]     		[-]
    Auxiliary
    We gratefully acknowledge the assistance of Nilanjan Ray, Ron Parr, Carlos Guestrin, Dan Lizotte, Avrim Blum, Thomas Dietterich, Daphne Koller, Tom Mitchell, Andrew Moore (especially his tutorials), Stuart Russell, Bart Selman, Botta Marco, Esposito Roberto, and David Stork, for allowing us to use some of their material as part of my lecture notes, assignments, etc.