Topics

Various
Texts 
Articles
/ Slides / Course /Book

Other
(Web)

Introduction: What is ML?

[1]
[A:1], [DHS:1], [M:1], [HTF:1,2], [RN:18.1] 


Probability
101 
[HTF:1,2]
[B: 2; AppendixB], [A:1],
[DHS:1], [M:1], [RN:18.1] Book:
Grimstead/Snell:
"Introduction to Probability"
For info on Statistics, check out
Wackerly, D et al.,
Mathematical Statistics with Applications, 7th ed 
a good resource wrt probability theory, probability distributions, maximum likelihood estimation
Online text:
Concepts and Applications of Inferential Statistics

Slides:
(from A. Moore)
STAT 221 (UofA: Kouritz)
PDF:
Cribsheet

Web: Conditonal Probably and Bayes' Rule
Relationships between
(Clickable) Inventory of Probability Distributions
Univariate Dist'n Rel'ns
Conjugate Priors
Conjugate #2

Learning Linear Regression Models 
[3,7]
[DHS: 3] [A:?]


Applet 
General Issues for Predictors

[3.2, 3.4]

Domingos, AAAI 2000
Domingos, ICML 2000


Linear Classifiers

[4]
[A: 11.2, 5.8, 10.1, 10.3, 105.10.7; 4.1, 4.2, 4.5; 5.1,
5.2, 5.45.6], [HTF:5], [M:4], [RN:1919.5]

Slides:
Linear Classifiers (Moore)
DHS Ch5 Slides
Chapter (from T Mitchell)

Applet 
Framework: Supervised Learning, Issues
Hypothesis
Testing for Evaluating Predictors

[A: 14],[DHS:9], [M:5]

DHS Ch9 Slides (ppt)

Web: Simple Interactive Statistical Analysis
A New View of Statistics
Discussion of ROC curves
Statistical
Engineering website

Artificial Neural
Nets
(BackPropagation, Repr'n, Conjugate Gradient,
Topics)

[11] [B: 5],
[A: 11], [M:4], [DHS:6], [RN:20.5]

Slides:
DHS Ch6 Sildes
Article:
Intro to Conjugate Gradient w/out ... Pain

Web:
Neural Nets FAQ

Decision Trees
(Entropy, Pruning, Topics)

[9.2]
[B: 14.4]
[A: 9.19.2], [DHS:9], [M:3], [RN:18.3]

Book: C4.5:
Programs for Machine Learning

Web:
AIxploratorium

Kernel Trick 
[6]
?Gaussian Process?
[A: ?]
[HTF ?4.5.2, 12.212.3], ...

Web:
Gaussian Process

Slides:
Dan Lizotte
Linear Algebra, etc: Matrix Cookbook
Cheat Sheet

Support Vector Machines
(Quadratic Programming, Kernel Trick, ...)

[HTF: 12]
[B: 6, 7, AppE],
[A: 10.9], [HTF 4.5.2, 12.212.3]

Slides:
SVM (R Meir)
Auxiliary notes
Books:
Vapnik:
"The nature of statistical learning theory"
[12 Xeroxed pages]
Kernel Methods for Pattern Analysis
Support Vector
Articles:
Intro to KernelBased Learning Algs

Web: Kernel Machines
Freeware:
SVM Light
Lagrange Multipliers:
SLIMY
GA
Tech Wikipedia

Computational Learning Theory

[7.1.5]
[A: 2.12.3], [M:7], [RN:18.5]

Book: An
Introduction to Computational Learning Theory
Ref: Probabilistic
inequalities
Factoid
WhoIsOccam

Course: Blum's
"Machine Learning Theory" 
Ensemble Methods
(Bagging, Boosting)

[B: 14.3],
[A: 15], [DHS: 9.5], [HTF:10]

Slides: Ensemble Learning
from
Ensemble
Methods in Machine Learning (Dietterich)
Notes from S Wang

Web:
Shapire, movie
AdaBoost Demo
Boosting.org
Boosting
Papers [UCSD]
Dietterich's Page
Papers on
Boosting/Bagging (Schapire)
Article:
"An Introduction to Boosting and Leveraging" (Meir/Raetsch),
On Bagging and Nonlinear Estimation (Friedman,Hall)

Directed Models (Belief Nets)

[8]
[A: 3.7]
[RN:20.2]
[2.11], [3.10], [M:?], [RN:?] 
Slides:
Cmput366Notes
on BeliefNets
GMM (Moore)
HMM (Moore)
Articles:
Heckerman:
A tutorial on learning with Bayesian networks
BN
 Classifiers
HMM (Rabiner)

Web: Belief Nets
Tutorials
Interactive Tutorial
Ten years of HMMs

Undirected Models
(Markov Random Fields)

[8.4]


MaxEnt (Berger)
References (Wang)

Unsupervised
Learning
 Kmeans clustering
 Principle Components Analysis
 Independent Component Analysis

[12], [A: 7], [DHS:10], [HTF:14] 

Web: Tutorial

Reinforcement Learning
Markov Decision Process, Known Model,
Adaptive Dynamic Programming,
Value/Policy Iteration
TDLearning
Value Approximation
Q Learning

[16], [M:13], [RN:20]

Book:
Reinforcement
Learning: An Introduction (download)
Slides: Barto
slides
ArticlesKaelbling's
survey article
Misc
pointers
POMDPs
for Dummies 
Web:Dietterich's
Reinforcement Learning
UMass Amherst
CMU
Reinforcement Learning Group 