CMPUT 656 - Matrix and Convex Methods in Machine Learning

This course will cover the fundamental concepts and key algorithms of matrix analysis and convex optimization as they apply to machine learning problems. A key subtopic will be representation learning based on sparse regularization and their supporting nonsmooth optimization methods. The lectures will be organized around important machine learning challenges, but will also focus on presenting globally optimal solution techniques from matrix theory and mathematical programming. An auxiliary goal of the course is to provide a familiarity with the key algorithmic ideas behind matrix analysis and convex optimization, which are of growing importance in machine learning.

Although there are no formal prerequisites for this course, I assume some prior familiarity with machine learning problems and techniques. Although much of the material is necessarily based on matrix analysis and multivariable optimization, I fully expect to fill in this background as the course proceeds. Nevertheless, a willingness to re-acquire basic linear algebra and calculus concepts, while acquiring some knowledge of convex analysis, particularly when it becomes obvious that these be extremely useful, will be most helpful.

Course work:

2 Assignments			25% each
Project			50%

References:

There will be no official text for this course. Most of the material will be covered in lecture notes and supplementary excerpts that will be provided.

Supplementary References On-line:

The Matrix Reference Manual, by Mike Brookes
www.ee.ic.ac.uk/hp/staff/dmb/matrix/intro.html
The Matrix Cookbook, by Kaare Brandt Petersen and Michael Syskind Pedersen
matrixcookbook.com
Convex Optimization, by Stephen Boyd and Lieven Vandenberghe
www.stanford.edu/~boyd/cvxbook.html
Lecture Notes on Convex Programming, by Arkadi Nemirovski
www2.isye.gatech.edu/~nemirovs

(More will be added as the course proceeds.)

Course outline