Matrix Methods in Machine Learning

Undergraduate course on matrix methods and their applications in machine learning.

Course Information

Instructor: Ulugbek S. Kamilov
Institution: University of Wisconsin–Madison
Term: Spring 2026

Course Description

This course introduces the linear-algebraic foundations of machine learning, with an emphasis on real-world applications of matrix methods in classification, clustering, denoising, and data analysis. Mathematical topics include systems of linear equations, least-squares regression, regularization, the singular value decomposition, and iterative algorithms. Machine learning topics include the lasso, support vector machines, kernel methods, clustering, dictionary learning, neural networks, and deep learning.

Lecture Topics

  1. Vectors and matrices in ML
  2. Matrix and vector operations
  3. Matrix and vector norms
  4. Linear independence
  5. Systems of linear equations
  6. Solutions to linear systems
  7. Orthogonality and subspaces
  8. Introduction to least squares
  9. Gradient descent for least squares
  10. Orthogonality, projections, and least squares
  11. Vector calculus
  12. Introduction to SVD
  13. Least-squares solutions via SVD
  14. Low-rank approximation and power method
  15. PCA and dimensionality reduction
  16. Linear autoencoders and ridge regression
  17. Review: SVD as a unifying tool in ML
  18. Beyond least squares
  19. Gradient descent
  20. Stochastic gradient descent (SGD)
  21. Subgradient and proximal methods
  22. Kernel Methods: From Features to Kernels
  23. Kernel Methods: Duality and RKHS
  24. Neural Networks