Overview


How can a machine learn from data? How should we construct appropriate models? How do we design algorithms for inference? What are the fundamental design principles of data-driven learning and decision-making?

This course will help you answer these fundamental questions in machine learning from a probabilistic perspective. You will read and discuss research papers in probabilistic machine learning, including but not limited to research topics about


  • Probabilistic models: Bayesian neural networks, variational autoencoder, normalizing flows, score-based generative models, diffusion models, Gaussian processes.
  • Probabilistic inference: Markov chain Monte Carlo (MCMC), variational inference, Laplace approximation, dropout, ensemble.
  • Evaluations and applications: uncertainty estimation, calibration, distribution shift, out-of-distribution detection, adversarial attacks.

Schedule

Aug. 23
Lecture #1: Overview
  • • Logistics
  • • Introduction
Aug. 25
Lecture #2: Probability Fundamentals
  • • Probability review
  • • Bayesian regression
  • • Conjugate priors
Aug. 30
Lecture #3: Model Comparison
  • • Model evidence
  • • Occam’s razor
  • • Bayes factor
Sept. 1
Lecture #4: Decision Theory
  • • Posterior expected loss
  • • Maximum expected utility principle
  • • COVID19 example
Sept. 6
Lecture #5: Neural Networks
  • • Feed-forward network functions
  • • Network training
  • • Error backpropagation
Sept. 8
Paper Discussion #1
Sept. 13
Lecture #6: Variational Inference
Sept. 15
Paper Discussion #2
Sept. 20
Lecture #7: Markov Chain Monte Carlo I
Sept. 22
Paper Discussion #3
Sept. 27
Lecture #8: Markov Chain Monte Carlo II
Sept. 29
Paper Discussion #4
Oct. 4
Lecture #9: Variational Autoencoder
Oct. 6
Paper Discussion #5
Oct. 11
October Break
Oct. 13
Lecture #10: Energy-based Models
Oct. 18
Paper Discussion #6
Oct. 20
Lecture #11: Diffusion Models
Oct. 25
Paper Discussion #7
Oct. 27
Lecture #12: Gaussian Processes
Nov. 1
Paper Discussion #8
Nov. 3
Lecture #13: Uncertainty Estimation and Calibration
Nov. 8
Paper Discussion #9
Nov. 10
Lecture #14: Out-of-Distribution Detection
Nov. 15
Paper Discussion #10
Nov. 17
Lecture #15: Bayesian Optimization
Nov. 22
Paper Discussion #11
Nov. 24
Thanksgiving Break
Nov. 29
Lecture #16: Meta-learning
Dec. 1
Paper Discussion #12
Dec. 6
Project Presentations #1
Dec. 8
Project Presentations #2