Lectures

You can download the lectures here. We will try to upload lecture slides prior to their corresponding classes.

• Lecture 1: Introduction to diffusion models
  [slides] [video]
  

  • What are generative models?
  • Applications of generative models
  • A brief history of generative modeling, from VAEs to GANs to Diffusion.
  • What are diffusion models?

  Suggested Readings:

  • Diffusion models from scratch
  • smalldiffusion codebase
  • Elucidating the Design Space of Diffusion-Based Generative Models
  • Analyzing and Improving the Training Dynamics of Diffusion Models
• Lecture 2: Perspectives on diffusion
  [slides] [video]
  

  • Introduce different interpretations and perspectives on diffusion
  • Derivation of DDPM and DDIM, SDE reversal and probability flow ODE.
  • Why do diffusion models generalize?

  Suggested Readings:

  • What are diffusion models?
  • SDE interpretation blog post
  • Step-by-step diffusion: an elementary tutorial
• Lecture 3: Conditioning and guidance
  [slides] [video]
  

  • Conditional diffusion training and inference.
  • Classifier and classifier-free guidance.
  • Using diffusion models to solve inverse problems.

  Suggested Readings:

  • Guidance: a cheat code for diffusion models
  • Classifier-free guidance paper
  • Latent diffusion models
  • Guiding a diffusion model with a bad version of itself
• Lecture 4: Score distillation and advanced applications
  [slides] [video]
  

  • What is score distillation sampling (SDS)? Derivation through score matching and KL divergence
  • New architectures and one-step distillation: Recurrent Interface Networks (RIN), Consistency Models and MeanFlow.

  Suggested Readings:

  • Score Distillation via Reparametrized DDIM
  • Differentiable 3D representations
• Lecture 5: Diffusion Models for Robotics
  [slides] [video]
  

  • Diffusion Policy vs Action-Chunking Transformers
  • Adapting diffusion architectures to new applications.

  Suggested Readings:

  • DreamFusion: Text-to-3D using 2D Diffusion
  • ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation
• Lecture 6: Model generalization, summary and conclusion
  [slides] [video]
  

  • Why do diffusion models generalize?
  • Inductive biases of diffusion models: training and architecture.
  • How to train better and focus on noise levels that matter most.

  Suggested Readings:

  • Interpreting and improving diffusion models from an optimization perspective
  • Generalization in diffusion models arises from geometry-adaptive harmonic representations
  • Closed-Form Diffusion Models
  • Nuclear Norm Regularization for Deep Learning
  • An analytic theory of creativity in convolutional diffusion models