What is a diffusion model?
Introduction
Project introduction
What is a diffusion model?
Turning a photo into static is easy. You add a little random noise, then a little more, until nothing of the photo is left. A diffusion model learns to run this backwards. Starting from pure static, it removes a little noise at a time until a new photo appears, one that was never in the training data.
The idea goes back to Sohl-Dickstein and colleagues in 2015. In 2020 Ho, Jain and Abbeel made it practical with DDPM (denoising diffusion probabilistic models), which trains a network to guess the noise that was added at a random step and then samples by walking back through all 1000 steps. Image generators such as Stable Diffusion grew out of this.
Why rectified flow?
Walking back through 1000 steps means 1000 calls to the network for every sample. Rectified flow (Liu et al. 2023), closely related to flow matching (Lipman et al. 2023), trains on straight lines between noise and data and learns a velocity along them. Straight paths can be followed in a few big steps.
In this project you build both, first on a 2D spiral where you can see every point, then on small clothing images. Then you take away most of their steps and measure which one still works, and why.
From noise to an image
DDPM and rectified flowWhat you will build
01
Noise on purpose
Noise schedules and the one-jump formula that noises data to any step.
02
Learn to walk back
The noise-prediction loss, a time-aware network, and DDPM sampling.
03
Go straight
Rectified flow with a velocity target and an Euler sampler, then the few-step race.
Start with the noise schedule
Turn a list of noise levels into how much signal is left after each step.