1 / 1

Denoising Score Matching Loss

Diffusion ModelsScore MatchingLoss Functions

Medium

Compute the noise-weighted denoising score matching loss of a small linear score model for a batch of clean samples.

\tilde x = x_0 + \sigma \epsilon, \qquad s = \frac{\tilde x W + b}{\sigma}, \qquad \ell = \frac{1}{2}\, \sigma^2 \sum_{d=1}^{D} \left( s_d + \frac{\epsilon_d}{\sigma} \right)^2

Here x0 holds the clean samples $x_0$ with shape (B, D), noise holds the standard Gaussian noise $\epsilon$ with the same shape, and sigmas holds one noise level $\sigma$ per sample with shape (B,). The toy score model multiplies $\tilde x$ by weight $W$ (shape (D, D)), adds bias $b$ (shape (D,)) and divides by $\sigma$ to give the predicted score $s$ . The target score is $-\epsilon / \sigma$ , and the weighting $\lambda(\sigma) = \sigma^2$ multiplies half the squared error summed over the D features, which gives the per-sample loss $\ell$ . The batch loss is the mean of $\ell$ over the B samples.

Return a dict with exactly the keys "loss" and "per_sample_loss", where "loss" holds the batch loss as a Python float and "per_sample_loss" holds a PyTorch tensor of shape (B,) with the dtype and device of x0.

Examples

Input: x0 = [[1.0,-0.5],[0.2,0.8]], noise = [[0.4,-1.2],[1.0,0.5]], sigmas = [0.5,2.0], weight = [[0.5,0.1],[-0.2,0.3]], bias = [0.1,-0.1]

Output: {"loss":2.188425,"per_sample_loss":[2.01125,2.3656]}

Explanation: The first sample is noised to (1.2, -1.1) with sigma 0.5, its target score is (-0.8, 2.4) and the model predicts (1.84, -0.62), giving a weighted loss of 2.0112. The batch loss is the mean of 2.0112 and 2.3656.

Input: x0 = [[0.3,-0.7,1.5]], noise = [[1.0,-2.0,0.5]], sigmas = [10.0], weight = [[0.0,0.0,0.0],[0.0,0.0,0.0],[0.0,0.0,0.0]], bias = [0.0,0.0,0.0]

Output: {"loss":2.625,"per_sample_loss":[2.625]}

Input: x0 = [[0.5],[-1.0],[2.0]], noise = [[1.0],[-0.5],[0.0]], sigmas = [0.01,1.0,25.0], weight = [[-0.9]], bias = [0.05]

Output: {"loss":0.70363,"per_sample_loss":[0.174641,0.405,1.53125]}

Hint 1

sigmas[:, None] gives a (B, 1) column that broadcasts over the features.

Hint 2

x @ weight + bias applies a linear model to every row at once.

Hint 3

.pow(2).sum(dim=1) gives one squared error per row, and .item() turns a one-element tensor into a Python float.

Requirements

Use PyTorch tensor operations.
Use each sample's own sigma for its noisy input, its score, its target and its weight $\sigma^2$ .
Multiply the row vector $\tilde x$ by weight on the right, so the output is $\tilde x W$ and never $\tilde x W^\top$ .
Leave all input tensors unchanged.

Constraints

x0 and noise are floating-point tensors of shape (B, D) with B between 1 and 4 and D between 1 and 5.
sigmas is a tensor of shape (B,) with every value strictly positive, between 0.01 and 50.
weight has shape (D, D) and bias has shape (D,).
All tensors share one dtype, float32 or float64, and all values are finite.

Try Similar Problems

Annealed Langevin DynamicsMedium Conditional Flow Matching LossMedium Gaussian Mixture ScoreMedium Min-SNR Loss WeightingMedium Noise Prediction LossEasy Cosine Noise ScheduleMedium Rescale to Zero Terminal SNRMedium Time-Conditioned Residual BlockMedium Higher-Order Derivatives & Loss CurvatureMedium Loss FunctionsMedium Bradley-Terry Reward Model LossMedium REINFORCE with BaselineMedium

Denoising Score Matching Loss

Diffusion ModelsScore MatchingLoss Functions

Medium

Compute the noise-weighted denoising score matching loss of a small linear score model for a batch of clean samples.

\tilde x = x_0 + \sigma \epsilon, \qquad s = \frac{\tilde x W + b}{\sigma}, \qquad \ell = \frac{1}{2}\, \sigma^2 \sum_{d=1}^{D} \left( s_d + \frac{\epsilon_d}{\sigma} \right)^2

Examples

Input: x0 = [[1.0,-0.5],[0.2,0.8]], noise = [[0.4,-1.2],[1.0,0.5]], sigmas = [0.5,2.0], weight = [[0.5,0.1],[-0.2,0.3]], bias = [0.1,-0.1]

Output: {"loss":2.188425,"per_sample_loss":[2.01125,2.3656]}

Input: x0 = [[0.3,-0.7,1.5]], noise = [[1.0,-2.0,0.5]], sigmas = [10.0], weight = [[0.0,0.0,0.0],[0.0,0.0,0.0],[0.0,0.0,0.0]], bias = [0.0,0.0,0.0]

Output: {"loss":2.625,"per_sample_loss":[2.625]}

Input: x0 = [[0.5],[-1.0],[2.0]], noise = [[1.0],[-0.5],[0.0]], sigmas = [0.01,1.0,25.0], weight = [[-0.9]], bias = [0.05]

Output: {"loss":0.70363,"per_sample_loss":[0.174641,0.405,1.53125]}

Hint 1

sigmas[:, None] gives a (B, 1) column that broadcasts over the features.

Hint 2

x @ weight + bias applies a linear model to every row at once.

Hint 3

.pow(2).sum(dim=1) gives one squared error per row, and .item() turns a one-element tensor into a Python float.

Requirements

Use PyTorch tensor operations.
Use each sample's own sigma for its noisy input, its score, its target and its weight $\sigma^2$ .
Multiply the row vector $\tilde x$ by weight on the right, so the output is $\tilde x W$ and never $\tilde x W^\top$ .
Leave all input tensors unchanged.

Constraints

x0 and noise are floating-point tensors of shape (B, D) with B between 1 and 4 and D between 1 and 5.
sigmas is a tensor of shape (B,) with every value strictly positive, between 0.01 and 50.
weight has shape (D, D) and bias has shape (D,).
All tensors share one dtype, float32 or float64, and all values are finite.

Try Similar Problems

x0 =

Accepts: array

noise =

Accepts: array

sigmas =

Accepts: array

weight =

Accepts: array

bias =

Accepts: array