1 / 1

Gaussian Mixture Score

Diffusion ModelsScore MatchingProbability

Medium

Compute the score, the gradient of the log density with respect to the point, of a mixture of isotropic Gaussians at every row of x.

\nabla_x \log p(x) = \sum_{k=1}^{K} r_k(x)\, \frac{\mu_k - x}{\sigma_k^2}, \qquad r_k(x) = \frac{\pi_k\, \mathcal N(x; \mu_k, \sigma_k^2 I)}{\sum_{j=1}^{K} \pi_j\, \mathcal N(x; \mu_j, \sigma_j^2 I)}

The mixture density is $p(x) = \sum_k \pi_k\, \mathcal N(x; \mu_k, \sigma_k^2 I)$ . Here x holds N points of dimension D, weights holds the mixture weights $\pi_k$ , means holds the component means $\mu_k$ as rows, and variances holds one variance $\sigma_k^2$ per component that applies to every coordinate. The responsibility $r_k(x)$ is the posterior probability that component k produced the point, and the log density of one component is

\log \mathcal N(x; \mu_k, \sigma_k^2 I) = -\frac{D}{2} \log(2\pi\sigma_k^2) - \frac{\lVert x - \mu_k \rVert^2}{2\sigma_k^2} .

Points can lie far from every mean, where each density underflows to zero in floating point.

Examples

Input: x = [[0.0],[1.0],[3.0]], weights = [0.5,0.5], means = [[-1.0],[1.0]], variances = [1.0,1.0]

Output: [[0.0],[-0.238406],[-2.004945]]

Explanation: At x = 0 the two components pull equally hard in opposite directions, so the score is 0. At x = 3 nearly all responsibility belongs to the component at 1, so the score is close to -(3 - 1) / 1 = -2.

Input: x = [[0.5,0.5],[-1.0,2.0]], weights = [0.8,0.2], means = [[0.0,0.0],[2.0,2.0]], variances = [0.5,2.0]

Output: [[-0.943351,-0.943351],[1.752823,-2.022588]]

Input: x = [[60.0,-40.0]], weights = [0.3,0.7], means = [[1.0,0.0],[0.0,1.0]], variances = [0.04,0.09]

Output: [[-666.666667,455.555556]]

Hint 1

x[:, None, :] - means[None, :, :] gives every pairwise difference at once, with shape (N, K, D).

Hint 2

Subtracting the row maximum of the log terms before np.exp keeps the softmax stable and leaves the normalized responsibilities unchanged.

Hint 3

A matrix product with @ between an (N, K) array of per-component coefficients and the (K, D) means sums over the components in one step.

Requirements

Use NumPy array operations.
Compute the responsibilities in log space with a numerically stable log-sum-exp, so the result stays finite for points far from every mean.
Keep the log weight and the full normalizing term with the factor D in each component's log density, since these terms do not cancel when the variances differ.
Return a float64 NumPy array of shape (N, D) with no NaN or infinite values.
Leave the inputs unchanged.

Constraints

x is a float64 NumPy array of shape (N, D) with N and D at least 1.
weights is a float64 array of shape (K,) with K at least 1, every value positive, and a sum of 1 up to rounding.
means is a float64 array of shape (K, D).
variances is a float64 array of shape (K,) with every value between 0.01 and 3.
Coordinates of x can be as large as a few thousand in absolute value. All values are finite.

Try Similar Problems

Reparameterized Gaussian SampleEasy Diagonal Gaussian KL DivergenceEasy Posterior Mean and VarianceMedium Denoising Score Matching LossMedium Annealed Langevin DynamicsMedium Cosine Noise ScheduleMedium Rescale to Zero Terminal SNRMedium Time-Conditioned Residual BlockMedium Spatial Self-Attention BlockMedium Prediction ParameterizationMedium Min-SNR Loss WeightingMedium DDPM Ancestral StepMedium

Gaussian Mixture Score

Diffusion ModelsScore MatchingProbability

Medium

Compute the score, the gradient of the log density with respect to the point, of a mixture of isotropic Gaussians at every row of x.

\nabla_x \log p(x) = \sum_{k=1}^{K} r_k(x)\, \frac{\mu_k - x}{\sigma_k^2}, \qquad r_k(x) = \frac{\pi_k\, \mathcal N(x; \mu_k, \sigma_k^2 I)}{\sum_{j=1}^{K} \pi_j\, \mathcal N(x; \mu_j, \sigma_j^2 I)}

\log \mathcal N(x; \mu_k, \sigma_k^2 I) = -\frac{D}{2} \log(2\pi\sigma_k^2) - \frac{\lVert x - \mu_k \rVert^2}{2\sigma_k^2} .

Points can lie far from every mean, where each density underflows to zero in floating point.

Examples

Input: x = [[0.0],[1.0],[3.0]], weights = [0.5,0.5], means = [[-1.0],[1.0]], variances = [1.0,1.0]

Output: [[0.0],[-0.238406],[-2.004945]]

Input: x = [[0.5,0.5],[-1.0,2.0]], weights = [0.8,0.2], means = [[0.0,0.0],[2.0,2.0]], variances = [0.5,2.0]

Output: [[-0.943351,-0.943351],[1.752823,-2.022588]]

Input: x = [[60.0,-40.0]], weights = [0.3,0.7], means = [[1.0,0.0],[0.0,1.0]], variances = [0.04,0.09]

Output: [[-666.666667,455.555556]]

Hint 1

x[:, None, :] - means[None, :, :] gives every pairwise difference at once, with shape (N, K, D).

Hint 2

Subtracting the row maximum of the log terms before np.exp keeps the softmax stable and leaves the normalized responsibilities unchanged.

Hint 3

A matrix product with @ between an (N, K) array of per-component coefficients and the (K, D) means sums over the components in one step.

Requirements

Use NumPy array operations.
Compute the responsibilities in log space with a numerically stable log-sum-exp, so the result stays finite for points far from every mean.
Keep the log weight and the full normalizing term with the factor D in each component's log density, since these terms do not cancel when the variances differ.
Return a float64 NumPy array of shape (N, D) with no NaN or infinite values.
Leave the inputs unchanged.

Constraints

x is a float64 NumPy array of shape (N, D) with N and D at least 1.
weights is a float64 array of shape (K,) with K at least 1, every value positive, and a sum of 1 up to rounding.
means is a float64 array of shape (K, D).
variances is a float64 array of shape (K,) with every value between 0.01 and 3.
Coordinates of x can be as large as a few thousand in absolute value. All values are finite.

Try Similar Problems

x =

Accepts: array

weights =

Accepts: array

means =

Accepts: array

variances =

Accepts: array