Problems
Loading...
Step 4 of 6
KL Divergence
Easy

Compute the Kullback-Leibler divergence between a diagonal Gaussian posterior and the standard-normal prior used by a variational autoencoder.

DKL(q(z∣x) ∥ N(0,I))=−12B∑i=1B∑j=1L(1+log⁡σij2−μij2−exp⁡(log⁡σij2))D_{\mathrm{KL}}\bigl(q(z\mid x)\,\|\,\mathcal{N}(0,I)\bigr) = -\frac{1}{2B}\sum_{i=1}^{B}\sum_{j=1}^{L} \left(1 + \log \sigma_{ij}^2 - \mu_{ij}^2 - \exp(\log \sigma_{ij}^2)\right)

Here, BB is batch size, LL is latent width, μij\mu_{ij} is a posterior mean, and log⁡σij2\log \sigma_{ij}^2 is its log-variance. Sum over latent coordinates, then average over the batch. Return the divergence as a Python float.

Loading visualization...

Examples

Input: mu = [[0,0]], log_var = [[0,0]]

Output: 0

Explanation: A zero mean and zero log-variance describe the standard normal, so the divergence is zero.

Input: mu = [[1,2]], log_var = [[0,0]]

Output: 2.5

Input: mu = [[0.5,-0.25],[1,0]], log_var = [[-1,0.5],[0.2,-0.4]]

Output: 0.480206

Hint 1

Recover variance with np.exp(log_var).

Hint 2

Reduce the latent axis before averaging the batch.

Requirements

  • Use NumPy.
  • Apply the closed-form diagonal-Gaussian KL expression.
  • Sum over latent coordinates and average over samples.
  • Return a finite Python float.

Constraints

  • mu and log_var have the same shape (B, L).
  • Both arrays have dtype float64 and contain finite values.
  • B and L are positive.
Prev4 / 6Next