Compute the KL divergence KL(p∥q) between two diagonal Gaussians for every row of a batch, summing over the last axis.
KL(p∥q)=21d=1∑D(logσq,d2−logσp,d2+σq,d2σp,d2+(μp,d−μq,d)2−1)Here mean_p and log_var_p hold the means μp and log variances logσp2 of the first Gaussian p, and mean_q and log_var_q hold those of the second Gaussian q. All four arrays have the same shape (..., D), where the last axis indexes the D independent coordinates d of one Gaussian and every leading index describes a separate pair of Gaussians. The variances are σ2=exp(logσ2). The divergence is measured from p to q, and swapping the two Gaussians gives a different value in general.
Return a NumPy array of dtype float64 with the leading shape of the inputs, so inputs of shape (N, D) give shape (N,).
Input: mean_p = [[1.0,0.0],[0.0,0.0]], log_var_p = [[0.0,0.0],[0.0,0.0]], mean_q = [[0.0,0.0],[0.0,0.0]], log_var_q = [[0.0,0.0],[1.3863,1.3863]]
Output: [0.5,0.636299]
Explanation: The first row compares N(1, 1) with N(0, 1) in its first coordinate, which contributes half the squared mean gap, 0.5. The second row compares unit variances with variances of about 4, which gives about 0.636.
Input: mean_p = [[0.5],[-2.0],[3.0]], log_var_p = [[-1.0],[0.0],[2.0]], mean_q = [[0.5],[-2.0],[3.0]], log_var_q = [[-1.0],[0.0],[2.0]]
Output: [0.0,0.0,0.0]
Input: mean_p = [[[0.0,1.0,-1.0]],[[2.0,0.0,0.5]]], log_var_p = [[[-0.6931,0.0,0.6931]],[[0.0,-2.0,0.0]]], mean_q = [[[0.0,0.0,0.0]],[[1.0,0.0,0.0]]], log_var_q = [[[0.0,0.0,0.0]],[[0.0,0.0,-1.0]]]
Output: [[1.249965],[1.766594]]
np.exp(log_var) recovers each variance from its log.
The ratio of two variances equals the exponential of the difference of their log variances.
np.sum(x, axis=-1) reduces only the coordinate axis, so shape (N, D) becomes (N,).
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Compute the KL divergence KL(p∥q) between two diagonal Gaussians for every row of a batch, summing over the last axis.
KL(p∥q)=21d=1∑D(logσq,d2−logσp,d2+σq,d2σp,d2+(μp,d−μq,d)2−1)Here mean_p and log_var_p hold the means μp and log variances logσp2 of the first Gaussian p, and mean_q and log_var_q hold those of the second Gaussian q. All four arrays have the same shape (..., D), where the last axis indexes the D independent coordinates d of one Gaussian and every leading index describes a separate pair of Gaussians. The variances are σ2=exp(logσ2). The divergence is measured from p to q, and swapping the two Gaussians gives a different value in general.
Return a NumPy array of dtype float64 with the leading shape of the inputs, so inputs of shape (N, D) give shape (N,).
Input: mean_p = [[1.0,0.0],[0.0,0.0]], log_var_p = [[0.0,0.0],[0.0,0.0]], mean_q = [[0.0,0.0],[0.0,0.0]], log_var_q = [[0.0,0.0],[1.3863,1.3863]]
Output: [0.5,0.636299]
Explanation: The first row compares N(1, 1) with N(0, 1) in its first coordinate, which contributes half the squared mean gap, 0.5. The second row compares unit variances with variances of about 4, which gives about 0.636.
Input: mean_p = [[0.5],[-2.0],[3.0]], log_var_p = [[-1.0],[0.0],[2.0]], mean_q = [[0.5],[-2.0],[3.0]], log_var_q = [[-1.0],[0.0],[2.0]]
Output: [0.0,0.0,0.0]
Input: mean_p = [[[0.0,1.0,-1.0]],[[2.0,0.0,0.5]]], log_var_p = [[[-0.6931,0.0,0.6931]],[[0.0,-2.0,0.0]]], mean_q = [[[0.0,0.0,0.0]],[[1.0,0.0,0.0]]], log_var_q = [[[0.0,0.0,0.0]],[[0.0,0.0,-1.0]]]
Output: [[1.249965],[1.766594]]
np.exp(log_var) recovers each variance from its log.
The ratio of two variances equals the exponential of the difference of their log variances.
np.sum(x, axis=-1) reduces only the coordinate axis, so shape (N, D) becomes (N,).
Accepts: array
Accepts: array
Accepts: array
Accepts: array
