Compute the Kullback-Leibler divergence from distribution P to distribution Q:
DKL(P∥Q)=i∑Pilog(QiPi)Here, Pi and Qi are corresponding probabilities. A term with Pi=0 contributes zero. Clamp positive-indexed Qi values to at least eps before division, then return the sum as a Python float.
Input: p = [0.4, 0.6], q = [0.5, 0.5], eps = 1e-12
Output: 0.020136
Explanation: The distributions are similar but not identical, so the divergence is small and positive.
Input: p = [0.3, 0.7], q = [0.3, 0.7], eps = 1e-12
Output: 0.0
Input: p = [0.9, 0.1], q = [0.5, 0.5], eps = 1e-12
Output: 0.368064
Create a mask with positive = p > 0.
Use np.clip(q[positive], eps, None) before computing the log ratio.
Accepts: array
Accepts: array
Accepts: number
Compute the Kullback-Leibler divergence from distribution P to distribution Q:
DKL(P∥Q)=i∑Pilog(QiPi)Here, Pi and Qi are corresponding probabilities. A term with Pi=0 contributes zero. Clamp positive-indexed Qi values to at least eps before division, then return the sum as a Python float.
Input: p = [0.4, 0.6], q = [0.5, 0.5], eps = 1e-12
Output: 0.020136
Explanation: The distributions are similar but not identical, so the divergence is small and positive.
Input: p = [0.3, 0.7], q = [0.3, 0.7], eps = 1e-12
Output: 0.0
Input: p = [0.9, 0.1], q = [0.5, 0.5], eps = 1e-12
Output: 0.368064
Create a mask with positive = p > 0.
Use np.clip(q[positive], eps, None) before computing the log ratio.
Accepts: array
Accepts: array
Accepts: number