Compute Information Gain of a binary split for classification labels. Given labels y and a boolean split_mask indicating the left/right partition, return the information gain using entropy (base 2).
Shannon Entropy (base 2):
H(Y)=−k∑pklog2pkInformation Gain:
IG=H(Y)−(NnLH(YL)+NnRH(YR))where Y is the parent set of labels, YL,YR are the labels in left/right subsets after the split, and N=nL+nR.
Input: y=[0,0,1,1], mask=[T,T,F,F]
Output: 1.0 (perfect split)
Input: y=[0,0,0,1], mask=[T,F,F,T]
Output: 0.311278
Input: y=[0,0,1,2], mask=[T,F,T,F]
Output: 0.5
Use np.unique(y, return_counts=True) to compute label probabilities and entropy.
If one side is empty, return 0.0 (the split provides no information).
IG = 0.0Sign in to take notes on this problem
Accepts: array
Accepts: array
Compute Information Gain of a binary split for classification labels. Given labels y and a boolean split_mask indicating the left/right partition, return the information gain using entropy (base 2).
Shannon Entropy (base 2):
H(Y)=−k∑pklog2pkInformation Gain:
IG=H(Y)−(NnLH(YL)+NnRH(YR))where Y is the parent set of labels, YL,YR are the labels in left/right subsets after the split, and N=nL+nR.
Input: y=[0,0,1,1], mask=[T,T,F,F]
Output: 1.0 (perfect split)
Input: y=[0,0,0,1], mask=[T,F,F,T]
Output: 0.311278
Input: y=[0,0,1,2], mask=[T,F,T,F]
Output: 0.5
Use np.unique(y, return_counts=True) to compute label probabilities and entropy.
If one side is empty, return 0.0 (the split provides no information).
IG = 0.0Sign in to take notes on this problem
Accepts: array
Accepts: array