Problems

1 / 1

Compute Information Gain for a Split

Classic ML

Medium

Compute the information gain produced by a boolean split of classification labels.

H(Y)=-\sum_{c=1}^{C}p_c\log_2 p_c

\operatorname{IG}=H(Y)-\frac{N_L}{N}H(Y_L)-\frac{N_R}{N}H(Y_R)

Here, $p_c$ is the proportion of class $c$ , $Y_L$ contains labels where split_mask is true, $Y_R$ contains the remaining labels, and $N=N_L+N_R$ . A split with an empty side has gain zero. Return the gain as a Python float.

Loading visualization...

Examples

Input: y = [0, 0, 1, 1], split_mask = [true, true, false, false]

Output: 1.0

Explanation: Each child is pure, so the split removes all one bit of parent entropy.

Input: y = [0, 0, 0, 1], split_mask = [true, false, false, true]

Output: 0.311278

Input: y = [0, 0, 1, 2], split_mask = [true, false, true, false]

Output: 0.5

Hint 1

Use np.unique(labels, return_counts=True) inside a small entropy helper.

Hint 2

Form children with labels[mask] and labels[~mask].

Requirements

Compute base-2 entropy from class frequencies
Partition labels with the boolean split mask
Weight child entropies by child size
Return a Python float, or zero when either child is empty

Constraints

y and split_mask are one-dimensional and have equal lengths
Labels are integers and the mask contains booleans
Use NumPy only

Try Similar Problems

Entropy Node Gini Impurity Decision Tree Split Random Forest Vote Naive Bayes Bernoulli

Prev Next