Given class labels from a decision tree node, compute the entropy using the stable logarithm formula.
Entropy is a fundamental concept from information theory that measures the amount of uncertainty or randomness in a dataset. In decision trees, it's used as a splitting criterion to build trees that maximize information gain.
Entropy Formula:
H(S)=−i=1∑Cpilog2(pi)Where pi is the proportion of samples belonging to class i, and C is the number of classes. By convention, 0log2(0)=0.
y: array-like - Class labels for samples in the nodeInput: y=[1,1,1,1]
Output: 0.0
Input: y=[0,1,0,1]
Output: 1.0
Use np.unique() with return_counts=True to get class frequencies.
Filter out zero probabilities before computing logarithms to avoid numerical issues.
Use np.log2() for base-2 logarithms in the entropy formula.
Sign in to take notes on this problem
Accepts: array
Given class labels from a decision tree node, compute the entropy using the stable logarithm formula.
Entropy is a fundamental concept from information theory that measures the amount of uncertainty or randomness in a dataset. In decision trees, it's used as a splitting criterion to build trees that maximize information gain.
Entropy Formula:
H(S)=−i=1∑Cpilog2(pi)Where pi is the proportion of samples belonging to class i, and C is the number of classes. By convention, 0log2(0)=0.
y: array-like - Class labels for samples in the nodeInput: y=[1,1,1,1]
Output: 0.0
Input: y=[0,1,0,1]
Output: 1.0
Use np.unique() with return_counts=True to get class frequencies.
Filter out zero probabilities before computing logarithms to avoid numerical issues.
Use np.log2() for base-2 logarithms in the entropy formula.
Sign in to take notes on this problem
Accepts: array