Expected Calibration Error measures the gap between predicted confidence and observed accuracy. Split probabilities into equal-width bins over [0,1]. A probability of exactly one belongs to the final bin.
For each nonempty bin, compute its mean target and mean predicted probability:
acc(Bm)=∣Bm∣1i∈Bm∑yi conf(Bm)=∣Bm∣1i∈Bm∑piThen compute:
ECE=m=1∑Mn∣Bm∣∣acc(Bm)−conf(Bm)∣Here, M is the number of bins, n is the number of predictions, yi is a binary target, and pi is its predicted probability. Return ECE as a Python float.
Input: y_true = [1, 0, 1, 0], y_pred = [0.9, 0.9, 0.9, 0.9], n_bins = 5
Output: 0.4
Explanation: All predictions share one bin whose accuracy is 0.5 and mean confidence is 0.9.
Input: y_true = [0, 0, 1, 1, 0, 1, 1, 1], y_pred = [0.1, 0.2, 0.3, 0.4, 0.6, 0.7, 0.8, 0.9], n_bins = 2
Output: 0.125
Use min(int(probability * n_bins), n_bins - 1) for the bin index.
Accumulate targets, probabilities, and counts separately for each bin.
Accepts: array
Accepts: array
Accepts: number
Expected Calibration Error measures the gap between predicted confidence and observed accuracy. Split probabilities into equal-width bins over [0,1]. A probability of exactly one belongs to the final bin.
For each nonempty bin, compute its mean target and mean predicted probability:
acc(Bm)=∣Bm∣1i∈Bm∑yi conf(Bm)=∣Bm∣1i∈Bm∑piThen compute:
ECE=m=1∑Mn∣Bm∣∣acc(Bm)−conf(Bm)∣Here, M is the number of bins, n is the number of predictions, yi is a binary target, and pi is its predicted probability. Return ECE as a Python float.
Input: y_true = [1, 0, 1, 0], y_pred = [0.9, 0.9, 0.9, 0.9], n_bins = 5
Output: 0.4
Explanation: All predictions share one bin whose accuracy is 0.5 and mean confidence is 0.9.
Input: y_true = [0, 0, 1, 1, 0, 1, 1, 1], y_pred = [0.1, 0.2, 0.3, 0.4, 0.6, 0.7, 0.8, 0.9], n_bins = 2
Output: 0.125
Use min(int(probability * n_bins), n_bins - 1) for the bin index.
Accumulate targets, probabilities, and counts separately for each bin.
Accepts: array
Accepts: array
Accepts: number