Label smoothing is a regularization technique that prevents a model from becoming overconfident. Instead of using hard one-hot targets (1 for the correct class, 0 for all others), it softens the target distribution by redistributing a small fraction of probability mass to the incorrect classes.
Given a predicted probability distribution, a target class index, and a smoothing parameter epsilon, compute the cross-entropy loss with smoothed labels.
Input: predictions = [0.9, 0.05, 0.05], target = 0, epsilon = 0.1
Output: 0.29805196618423724
Explanation: The correct class receives 0.933333 target mass and each other class receives 0.033333.
Input: predictions = [0.7, 0.3], target = 0, epsilon = 0.2
Output: 0.44140472997745284
Give every class epsilon divided by the class count.
Add one minus epsilon to the target class before accumulating cross-entropy.
Accepts: array
Accepts: number
Accepts: number
Label smoothing is a regularization technique that prevents a model from becoming overconfident. Instead of using hard one-hot targets (1 for the correct class, 0 for all others), it softens the target distribution by redistributing a small fraction of probability mass to the incorrect classes.
Given a predicted probability distribution, a target class index, and a smoothing parameter epsilon, compute the cross-entropy loss with smoothed labels.
Input: predictions = [0.9, 0.05, 0.05], target = 0, epsilon = 0.1
Output: 0.29805196618423724
Explanation: The correct class receives 0.933333 target mass and each other class receives 0.033333.
Input: predictions = [0.7, 0.3], target = 0, epsilon = 0.2
Output: 0.44140472997745284
Give every class epsilon divided by the class count.
Add one minus epsilon to the target class before accumulating cross-entropy.
Accepts: array
Accepts: number
Accepts: number