Compute the mean cross-entropy loss over a batch of B examples with C classes. Given a logits matrix of shape (B,C) and integer targets of length B with values in [0,C), the per-row log-softmax is:
logsoftmax(z)j=zj−kmaxzk−logk∑exp(zk−kmaxzk)and the mean loss across the batch is:
L=−B1i=0∑B−1logsoftmax(logitsi)targetilogits is a device pointer to a row-major B×C float32 buffer, target is a device pointer to a length-B int32 buffer, and loss is a single float32 scalar. Use the max-subtract trick for numerical stability.
Input: B = 2, C = 3, logits = [[1, 2, 3], [3, 2, 1]], target = [2, 0]
Output: 0.4076
Both rows have log-sum-exp ≈0.4076, the target logit equals the row max in each case, so −logsoftmaxtarget≈0.4076 for each row, averaging to the same value.
Input: B = 1, C = 4, logits = [[10, 0, 0, 0]], target = [0]
Output: 0.000136
The target logit dominates the row, so softmaxtarget≈1 and the loss is nearly zero.
Subtract the row max from each logit before calling expf, otherwise large logits overflow float32.
Use one block per row to compute the max and log-sum-exp with shared-memory reductions, then have thread 0 contribute that row's loss with atomicAdd into a single scalar.
A small finalize kernel that divides the accumulated sum by B keeps the launcher clean and avoids host-side math on the scalar.
Accepts: array
Accepts: array
Submit a correct solution to see your kernel performance.
Compute the mean cross-entropy loss over a batch of B examples with C classes. Given a logits matrix of shape (B,C) and integer targets of length B with values in [0,C), the per-row log-softmax is:
logsoftmax(z)j=zj−kmaxzk−logk∑exp(zk−kmaxzk)and the mean loss across the batch is:
L=−B1i=0∑B−1logsoftmax(logitsi)targetilogits is a device pointer to a row-major B×C float32 buffer, target is a device pointer to a length-B int32 buffer, and loss is a single float32 scalar. Use the max-subtract trick for numerical stability.
Input: B = 2, C = 3, logits = [[1, 2, 3], [3, 2, 1]], target = [2, 0]
Output: 0.4076
Both rows have log-sum-exp ≈0.4076, the target logit equals the row max in each case, so −logsoftmaxtarget≈0.4076 for each row, averaging to the same value.
Input: B = 1, C = 4, logits = [[10, 0, 0, 0]], target = [0]
Output: 0.000136
The target logit dominates the row, so softmaxtarget≈1 and the loss is nearly zero.
Subtract the row max from each logit before calling expf, otherwise large logits overflow float32.
Use one block per row to compute the max and log-sum-exp with shared-memory reductions, then have thread 0 contribute that row's loss with atomicAdd into a single scalar.
A small finalize kernel that divides the accumulated sum by B keeps the launcher clean and avoids host-side math on the scalar.
Accepts: array
Accepts: array
Submit a correct solution to see your kernel performance.