Compute monitoring metrics for one of three ML system types. Return a dictionary whose keys depend on system_type.
For "classification", return accuracy, precision, recall, and F1. Use zero when a precision, recall, or F1 denominator is zero.
accuracy=nTP+TN precision=TP+FPTP recall=TP+FNTP F1=P+R2PRFor "regression", return mean absolute error and root mean squared error.
MAE=n1i∑∣yi−y^i∣ RMSE=n1i∑(yi−y^i)2For "ranking", sort items by descending predicted score and return precision at 3 and recall at 3. Tied scores retain input order. Precision at 3 always divides by 3. Here, P is precision, R is recall, yi is a target, and y^i is a prediction.
Input: system_type = "classification", y_true = [1, 0, 1, 1, 0, 1, 0, 0], y_pred = [1, 0, 0, 1, 0, 1, 1, 0]
Output: {"accuracy": 0.75, "precision": 0.75, "recall": 0.75, "f1": 0.75}
Explanation: There are three true positives, three true negatives, one false positive, and one false negative.
Input: system_type = "regression", y_true = [3, 5, 2.5, 7], y_pred = [2.5, 5.5, 2, 8]
Output: {"mae": 0.625, "rmse": 0.661438}
For classification, count tp, fp, fn, and tn in one pass over paired values.
For ranking, sort indices with key=lambda i: y_pred[i], reverse=True and inspect the first three.
Accepts: string
Accepts: array
Accepts: array
Compute monitoring metrics for one of three ML system types. Return a dictionary whose keys depend on system_type.
For "classification", return accuracy, precision, recall, and F1. Use zero when a precision, recall, or F1 denominator is zero.
accuracy=nTP+TN precision=TP+FPTP recall=TP+FNTP F1=P+R2PRFor "regression", return mean absolute error and root mean squared error.
MAE=n1i∑∣yi−y^i∣ RMSE=n1i∑(yi−y^i)2For "ranking", sort items by descending predicted score and return precision at 3 and recall at 3. Tied scores retain input order. Precision at 3 always divides by 3. Here, P is precision, R is recall, yi is a target, and y^i is a prediction.
Input: system_type = "classification", y_true = [1, 0, 1, 1, 0, 1, 0, 0], y_pred = [1, 0, 0, 1, 0, 1, 1, 0]
Output: {"accuracy": 0.75, "precision": 0.75, "recall": 0.75, "f1": 0.75}
Explanation: There are three true positives, three true negatives, one false positive, and one false negative.
Input: system_type = "regression", y_true = [3, 5, 2.5, 7], y_pred = [2.5, 5.5, 2, 8]
Output: {"mae": 0.625, "rmse": 0.661438}
For classification, count tp, fp, fn, and tn in one pass over paired values.
For ranking, sort indices with key=lambda i: y_pred[i], reverse=True and inspect the first three.
Accepts: string
Accepts: array
Accepts: array