Problems

1 / 1

Compute Mean Average Precision (mAP)

Metrics & Evaluation

Hard

For a set of retrieval queries, compute Average Precision (AP) per query and Mean Average Precision (mAP) across all queries. Each query has items with binary relevance labels and real-valued scores.

Mean Average Precision is a standard evaluation metric in information retrieval and object detection. It measures the quality of ranked retrieval results by considering both precision and the ranking order of relevant items.

mAP Formulation:

Average Precision for a single query:

\mathrm{AP} = \frac{1}{R} \sum_{k=1}^{n} P(k)\,\mathrm{rel}(k)

Mean Average Precision across queries:

\mathrm{mAP} = \frac{1}{Q} \sum_{q=1}^{Q} \mathrm{AP}_{q}

Where P(k) is precision at rank k, rel(k) is relevance at rank k, R is total relevant items, and Q is number of queries.

Function Arguments

y_true_list: list of arrays - Binary relevance labels {0,1} for each query
y_score_list: list of arrays - Real-valued scores for each query (same lengths)
k: optional int - Cutoff rank; if None, use full length

Loading visualization...

Examples

Input: y_true_list = [[1, 0, 1, 0]], y_score_list = [[0.9, 0.8, 0.7, 0.1]]

Output: (0.8333, [0.8333])

Sorted by score: labels become [1, 0, 1, 0]. P(1)=1.0, P(3)=2/3. AP = (1.0 + 0.667)/2 = 0.8333

Input: y_true_list = [[1, 0, 1], [1, 1, 0]], y_score_list = [[0.9, 0.8, 0.7], [0.9, 0.8, 0.7]]

Output: (0.9167, [0.8333, 1.0])

Query 1: AP = 0.8333. Query 2: both relevant items at top, AP = 1.0. mAP = (0.8333 + 1.0)/2 = 0.9167

Hint 1

Sort indices by descending score using np.argsort() for each query.

Hint 2

Use np.cumsum() to get cumulative relevant items and compute precision at each rank.

Hint 3

Average precision is the mean of precisions at positions where items are relevant.

Requirements

Return tuple: (map_value, ap_per_query)
Sort items by score descending for each query
AP = average of P@i over ranks i where item i is relevant (up to k)
Handle queries with zero relevant items → AP = 0
Support optional cutoff k (e.g., mAP@10)
Vectorized within each query where possible

Constraints

Total items across queries ≤ 1e6; NumPy only
Time limit: 800ms; Memory: 512MB

Try Similar Problems

Precision Recall At K Hit Rate At K Classification Metrics Iou Bounding Box Ndcg

Prev Next

Compute Mean Average Precision (mAP)

Metrics & Evaluation

Hard

mAP Formulation:

Average Precision for a single query:

\mathrm{AP} = \frac{1}{R} \sum_{k=1}^{n} P(k)\,\mathrm{rel}(k)

Mean Average Precision across queries:

\mathrm{mAP} = \frac{1}{Q} \sum_{q=1}^{Q} \mathrm{AP}_{q}

Where P(k) is precision at rank k, rel(k) is relevance at rank k, R is total relevant items, and Q is number of queries.

Function Arguments

y_true_list: list of arrays - Binary relevance labels {0,1} for each query
y_score_list: list of arrays - Real-valued scores for each query (same lengths)
k: optional int - Cutoff rank; if None, use full length

Loading visualization...

Examples

Input: y_true_list = [[1, 0, 1, 0]], y_score_list = [[0.9, 0.8, 0.7, 0.1]]

Output: (0.8333, [0.8333])

Sorted by score: labels become [1, 0, 1, 0]. P(1)=1.0, P(3)=2/3. AP = (1.0 + 0.667)/2 = 0.8333

Input: y_true_list = [[1, 0, 1], [1, 1, 0]], y_score_list = [[0.9, 0.8, 0.7], [0.9, 0.8, 0.7]]

Output: (0.9167, [0.8333, 1.0])

Query 1: AP = 0.8333. Query 2: both relevant items at top, AP = 1.0. mAP = (0.8333 + 1.0)/2 = 0.9167

Hint 1

Sort indices by descending score using np.argsort() for each query.

Hint 2

Use np.cumsum() to get cumulative relevant items and compute precision at each rank.

Hint 3

Average precision is the mean of precisions at positions where items are relevant.

Requirements

Return tuple: (map_value, ap_per_query)
Sort items by score descending for each query
AP = average of P@i over ranks i where item i is relevant (up to k)
Handle queries with zero relevant items → AP = 0
Support optional cutoff k (e.g., mAP@10)
Vectorized within each query where possible

Constraints

Total items across queries ≤ 1e6; NumPy only
Time limit: 800ms; Memory: 512MB

Try Similar Problems

Precision Recall At K Hit Rate At K Classification Metrics Iou Bounding Box Ndcg

Case 1

Case 2

y_true_list =

Accepts: array

y_score_list =

Accepts: array

k =

Accepts: any

mAP Formulation:

Average Precision for a single query:

\mathrm{AP} = \frac{1}{R} \sum_{k=1}^{n} P(k)\,\mathrm{rel}(k)

Mean Average Precision across queries:

\mathrm{mAP} = \frac{1}{Q} \sum_{q=1}^{Q} \mathrm{AP}_{q}

Where P(k) is precision at rank k, rel(k) is relevance at rank k, R is total relevant items, and Q is number of queries.

Function Arguments

y_true_list: list of arrays - Binary relevance labels {0,1} for each query
y_score_list: list of arrays - Real-valued scores for each query (same lengths)
k: optional int - Cutoff rank; if None, use full length