Problems

1 / 1

Perplexity Computation

NLP

Easy

Perplexity is the standard metric for evaluating language models. It measures how "surprised" a model is by a sequence of tokens. A lower perplexity means the model assigns higher probability to the observed sequence, indicating better predictions.

Given a list of probability distributions (one per position) and the actual token indices, compute the perplexity of the sequence.

Algorithm

For each position i, extract the probability assigned to the actual token:

p_i = P_i[t_i]

Compute the average negative log-probability (cross-entropy):

H = -\frac{1}{N} \sum_{i=1}^{N} \log p_i

Perplexity is the exponential of the cross-entropy:

PP = e^{H}

Return perplexity as a positive float rounded to four decimals.

Loading visualization...

Examples

Input: prob_distributions = [[0.5, 0.5], [0.5, 0.5]], actual_tokens = [0, 1]

Output: 2.0

Explanation: Both observed tokens have probability 0.5, so exponentiating the mean negative log probability gives 2.

Input: prob_distributions = [[1, 0], [0, 1]], actual_tokens = [0, 1]

Output: 1.0

Hint 1

Read the observed-token probability from each row before taking its logarithm.

Hint 2

Negate the mean log probability, exponentiate it, and round the result.

Requirements

Extract the predicted probability for each actual token from the corresponding distribution
Compute cross-entropy as the negative mean of log-probabilities
Return perplexity as exp(cross-entropy)
Return a float

Constraints

prob_distributions is a list of N probability distributions (each a list of positive floats summing to 1)
actual_tokens is a list of N valid indices into the corresponding distributions
All relevant probabilities are positive (no log(0) issues)
Return a positive float
Time limit: 300 ms

Try Similar Problems

Bigram Probabilities Cross Entropy Loss Word Count Dict Bag Of Words Bleu Score

Prev Next

Perplexity Computation

NLP

Easy

Given a list of probability distributions (one per position) and the actual token indices, compute the perplexity of the sequence.

Algorithm

For each position i, extract the probability assigned to the actual token:

p_i = P_i[t_i]

Compute the average negative log-probability (cross-entropy):

H = -\frac{1}{N} \sum_{i=1}^{N} \log p_i

Perplexity is the exponential of the cross-entropy:

PP = e^{H}

Return perplexity as a positive float rounded to four decimals.

Loading visualization...

Examples

Input: prob_distributions = [[0.5, 0.5], [0.5, 0.5]], actual_tokens = [0, 1]

Output: 2.0

Explanation: Both observed tokens have probability 0.5, so exponentiating the mean negative log probability gives 2.

Input: prob_distributions = [[1, 0], [0, 1]], actual_tokens = [0, 1]

Output: 1.0

Hint 1

Read the observed-token probability from each row before taking its logarithm.

Hint 2

Negate the mean log probability, exponentiate it, and round the result.

Requirements

Extract the predicted probability for each actual token from the corresponding distribution
Compute cross-entropy as the negative mean of log-probabilities
Return perplexity as exp(cross-entropy)
Return a float

Constraints

prob_distributions is a list of N probability distributions (each a list of positive floats summing to 1)
actual_tokens is a list of N valid indices into the corresponding distributions
All relevant probabilities are positive (no log(0) issues)
Return a positive float
Time limit: 300 ms

Try Similar Problems

Bigram Probabilities Cross Entropy Loss Word Count Dict Bag Of Words Bleu Score

Case 1

Case 2

prob_distributions =

Accepts: array

actual_tokens =

Accepts: array

Given a list of probability distributions (one per position) and the actual token indices, compute the perplexity of the sequence.

Algorithm

For each position i, extract the probability assigned to the actual token:

p_i = P_i[t_i]

Compute the average negative log-probability (cross-entropy):

H = -\frac{1}{N} \sum_{i=1}^{N} \log p_i

Perplexity is the exponential of the cross-entropy:

PP = e^{H}

Return perplexity as a positive float rounded to four decimals.