Compute the noise distribution used by Word2Vec negative sampling. Raise each corpus count to a supplied power and normalize the results.
Pn(w)=∑u=1Vf(u)αf(w)αHere, f(w) is the corpus count of word w, V is the vocabulary size, and alpha is α. The Word2Vec negative-sampling experiments use α=3/4. Return the probabilities as a float64 PyTorch tensor with shape (V).
Examples
Input: counts = [100,10,1], alpha = 0.75
Output: [0.826822,0.147032,0.026146]
Explanation: The three-quarter power flattens the count ratios before normalization.
Input: counts = [5,5,5,5], alpha = 0.75
Output: [0.25,0.25,0.25,0.25]
Input: counts = [100,10,1], alpha = 1
Output: [0.900901,0.09009,0.009009]
Hint 1
Raise the float64 count tensor to alpha before taking the sum.
Hint 2
Divide every powered count by the same partition value.
Requirements
- Use PyTorch.
- Apply alpha to the raw counts before normalization.
- Preserve vocabulary order.
- Return a nonnegative float64 tensor whose entries sum to one.
Constraints
- counts is a one-dimensional tensor of positive word counts.
- alpha is between zero and one, inclusive.
- The vocabulary contains at least two words.
- The runner supplies counts as torch.float64.
All steps in Word2Vec
- Subsampling
- Skip-gram Pairs
- Negative Sampling
- SGNS Loss
- CBOW Forward
- SGNS Step
Accepts: array
Accepts: number
Negative Sampling
Compute the noise distribution used by Word2Vec negative sampling. Raise each corpus count to a supplied power and normalize the results.
Pn(w)=∑u=1Vf(u)αf(w)αHere, f(w) is the corpus count of word w, V is the vocabulary size, and alpha is α. The Word2Vec negative-sampling experiments use α=3/4. Return the probabilities as a float64 PyTorch tensor with shape (V).
Examples
Input: counts = [100,10,1], alpha = 0.75
Output: [0.826822,0.147032,0.026146]
Explanation: The three-quarter power flattens the count ratios before normalization.
Input: counts = [5,5,5,5], alpha = 0.75
Output: [0.25,0.25,0.25,0.25]
Input: counts = [100,10,1], alpha = 1
Output: [0.900901,0.09009,0.009009]
Hint 1
Raise the float64 count tensor to alpha before taking the sum.
Hint 2
Divide every powered count by the same partition value.
Requirements
- Use PyTorch.
- Apply alpha to the raw counts before normalization.
- Preserve vocabulary order.
- Return a nonnegative float64 tensor whose entries sum to one.
Constraints
- counts is a one-dimensional tensor of positive word counts.
- alpha is between zero and one, inclusive.
- The vocabulary contains at least two words.
- The runner supplies counts as torch.float64.
All steps in Word2Vec
- Subsampling
- Skip-gram Pairs
- Negative Sampling
- SGNS Loss
- CBOW Forward
- SGNS Step
Accepts: array
Accepts: number