Problems
Loading...
Step 1 of 6
Subsampling
Easy

Compute the keep probability associated with the Word2Vec frequent-word subsampling rule. The paper gives the discard probability as 1−t/f(w)1-\sqrt{t/f(w)}, so the corresponding keep probability is clipped to the valid probability range.

f(w)=count⁡(w)∑u=1Vcount⁡(u)f(w)=\frac{\operatorname{count}(w)}{\sum_{u=1}^{V}\operatorname{count}(u)} Pkeep(w)=min⁡ ⁣(1,tf(w))P_{\mathrm{keep}}(w)=\min\!\left(1,\sqrt{\frac{t}{f(w)}}\right)

Here, f(w)f(w) is the corpus frequency of word w, t is the subsampling threshold, and V is the vocabulary size. Return the keep probabilities in vocabulary order as a float64 PyTorch tensor with shape (V)(V).

Loading visualization...

Examples

Input: counts = [100,50,10], t = 0.00001

Output: [0.004,0.005657,0.012649]

Explanation: The most frequent word receives the smallest keep probability after counts become corpus frequencies.

Input: counts = [25,25,25,25], t = 0.001

Output: [0.063246,0.063246,0.063246,0.063246]

Input: counts = [10000,1,1], t = 0.001

Output: [0.031626,1,1]

Hint 1

Divide counts by counts.sum() before applying the threshold.

Hint 2

Use torch.clamp with max=1.0 after the square root.

Requirements

  • Use PyTorch.
  • Convert corpus counts to relative frequencies.
  • Apply the paper's square-root subsampling rule.
  • Clamp keep probabilities to at most one.
  • Return the documented float64 tensor.

Constraints

  • counts is a one-dimensional tensor of positive word counts.
  • t is positive.
  • The vocabulary contains at least two words.
  • The runner supplies counts as torch.float64.
Prev1 / 6Next