Compute sampling probabilities and expected exposure for a mixture of data sources. Let ni be source i's token count, α the supplied exponent, and T the training-token budget. For positive-size sources:
pi=∑j:nj>0njαniα ti=Tpi ei=niti.Here, pi is the sampling probability, ti the expected sampled tokens, and ei the expected passes through source i. The sum includes only positive-size sources. A zero-size source receives zero for all three values, including when the exponent is zero. If all sources are empty, return zero arrays.
Return a Python dictionary with probabilities, expected_tokens, and expected_epochs, each a float64 NumPy array of shape (N,) in source order, where N is the number of sources.
Input: token_counts = [100,400], alpha = 1, training_budget = 1000
Output: {"probabilities":[0.2,0.8],"expected_tokens":[200,800],"expected_epochs":[2,2]}
Explanation: With exponent one, sampling probability is proportional to source size.
Input: token_counts = [1,100], alpha = 0, training_budget = 20
Output: {"probabilities":[0.5,0.5],"expected_tokens":[10,10],"expected_epochs":[10,0.1]}
Input: token_counts = [0,10], alpha = 0.5, training_budget = 30
Output: {"probabilities":[0,1],"expected_tokens":[0,30],"expected_epochs":[0,3]}
np.log expresses power weights on a logarithmic scale.
Subtracting a shared log-weight offset leaves normalized probabilities unchanged.
np.divide supports a mask through its where argument.
Accepts: array
Accepts: number
Accepts: number
Compute sampling probabilities and expected exposure for a mixture of data sources. Let ni be source i's token count, α the supplied exponent, and T the training-token budget. For positive-size sources:
pi=∑j:nj>0njαniα ti=Tpi ei=niti.Here, pi is the sampling probability, ti the expected sampled tokens, and ei the expected passes through source i. The sum includes only positive-size sources. A zero-size source receives zero for all three values, including when the exponent is zero. If all sources are empty, return zero arrays.
Return a Python dictionary with probabilities, expected_tokens, and expected_epochs, each a float64 NumPy array of shape (N,) in source order, where N is the number of sources.
Input: token_counts = [100,400], alpha = 1, training_budget = 1000
Output: {"probabilities":[0.2,0.8],"expected_tokens":[200,800],"expected_epochs":[2,2]}
Explanation: With exponent one, sampling probability is proportional to source size.
Input: token_counts = [1,100], alpha = 0, training_budget = 20
Output: {"probabilities":[0.5,0.5],"expected_tokens":[10,10],"expected_epochs":[10,0.1]}
Input: token_counts = [0,10], alpha = 0.5, training_budget = 30
Output: {"probabilities":[0,1],"expected_tokens":[0,30],"expected_epochs":[0,3]}
np.log expresses power weights on a logarithmic scale.
Subtracting a shared log-weight offset leaves normalized probabilities unchanged.
np.divide supports a mask through its where argument.
Accepts: array
Accepts: number
Accepts: number
