Fit a data scaling law to observed data sizes and losses, using the supplied candidate loss floors:
L(D)=E+AD−α.Here, D is a data size, L(D) is its predicted loss, E is the irreducible-loss floor, A is a positive coefficient, and α is a positive decay exponent. For observation i, Di and Li are its data size and loss. A candidate floor is eligible only when it is below every observed loss. Its unweighted least-squares fit is defined in natural-log space by:
log(Li−E)=a+blogDi.The intercept is a=logA and the slope is b=−α. Consider only fits with positive, finite coefficient and exponent. Select the fit with the smallest root mean squared error (RMSE) on the original loss scale, preserving candidate order on exact ties:
RMSE=n1i=0∑n−1(L(Di)−Li)2.Here, n is the number of observations. Return a Python dictionary with floor, coefficient, exponent, and rmse as Python floats, plus predictions as a float64 PyTorch tensor matching target_sizes in shape, order, and device.
Input: data_sizes = [1,4,16], losses = [5,3,2], target_sizes = [64], floor_candidates = [0,1]
Output: {"floor":1,"coefficient":4,"exponent":0.5,"predictions":[1.5],"rmse":0}
Explanation: Subtracting floor 1 leaves the exact curve 4D^(-0.5), so its raw-loss error is zero.
Input: data_sizes = [1,2,4], losses = [3,2,1.5], target_sizes = [8], floor_candidates = [1]
Output: {"floor":1,"coefficient":2,"exponent":1,"predictions":[1.25],"rmse":0}
Input: data_sizes = [1,2,4], losses = [2.5,1.5,1], target_sizes = [8,16], floor_candidates = [0,0.5]
Output: {"floor":0.5,"coefficient":2,"exponent":1,"predictions":[0.75,0.625],"rmse":0}
torch.log computes natural logarithms elementwise.
torch.linalg.lstsq solves a least-squares system.
Tensor.square and Tensor.mean help measure errors on the original scale.
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Fit a data scaling law to observed data sizes and losses, using the supplied candidate loss floors:
L(D)=E+AD−α.Here, D is a data size, L(D) is its predicted loss, E is the irreducible-loss floor, A is a positive coefficient, and α is a positive decay exponent. For observation i, Di and Li are its data size and loss. A candidate floor is eligible only when it is below every observed loss. Its unweighted least-squares fit is defined in natural-log space by:
log(Li−E)=a+blogDi.The intercept is a=logA and the slope is b=−α. Consider only fits with positive, finite coefficient and exponent. Select the fit with the smallest root mean squared error (RMSE) on the original loss scale, preserving candidate order on exact ties:
RMSE=n1i=0∑n−1(L(Di)−Li)2.Here, n is the number of observations. Return a Python dictionary with floor, coefficient, exponent, and rmse as Python floats, plus predictions as a float64 PyTorch tensor matching target_sizes in shape, order, and device.
Input: data_sizes = [1,4,16], losses = [5,3,2], target_sizes = [64], floor_candidates = [0,1]
Output: {"floor":1,"coefficient":4,"exponent":0.5,"predictions":[1.5],"rmse":0}
Explanation: Subtracting floor 1 leaves the exact curve 4D^(-0.5), so its raw-loss error is zero.
Input: data_sizes = [1,2,4], losses = [3,2,1.5], target_sizes = [8], floor_candidates = [1]
Output: {"floor":1,"coefficient":2,"exponent":1,"predictions":[1.25],"rmse":0}
Input: data_sizes = [1,2,4], losses = [2.5,1.5,1], target_sizes = [8,16], floor_candidates = [0,0.5]
Output: {"floor":0.5,"coefficient":2,"exponent":1,"predictions":[0.75,0.625],"rmse":0}
torch.log computes natural logarithms elementwise.
torch.linalg.lstsq solves a least-squares system.
Tensor.square and Tensor.mean help measure errors on the original scale.
Accepts: array
Accepts: array
Accepts: array
Accepts: array
