Estimate critical batch size from a training sweep. For each reached target, let Bi be batch size, Si the steps taken, and Ei the number of examples consumed at sweep index i:
Ei=BiSi.Let Smin be the smallest step count and Emin the smallest example count among reached targets. These minima may come from different sweep points. The critical-batch estimate is:
Bcrit=SminEmin.Ignore unreached points when calculating the minima. Return a Python dictionary with examples, an input-aligned list of Python floats with None at unreached positions, and min_steps, min_examples, critical_batch_size, each a Python float.
Input: batch_sizes = [8,16,32], steps_to_target = [100,60,50], reached_mask = [true,true,true]
Output: {"examples":[800,960,1600],"min_steps":50,"min_examples":800,"critical_batch_size":16}
Explanation: The minimum examples and minimum steps occur at different sweep points, and their ratio is 16.
Input: batch_sizes = [4,8,16], steps_to_target = [120,80,60], reached_mask = [true,true,true]
Output: {"examples":[480,640,960],"min_steps":60,"min_examples":480,"critical_batch_size":8}
Input: batch_sizes = [2,4,8,16], steps_to_target = [200,130,90,85], reached_mask = [true,true,true,true]
Output: {"examples":[400,520,720,1360],"min_steps":85,"min_examples":400,"critical_batch_size":4.705882}
Boolean tensor indexing selects only marked entries.
Tensor.min returns the smallest selected value.
Tensor.item converts a one-element tensor to a Python scalar.
Accepts: array
Accepts: array
Accepts: array
Estimate critical batch size from a training sweep. For each reached target, let Bi be batch size, Si the steps taken, and Ei the number of examples consumed at sweep index i:
Ei=BiSi.Let Smin be the smallest step count and Emin the smallest example count among reached targets. These minima may come from different sweep points. The critical-batch estimate is:
Bcrit=SminEmin.Ignore unreached points when calculating the minima. Return a Python dictionary with examples, an input-aligned list of Python floats with None at unreached positions, and min_steps, min_examples, critical_batch_size, each a Python float.
Input: batch_sizes = [8,16,32], steps_to_target = [100,60,50], reached_mask = [true,true,true]
Output: {"examples":[800,960,1600],"min_steps":50,"min_examples":800,"critical_batch_size":16}
Explanation: The minimum examples and minimum steps occur at different sweep points, and their ratio is 16.
Input: batch_sizes = [4,8,16], steps_to_target = [120,80,60], reached_mask = [true,true,true]
Output: {"examples":[480,640,960],"min_steps":60,"min_examples":480,"critical_batch_size":8}
Input: batch_sizes = [2,4,8,16], steps_to_target = [200,130,90,85], reached_mask = [true,true,true,true]
Output: {"examples":[400,520,720,1360],"min_steps":85,"min_examples":400,"critical_batch_size":4.705882}
Boolean tensor indexing selects only marked entries.
Tensor.min returns the smallest selected value.
Tensor.item converts a one-element tensor to a Python scalar.
Accepts: array
Accepts: array
Accepts: array
