Construct a deterministic GPipe schedule with equal-duration stage operations and no overlap between the forward and backward waves. Let p be num_stages and m be num_microbatches. Each wave lasts T time slots:
T=m+p−1.In forward time slot t, stage s processes microbatch iforward:
iforward=t−s.When include_backward is true, the backward wave follows the completed forward wave. At backward offset u, stage s processes microbatch ibackward:
ibackward=m−1−(u−(p−1−s)).All indices start at zero: s ranges from 0 through p−1, while t and u range from 0 through T−1. A microbatch index outside 0 through m−1 means that stage is idle. Label work F0, F1, and so on for forward, or B0, B1, and so on for backward; use None for idle cells. With q=2 when backward is included and q=1 otherwise, useful and total stage-slot counts are:
Cuseful=qpm. Ctotal=qTp.Return a Python dictionary with schedule, a time-ordered list of stage-ordered lists containing strings or None; bubble_slots and total_slots, Python integer cell counts; and utilization, a Python float equal to useful cells divided by total cells.
Input: num_stages = 2, num_microbatches = 3, include_backward = false
Output: {"schedule":[["F0",null],["F1","F0"],["F2","F1"],[null,"F2"]],"bubble_slots":2,"total_slots":8,"utilization":0.75}
Explanation: Six forward computations occupy eight stage-slot cells, leaving two bubbles.
Input: num_stages = 2, num_microbatches = 2, include_backward = true
Output: {"schedule":[["F0",null],["F1","F0"],[null,"F1"],[null,"B1"],["B1","B0"],["B0",null]],"bubble_slots":4,"total_slots":12,"utilization":0.666667}
Input: num_stages = 1, num_microbatches = 3, include_backward = false
Output: {"schedule":[["F0"],["F1"],["F2"]],"bubble_slots":0,"total_slots":3,"utilization":1}
range supplies zero-based slot and stage indices.
An f-string can combine a phase letter with a microbatch index.
sum can count Boolean conditions across nested lists.
Accepts: number
Accepts: number
Construct a deterministic GPipe schedule with equal-duration stage operations and no overlap between the forward and backward waves. Let p be num_stages and m be num_microbatches. Each wave lasts T time slots:
T=m+p−1.In forward time slot t, stage s processes microbatch iforward:
iforward=t−s.When include_backward is true, the backward wave follows the completed forward wave. At backward offset u, stage s processes microbatch ibackward:
ibackward=m−1−(u−(p−1−s)).All indices start at zero: s ranges from 0 through p−1, while t and u range from 0 through T−1. A microbatch index outside 0 through m−1 means that stage is idle. Label work F0, F1, and so on for forward, or B0, B1, and so on for backward; use None for idle cells. With q=2 when backward is included and q=1 otherwise, useful and total stage-slot counts are:
Cuseful=qpm. Ctotal=qTp.Return a Python dictionary with schedule, a time-ordered list of stage-ordered lists containing strings or None; bubble_slots and total_slots, Python integer cell counts; and utilization, a Python float equal to useful cells divided by total cells.
Input: num_stages = 2, num_microbatches = 3, include_backward = false
Output: {"schedule":[["F0",null],["F1","F0"],["F2","F1"],[null,"F2"]],"bubble_slots":2,"total_slots":8,"utilization":0.75}
Explanation: Six forward computations occupy eight stage-slot cells, leaving two bubbles.
Input: num_stages = 2, num_microbatches = 2, include_backward = true
Output: {"schedule":[["F0",null],["F1","F0"],[null,"F1"],[null,"B1"],["B1","B0"],["B0",null]],"bubble_slots":4,"total_slots":12,"utilization":0.666667}
Input: num_stages = 1, num_microbatches = 3, include_backward = false
Output: {"schedule":[["F0"],["F1"],["F2"]],"bubble_slots":0,"total_slots":3,"utilization":1}
range supplies zero-based slot and stage indices.
An f-string can combine a phase letter with a microbatch index.
sum can count Boolean conditions across nested lists.
Accepts: number
Accepts: number