Implement a learning-rate schedule with linear warmup followed by linear decay. The step index is zero-based, and the rate remains at the final value after training ends.
LR(t)=⎩⎨⎧η0Wt,η0+T−Wt−W(ηf−η0),ηf,W>0 and t<WW≤t<Tt≥THere, t is step, W is warmup_steps, T is total_steps, η0 is initial_lr, and ηf is final_lr. When total_steps is zero, return final_lr. Return the learning rate as a Python float.
Input: step = 0, total_steps = 100, initial_lr = 0.001, final_lr = 0.0, warmup_steps = 10
Output: 0.0
Explanation: The first warmup step uses zero percent of the initial learning rate.
Input: step = 10, total_steps = 100, initial_lr = 0.001, final_lr = 0.0, warmup_steps = 10
Output: 0.001
Input: step = 50, total_steps = 100, initial_lr = 0.001, final_lr = 0.0, warmup_steps = 10
Output: 0.000556
Handle warmup, decay, and completed training as separate branches.
For decay, use (step - warmup_steps) / (total_steps - warmup_steps) as the interpolation fraction.
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Implement a learning-rate schedule with linear warmup followed by linear decay. The step index is zero-based, and the rate remains at the final value after training ends.
LR(t)=⎩⎨⎧η0Wt,η0+T−Wt−W(ηf−η0),ηf,W>0 and t<WW≤t<Tt≥THere, t is step, W is warmup_steps, T is total_steps, η0 is initial_lr, and ηf is final_lr. When total_steps is zero, return final_lr. Return the learning rate as a Python float.
Input: step = 0, total_steps = 100, initial_lr = 0.001, final_lr = 0.0, warmup_steps = 10
Output: 0.0
Explanation: The first warmup step uses zero percent of the initial learning rate.
Input: step = 10, total_steps = 100, initial_lr = 0.001, final_lr = 0.0, warmup_steps = 10
Output: 0.001
Input: step = 50, total_steps = 100, initial_lr = 0.001, final_lr = 0.0, warmup_steps = 10
Output: 0.000556
Handle warmup, decay, and completed training as separate branches.
For decay, use (step - warmup_steps) / (total_steps - warmup_steps) as the interpolation fraction.
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Accepts: number