TensorTonic
Problems

Linear Warmup

Train with a learning rate that increases linearly during warmup and remains constant for later gradient steps.