TensorTonic
Problems

Cosine Annealing with Warm Restarts

Train with cosine-annealed learning rates that periodically restart at the configured maximum value.