Back to Study Plans

25 Problems

Optimization

Study the optimization algorithms that power ML training. From vanilla gradient descent to Adam, schedulers, and convergence analysis.

6 Easy12 Medium7 Hard

Unlock the full study plan

Upgrade to Pro to access the full curriculum.

6 sections

Minimum of a Univariate Quadratic Minimum of an Axis-Aligned Paraboloid Convexity Certificate via the Hessian Convex Polytope Membership Classify a Critical Point from the Hessian

Vanilla Gradient Descent Batch vs Stochastic vs Mini-Batch GD Classical Momentum Nesterov Momentum Gradient Descent Basics Quiz10 questions

AdaGrad RMSProp Adam Optimizer AdamW (Decoupled Weight Decay)Adaptive Methods Quiz10 questions

Learning Rate Sweep Linear Warmup Cosine Annealing with Warm Restarts LR Range Test and One-Cycle Policy Learning Rate Scheduling Quiz10 questions

Lasso Regression using ISTA Gradient Accumulation Dropout from Scratch Early Stopping Regularization Quiz10 questions

Newton's Method Gradient Clipping L-BFGS Optimizer Showdown Advanced Optimization Quiz10 questions

Summary

Total25

Easy6

Medium12

Hard7

Progress0%

Quizzes0/5