30 Problems

Cracking RL

From dynamic programming and tabular methods to PPO, DPO, and GRPO. Thirty implement-from-scratch problems covering the math behind every major RL algorithm.

13 Easy14 Medium3 Hard

Unlock the full study plan

Upgrade to Pro to access the full curriculum.

Upgrade to Pro

6 sections

Discounted Returns Bellman Expectation Equation Bellman Optimality Equation Value Iteration Policy Iteration

Monte Carlo Policy Evaluation TD(0) Value Update SARSA Update Q-Learning Update Expected SARSA

Epsilon-Greedy Action Selection UCB1 for Multi-Armed Bandits Thompson Sampling (Beta-Bernoulli)Softmax (Boltzmann) Exploration LinUCB Contextual Bandit

DQN Loss with Target Network Experience Replay Buffer Prioritized Experience Replay Double DQN Target Dueling DQN Decomposition

REINFORCE Gradient REINFORCE with Baseline Generalized Advantage Estimation (GAE)Actor-Critic (A2C) Loss with Entropy PPO Clipped Surrogate Objective

Bradley-Terry Reward Model Loss PPO with Reference-Policy KL DPO Closed-Form Loss GRPO Group-Relative Advantage RLOO (REINFORCE Leave-One-Out)

Summary

Total30

Easy13

Medium14

Hard3

Progress0%