TensorTonic
18 Problems

Build AlphaGo from Scratch

18 NumPy and PyTorch problems covering Go rules, a policy-value network, neural-guided MCTS, self-play, and training.

5 Easy8 Medium5 Hard
5 sections
Find a Go Group and Its LibertiesEasyScore a Terminal Position with Tromp-Taylor RulesMediumBuild the Legal Go Action MaskHardApply a Go Action with Captures and RepetitionHardGo Rules Quiz10 questions
Encode AlphaGo Zero Board HistoryMediumApply an AlphaGo Residual BlockMediumCompute the Policy and Value HeadsMediumAssemble a Compact Dual-Head NetworkHardPolicy-Value Network Quiz10 questions
Initialize Legal MCTS Edge StatisticsEasySelect an Action with PUCTEasyBack Up an MCTS Simulation PathMediumNeural-Guided MCTS Quiz10 questions
Mix Dirichlet Noise into Root PriorsEasyTurn Root Visits into a Search PolicyMediumImplement the AlphaGo Policy-Value LossMediumSelf-Play and Training Targets Quiz10 questions
Sample a Recent Self-Play Replay BatchEasyApply a Dihedral Go SymmetryMediumApply One Policy-Value Head Training StepHardRun Deterministic Neural-Guided MCTSHardIntegrated AlphaGo Zero Pipeline Quiz10 questions

What you will build

You build the core of an AlphaGo Zero style system in 18 NumPy and PyTorch problems. It starts with the rules of Go, adds a neural network that predicts good moves and who is winning, uses that network to guide Monte Carlo tree search, and turns self-play games into training data.

Prerequisites

You should know Python and NumPy, and be comfortable building small networks in PyTorch. The first section teaches the rules of Go you need, through code. If you are new to reinforcement learning, the reinforcement learning study plan covers the fundamentals.

The 18 problems, in order

  1. Go rules (4 problems): find groups and liberties, score a finished game with Tromp-Taylor rules, build the legal move mask and apply a move with captures.
  2. Policy-value network (4): encode the board history, apply a residual block, compute the policy and value heads, and assemble the dual-head network.
  3. Neural-guided MCTS (3): initialize edge statistics, select moves with PUCT and back up a simulation.
  4. Self-play and training targets (3): add Dirichlet noise at the root, turn visit counts into a search policy and implement the policy-value loss.
  5. Integrated pipeline (4): sample a replay batch, apply board symmetries, run a training step and run deterministic neural-guided search.

What you will be able to do

By the end you can explain and implement each part of the AlphaGo Zero training loop: how search improves on the raw network, how those search results become training targets, and how the network improves from them. Each section ends with a quiz. For other systems built from first principles, try Language Modeling from Scratch or Build Micrograd from Scratch.