Study Plans Labs Projects Interview Pricing Feedback

Implement ML Papers From Scratch

Each track breaks one landmark paper into small steps. You implement every step in Python in the browser, and tests check your code before you move on. If you are new, start with the Transformer, or read which ML papers to implement from scratch to choose a track.

Transformers and LLMs

Convolutional networks

Sequence models and embeddings

Generative models

Attention Is All You Need

Attention Is All You Need badge

Tokenization

Convert text to token IDs

Embedding Layer

Map tokens to vectors

Positional Encoding

Inject position via sinusoids

Scaled Dot-Product Attention

Core QKV mechanism

Multi-Head Attention

Parallel attention heads

Feed-Forward Network

Position-wise MLP

Layer Normalization

Normalize activations

Encoder Block

Complete encoder layer

End-to-endAssemble and train a TransformerOpen

Loading architecture visualization...