How to Turn ML Theory Into Code When You Understand the Math
Go from ML theory to working code with a step-by-step method, a worked softmax example you can run, and a four-week plan of free problems that test your code.
How do you turn ML theory into code when you already understand the math?
To turn ML theory into code, pick one concept, write out its formula with the shape of every symbol, implement it in NumPy without a library call for the core step, and check the result against tests. TensorTonic is built around this loop, with 1,400+ machine learning coding problems where you implement algorithms from scratch in the browser and every submission is checked by tests.
Repeat the loop for one algorithm at a time, starting with small ones like softmax and mean squared error, and move to training loops and optimizers once those feel routine. The rest of this guide shows the method step by step, a worked example and a four-week plan of free problems.
Why can you understand ML theory but struggle to code it?
Understanding the theory means you know what an algorithm does, while coding it means deciding every detail the formula leaves out. A formula like softmax says nothing about which axis to sum over, what shape the input has, how to handle a batch or what happens when a value is 1000. Courses and textbooks rarely make you decide those details, so the first time you meet them is when your code gives the wrong answer.
- Shapes are the most common gap, because a formula written for one vector has to work on a batch of vectors in code.
- Numerical stability is the second, because values that are fine on paper can overflow or underflow in floating point.
- The order of operations is the third, since training loops need the forward pass, the loss, the gradients and the update to happen in a precise sequence.
- Without tests, you cannot tell whether the code is wrong or your understanding is, so mistakes stay hidden.
How do you turn a formula into code, step by step?
Turn a formula into code by moving from the equation to a hand-worked example, then to a slow loop, then to vectorized NumPy, and testing at every stage. Each step catches a different kind of mistake, so skipping one usually means a bug that is harder to find later. The table uses softmax as the running example.
| Step | What you do | Softmax example |
|---|---|---|
| 1. Write the formula | Copy the equation and name every symbol, with its shape next to it. | z has shape (n,), and the output has shape (n,) and sums to 1. |
| 2. Work a tiny example by hand | Pick two or three numbers and compute the answer on paper. | For z = [0, 0], each output is 0.5. |
| 3. Write a plain loop | Translate the formula line by line, even if it is slow. | Loop over z, exponentiate each value, then divide by the total. |
| 4. Vectorize with NumPy | Replace the loop with array operations once the loop gives the right answer. | Use np.exp on the whole array and sum along the right axis. |
| 5. Test against your example | Check that the code reproduces the answer you computed by hand. | softmax([0, 0]) should return [0.5, 0.5]. |
| 6. Test the edge cases | Try very large, very small and repeated values, and batched inputs. | Large logits like 1000 expose the overflow bug shown below. |
What does turning theory into code look like for softmax?
Softmax turns a vector of scores into probabilities by exponentiating each score and dividing by the sum of the exponentials. The direct translation of that formula works on small numbers, which is why it looks correct at first. Running it on large scores shows the problem.
import numpy as np
def softmax(z):
e = np.exp(z)
return e / e.sum()
print(softmax(np.array([1000.0, 1001.0])))This prints [nan nan], because np.exp(1000) is larger than the biggest number a 64-bit float can hold, so it becomes infinity, and infinity divided by infinity is not a number. The math is correct and the code is still wrong, which is exactly the gap between theory and implementation.
The fix is to subtract the largest score before exponentiating. Shifting every score by the same constant multiplies the numerator and the denominator by the same factor, so the probabilities stay the same, and the largest exponent becomes exp(0), which is 1. Summing along the last axis with keepdims=True makes the same function work for a single vector and for a batch.
import numpy as np
def softmax(z):
shifted = z - np.max(z, axis=-1, keepdims=True)
e = np.exp(shifted)
return e / e.sum(axis=-1, keepdims=True)
print(softmax(np.array([1000.0, 1001.0])))This prints about [0.269 0.731], which matches the hand calculation for scores that differ by 1. The Softmax Function problem checks this implementation against tests for batches and extreme values, and the cross-entropy lesson explains where softmax outputs go next.
Where can you practise ML coding when you have more theory than code?
The best place to practise ML coding when you have more theory than code is a platform where you implement algorithms from scratch and tests check every submission. TensorTonic has 1,400+ such problems, 700+ of them free, covering ML fundamentals, deep learning, CUDA kernels and research paper implementations, with the theory behind each problem and interactive visualizations.
Other resources can complement that practice.
- Deep-ML is a free platform with machine learning coding problems that you solve in the browser.
- Kaggle Learn offers short free courses with exercises in hosted notebooks.
- fast.ai offers a free deep learning course that starts from working code.
The guide to ML practice platforms covers more options by goal, and the Deep-ML and TensorTonic comparison helps if you are choosing between those two.
What is a good four-week plan to go from theory to code?
A good four-week plan moves from single formulas to full training loops, with three free problems a week at about five hours a week. Each week adds one new coding habit, such as checking that a loss decreases or handling a batch dimension, so by the end you can take an unfamiliar algorithm and implement it with the same method.
| Week | Topic | Problems | Skill you build |
|---|---|---|---|
| Week 1 | Vectors, losses and activations | Dot Product, Mean Squared Error, Sigmoid in NumPy | Turning a sum or an elementwise formula into one line of NumPy |
| Week 2 | Linear models and gradient descent | Linear Regression Closed Form, Gradient Descent for a 1D Quadratic, Logistic Regression Training Loop | Writing an update rule and checking that the loss goes down |
| Week 3 | Neural network building blocks | ReLU Activation, Softmax Function, Cross-Entropy Loss | Handling axes, batches and numerical stability |
| Week 4 | Optimizers and clustering | Adam Optimizer Step, K-Means Assignment Step, K-Means Centroid Update | Keeping state between steps and using broadcasting for distances |
If the linear algebra in week 2 feels shaky, the guide on how to learn linear algebra for machine learning with coding pairs each concept with a lesson and a NumPy problem.
What are the common mistakes when turning ML theory into code?
- Writing the vectorized version first makes bugs hard to find, so write a loop that matches the formula and vectorize it afterwards.
- Testing only on random inputs hides mistakes, because a small example with a known answer is the only way to know the output is right.
- Summing over the wrong axis gives outputs with the right shape and the wrong values, so print the shape after every reduction.
- Ignoring numerical stability works until real data arrives, so always try very large and very small values.
- Copying a library call skips the learning, since the goal is to know what the library does before you rely on it.
Practice this
- Implement Softmax Function is the worked example on this page, now with tests for batches and extreme values.
- Linear Regression Closed Form turns the normal equation into a few lines of matrix code.
- Logistic Regression Training Loop combines a model, a loss and gradient descent in one implementation.
- Implement Adam Optimizer Step shows how a formula with several moving parts becomes code that keeps state.
Then browse the full list of machine learning coding problems by topic and difficulty.
Why can I explain an ML algorithm but not code it?
Explaining an algorithm uses the idea, while coding it forces every detail into the open: the shape of each array, the axis of each sum, the order of the steps and what happens with extreme values. Most people who know the theory have never had to decide those details, so the gap closes once you practise making them one algorithm at a time.
Should I implement ML algorithms from scratch or use libraries?
Implement each core algorithm from scratch once, then use the library. Writing softmax, linear regression or an optimizer step yourself shows exactly what the library call does, and comparing your output with the library is a quick way to check your code. In real projects, use the tested library version.
How much Python do I need to turn ML theory into code?
You need functions, loops, lists and basic NumPy arrays, including indexing, broadcasting and sums along an axis. The guide on how much Python you need for machine learning lists the exact skills and has a short self-test.
How long does it take to get comfortable coding ML algorithms?
At about five hours a week, the four-week plan on this page covers twelve core algorithms.
Can I practise turning ML theory into code for free?
Yes. Every problem in the four-week plan on this page is free on TensorTonic, along with 700+ other problems. You write your code in the browser and every submission is checked by tests, so you know straight away whether your implementation is right.