From Pieces to an Environment
Introduction
Project introduction
From Pieces to an Environment
You have written every piece an environment needs: moving the agent, scoring a step, deciding whether the episode ended, the target a learning agent moves toward, a safe progress bonus, and an honest summary of many runs.
In the notebook these pieces become one environment with reset and step. You will test it with scripted routes before any training, then train a learning agent on it three times with three different rewards.
What you will see
With a plain reward, the agent learns the shortest route. With a bonus for getting closer, it learns to step closer and back again forever and never reaches the goal, while scoring more reward than the agent that does. You will explain why with simple arithmetic.
Then you will replace that bonus with the safe version and watch the agent find the shortest route again, and you will see what goes wrong when a time-out is mistaken for a real ending.
One episode, step by step
5 × 5 gridThis step
How the notebook is organized
01
Build and test
Assemble the environment and check it with routes whose outcome you already know.
02
Break and fix
Train with a reward that gets gamed, explain the loop, and fix the reward.
03
Judge honestly
Compare all three rewards across ten random seeds.
Build and break the environment
Run the whole investigation in the browser, about 45 minutes.