How to Build an AI Agent from Scratch, from the ReAct Loop to MCP

How do you build an AI agent from scratch?

You build an AI agent from scratch by putting a language model inside a loop that parses each reply, runs the tool the model asks for, feeds the result back, and stops when the task is done or a limit is reached. Around that loop you add tool schemas, safe execution, memory, retrieval, evaluation and a sandbox. TensorTonic has three projects that build the core of this in Python, a ReAct agent, an MCP server and a RAG pipeline, and this guide pairs each stage with a page to read and a project or problem to practise.

The agent you end up with is small, and it uses the same parts as the agents in production coding tools and support systems. Those systems add more tools, better models and more infrastructure, while the loop, the tool contract and the safety checks stay the same. Building the small version shows you where agents fail, which is the knowledge you need to build a large one.

What is an AI agent?

An AI agent is a language model running in a loop with tools, where the model chooses an action, your code runs it, and the result goes back to the model before it chooses again. The model only produces text or a structured tool call. Your application decides which tools exist, checks every request, runs the tool and keeps the record of what happened.

That split of responsibility is the most important idea in agent design. The model suggests actions and the application decides which ones run, so any rule that must always hold, such as a permission check or a spending limit, belongs in code outside the model. Instructions in the prompt make good behaviour more likely, and checks in code are what make it certain.

What do you need to know before building an AI agent?

You need comfortable Python, a working knowledge of JSON and a rough idea of how a language model generates text before you build an AI agent. You do not need to train a model or own a GPU, since the agent calls a model that already exists. Some regular expressions and basic statistics help in the parsing and evaluation stages.

• For Python, you should be able to write functions, dictionaries and loops without looking things up, and this checklist of the Python you need for ML lets you test yourself.
• For JSON, you should know how objects, arrays and types map onto Python dictionaries, lists and values, because tool arguments and protocol messages are JSON.
• For language models, you should know that a model predicts one token at a time from everything in its context, which the lesson on next-token prediction explains.
• For evaluation, you should be comfortable with a success rate and a confidence interval, which the confidence interval lesson covers.

What are the steps to build an LLM agent from scratch?

The steps follow the life of one agent run, from the first model call through the loop, the tools and the stopping rule, and then outward to memory, retrieval, a protocol server, evaluation, safety and cost. The first five stages give you a working agent, and the last six make it useful and safe to run. Each stage below has a short explanation and a table that pairs every topic with a page to read and a project or problem to practise.

Stage 1: How do you call a model and parse its output?

You call the model with a system prompt that defines a strict reply format, then parse the reply into fields that your code can act on. In the ReAct format the model writes lines labelled Thought, Action and Action Input when it wants a tool, or a line labelled Final Answer when it is done. The parser reads each label and returns an empty value when one is missing, so the loop can tell a tool call from a final answer without guessing.

Set the temperature to zero or close to it while you build, because the same prompt then gives the same reply and bugs become reproducible. Many model APIs also offer native tool calling, where the model returns a structured call with JSON arguments, and it is worth switching to once you understand what the text parser was doing for you.

Stage 2: How does the ReAct loop work?

The ReAct loop alternates a reasoning step, an action and an observation until the model gives a final answer. The model writes a short thought about what to do next and names a tool, your code runs that tool, and the tool result is appended to the scratchpad as an observation. On the next call the model sees its earlier thoughts and every observation, so it can choose a second tool or finish.

The name comes from the 2022 paper ReAct: Synergizing Reasoning and Acting in Language Models, which compared interleaved reasoning and tool use with reasoning alone and with acting alone on several benchmarks. The chain of thought lesson explains why writing intermediate steps helps a model, and why those written steps still need checking.

Stage 3: How do you define tools with schemas?

You define each tool with a stable name, a description of what it does and when to use it, and a JSON Schema that lists its arguments, their types and which ones are required. The model reads these definitions before it decides which tool to call, so the description is effectively part of your prompt. A vague description leads to the wrong tool being chosen, and a loose schema lets malformed arguments through.

Keep tools narrow and start with read-only ones. A tool that looks up one order by its ID is easy to describe, easy to validate and hard to misuse, while a tool that runs any database query is none of those things.

Stage 4: How do you execute tool calls safely?

You execute tool calls safely by looking up the requested name in an explicit registry, validating the arguments against the schema, running only the matching function, and returning any error to the model as an observation. A name that is not in the registry never runs, whatever the model wrote. Missing fields, unknown fields and values of the wrong type are reported back so the model can correct the call on its next step.

Add a timeout to every tool and an approval step to any tool that changes something outside the agent, such as sending a message or writing to a database. The approval check runs in your code before the tool, which means a persuaded model still cannot take the action on its own.

Stage 5: When should an agent stop?

An agent should stop when the model gives a final answer, when it reaches a fixed step limit, or when it exceeds a budget for time, tokens or tool calls. Without a step limit a confused agent can repeat the same failing tool call until something outside it breaks. Every run should return a status alongside the answer, such as completed or step limit reached, so the caller knows whether to trust the result.

Detecting repetition is a useful extra rule. If the agent asks for the same tool with the same arguments twice in a row, it is usually stuck, and ending the run early with a clear status costs less than letting it use up the remaining steps.

Stage 6: How do you manage memory and context?

You manage memory by keeping a scratchpad of the current run, a trace of every step for debugging, and a rule for what to drop or summarize when the context grows too long. The scratchpad is what the model sees on each call, while the trace is what you read afterwards, and they do not need to be the same. A common rule keeps the system prompt, the task and the latest few steps in full and replaces older steps with a short summary.

Memory that lasts across runs is usually a search problem. The agent writes facts or past results to a store, and a later run retrieves the most relevant ones by embedding similarity, which is the same machinery as retrieval in the next stage.

Stage 7: How do you give an agent retrieval as a tool?

You give an agent retrieval by wrapping a search over your documents in a tool that takes a query and returns the most relevant passages with their sources. Behind the tool, the documents are split into chunks, each chunk is indexed by keywords, by embeddings or by both, and a query returns the top matches. The agent decides when to search and what to search for, which handles questions that need several lookups better than a single retrieval step does.

The RAG project builds this pipeline from loading documents to answering with citations and checking whether the system correctly declines when the evidence is missing. Our guide on building a RAG system from scratch covers each part in more depth.

Stage 8: How do you expose tools through an MCP server?

You expose tools through an MCP server by writing a handler that accepts JSON-RPC 2.0 requests, lists your tools and resources with their schemas, validates each call and returns either a result or a structured error. MCP stands for Model Context Protocol, an open protocol introduced in 2024 that lets any compatible AI client discover and use the same server. Each response carries the id of the request it answers, so a client can match replies to calls.

An MCP server separates two kinds of failure. A protocol error, such as an unknown method or a malformed request, is a JSON-RPC error response, while a tool that runs and fails returns a normal result marked as an error, so the model can read what went wrong and try again. Keeping the handler independent of the network transport means you can test the whole server by passing it dictionaries.

Stage 9: How do you evaluate and trace an agent?

You evaluate an agent by running it on a fixed set of tasks and checking both the final answer and the trajectory, which is the sequence of tool calls, arguments and observations that produced it. A run that reaches the right answer through the wrong tools will often fail on a slightly different task. Trajectory checks ask whether the run completed, used a registered tool, called tools in a sensible order and stayed within its step budget.

Test the loop with scripted model replies before you test it with a live model. A fake model that returns a fixed tool call and then a fixed answer makes the loop deterministic, so a failing test points at your code. Once that passes, run the live model on the task set, record a trace of every step, and report the success rate with a confidence interval, since small task sets give noisy numbers.

Stage 10: How do you make an agent safe to run?

You make an agent safe by assuming that any text it reads may contain instructions from an attacker and by enforcing every important rule in code outside the model. This threat is called prompt injection, and it can arrive directly from a user or indirectly through a web page, an email or a retrieved document. Instructions and data share the same token stream, so no prompt can fully separate them.

• Give each tool the least privilege it needs and deny everything else by default.
• Validate tool arguments on the trusted side and use parameterized queries instead of building commands from model text.
• Keep secrets such as API keys out of the model context, and let your code attach credentials when a tool runs.
• Require human approval for actions that send data out or change state, and show the approver the exact action.
• Run model-written code in a fresh sandbox with no network access, limited time and memory, and no host files.
• Log every tool call with its arguments and result so you can investigate what happened.

The most dangerous combination is an agent that can read private data, read untrusted content and send data out. Removing any one of those three abilities from a given task closes the easiest path for an attacker to steal data.

Stage 11: How do you control cost and latency?

You control cost and latency by limiting the number of steps, keeping the context short, reusing a stable prompt prefix and measuring the latency of every run. Each step is a full model call, and the loop resends the growing scratchpad each time, so the total input tokens of a run grow roughly with the square of the number of steps. Trimming old observations and returning short tool results keeps that growth in check.

Put the system prompt and tool definitions first and keep them identical between calls, because many serving systems can reuse the computation for a repeated prefix. Report latency as percentiles instead of an average, since a few long runs with many steps dominate what users notice.

Can you build an AI agent without a GPU?

Yes, you can build an AI agent without a GPU, because the agent calls a model hosted somewhere else and everything you write runs on an ordinary laptop. The parser, the tool registry, the loop, the MCP handler and the evaluation code are plain Python. You can also test the whole loop with a scripted fake model, which needs no model access at all.

A GPU matters only if you want to run or fine-tune the model yourself. Our guide on practising ML without a GPU covers what you can do on a laptop and where to find compute when you need it.

How do you put the pieces together into a working agent?

You put the pieces together by building the parser and the tools first, checking the loop with scripted replies, and only then running it against a live model and evaluating the trajectory. The capstone of the ReAct agent project follows this order in a notebook, where the agent investigates a support incident with two read-only tools, one for service status and one for runbook search.

1. Configure the model and write down the task the agent has to solve.
2. Register the tools in an explicit registry, starting with read-only ones.
3. Write the parser and test it on a tool call and on a final answer.
4. Write the system prompt with the reply format, the tool descriptions and a rule against inventing facts.
5. Build the loop with a step limit and check it with scripted replies that make no model request.
6. Run the agent with the live model and print every step of the trace.
7. Evaluate the run for completion, real tool use, the step budget and a non-empty answer.

The capstone of the MCP server project does the same for the protocol side. It publishes a support service with a document search tool, an order status tool and a policy resource, then sends a sequence of client requests through one handler and checks that request ids are preserved and bad arguments are rejected.

What should you build after your first agent?

After your first agent, the useful next steps are grounding it in your own documents, adapting the model to your task, serving the model yourself and training agents with rewards. Each direction has an explanation in the LLM Internals lessons and a project where you build it. The Agentic Systems project sheet lists the agent projects in the order to take them.

Frameworks such as LangChain wrap the same loop, parser and tool registry you have written, and the RAG project ends by mapping each function you built onto one. If you are preparing for interviews, our guide to AI engineer interview questions covers the agent, retrieval and evaluation questions you are likely to be asked.

What mistakes do people make when building AI agents?

• Running whatever tool name the model writes, instead of looking it up in an allowlist, lets a hallucinated or injected name reach your code.
• Raising an exception on a bad tool call ends the run, when returning the error as an observation would let the model correct itself.
• Leaving out the step limit lets a confused agent loop until a timeout or a bill stops it.
• Parsing replies with a loose pattern and no check lets a missing label pass silently as an empty action.
• Putting API keys or passwords in the prompt means a single successful injection can leak them.
• Adding tools that write or send data before the read-only version works and has an approval step makes every bug far more costly.
• Judging the agent only by its final answer hides runs that reached the right answer through the wrong tools.
• Testing only against a live model makes failures hard to reproduce, so check the loop with scripted replies first.
• Writing short, vague tool descriptions makes the model pick the wrong tool, because the description is all it knows about the tool.

Practice this

Then practise the retrieval pieces on their own with Text Chunking for RAG and BM25 Ranking, or follow the Agentic Systems project sheet from start to finish.

Related guides

• How to build a RAG system from scratch
• AI engineer interview questions
• How to build an LLM from scratch, one component at a time
• LLM interview questions: 43 questions with answers
• Inference engineering: what it is and how to learn it
• How to build a machine learning portfolio that gets interviews