AI Engineer Interview Questions and Answers, Grouped by Topic

Name: TensorTonic
Author: TensorTonic

Thirty-eight AI engineer interview questions on prompting, embeddings, RAG, reranking, evaluation, agents, tool calling and MCP, guardrails, caching and serving cost, each with a short answer and a project, lesson or problem to practise it.

By TensorTonic's founderLast updated November 11, 202630 min read

What AI engineer interview questions should you expect?

AI engineer interview questions ask how you build reliable products on top of large language models, from prompting and structured outputs to embeddings, retrieval, evaluation, agents, tool calling, guardrails, caching and the cost of serving. Interviews for LLM application engineer roles cover the same ground, and most loops add a coding round and a system design round for an LLM application. TensorTonic pairs each of the 38 questions below with a project you can build, a lesson that explains the answer or a coding problem that checks your implementation against tests.

The questions are grouped by topic in the order a design discussion usually moves through them. Each one has a short model answer written in full sentences, which is roughly the length of a good first reply before the interviewer follows up. Questions about the model itself, such as attention, training, fine-tuning methods and alignment, are covered in the LLM interview questions guide, so this guide stays at the level of the application.

What does an AI engineer do, and how is the interview different from an ML engineer interview?

An AI engineer builds products on top of pretrained language models, so the interview tests how you design, evaluate and run those products more than how you train models. Interviewers usually open with the role itself to see whether you understand where the hard problems sit.

What is the difference between an AI engineer and an ML engineer?

An AI engineer usually starts from a pretrained model and builds the system around it, including prompts, retrieval, tools, evaluation and serving, while an ML engineer more often trains and deploys models on the company's own data. The two roles overlap in evaluation, monitoring and latency work. The main difference in interviews is that AI engineer rounds spend more time on application design and failure modes and less on training loops and loss functions. The ML engineer interview preparation guide covers the other side of that comparison.

What does an AI engineer interview loop usually include?

An AI engineer interview loop usually includes a coding round, a system design round for an LLM application and a discussion of a project you built. The coding round often asks for application pieces such as chunking, ranking or parsing model output, and the design round often asks you to design a document assistant or a support agent. Some loops add a take-home where you build a small application and explain how you measured it. Read the ML system design interview practice guide for how design rounds are run and scored.

What prompting and structured output questions do AI engineer interviews ask?

Prompting questions ask how you turn a model into a component with predictable behaviour and output that code can consume. Interviewers look for answers that treat a prompt as something you version and test against examples.

How do you write a system prompt that behaves reliably?

You write a reliable system prompt by stating the task, the inputs the model will receive, the exact output format and what to do when the input does not allow an answer. Concrete rules and a few examples work better than general instructions about tone, and each rule should be one the model can check against the input. You then run the prompt against a fixed set of test inputs after every change, because a small edit can fix one case and break another. The ReAct Agent from Scratch project has you write an agent contract with a strict response format and a grounding rule.

How do you get valid JSON or another structured format from an LLM?

You get valid structured output by asking for a schema, constraining generation when the serving stack supports it, and validating the result in code before anything downstream uses it. Constrained decoding masks out tokens that would break the schema, so the output always parses, although it can still contain wrong values. When the output fails validation, a common pattern is to send the error message back to the model once and ask for a corrected version, and then fail cleanly if that also fails. The MCP Server from Scratch project has you validate model-supplied arguments against a JSON Schema and report each kind of error.

When do few-shot examples and chain of thought help, and what do they cost?

Few-shot examples help when the format or the decision rule is easier to show than to describe, and chain of thought helps on tasks with several steps of reasoning such as arithmetic or multi-step lookups. Both add tokens, since examples lengthen every prompt and reasoning lengthens every output, and output tokens are the slower and usually more expensive part of a call. Examples can also bias the model towards their surface features, so choose them to cover different cases. Read the chain of thought lesson for how step-by-step reasoning changes the output.

What embedding and vector search questions come up in AI engineer interviews?

Embedding questions ask how text is turned into vectors that can be searched by meaning and what the search index gives up for speed. They are the first step of almost every retrieval design question.

What is a text embedding, and why is cosine similarity used to compare embeddings?

A text embedding is a fixed-length vector produced by an encoder model so that texts with similar meanings land close together. Cosine similarity compares the direction of two vectors and ignores their length, which makes the score depend on what the texts are about and less on how long they are. Many embedding models return normalized vectors, and in that case cosine similarity equals the dot product, which is cheaper to compute at scale. Practise with Cosine Similarity, and read the embeddings lesson in ML Design for how embeddings are used in production systems.

How does approximate nearest neighbour search work, and what does it give up?

Approximate nearest neighbour search finds vectors close to a query without comparing the query to every vector in the index. Graph indexes link each vector to its neighbours and walk the graph towards the query, while cluster-based indexes search only the clusters whose centres are closest to the query. Both give up some recall, meaning a few true nearest neighbours are missed, in exchange for much lower latency, and their parameters let you move along that trade-off. Read the retrieval and ranking lesson for where this search sits in a two-stage pipeline.

What happens when you switch to a new embedding model?

When you switch to a new embedding model, you have to re-embed every document, because vectors from two different models live in different spaces and cannot be compared. Queries and documents must always be embedded by the same model. Teams usually build the new index alongside the old one, compare retrieval metrics on a labelled set of queries, and switch traffic only when the new index is better. Practise the comparison metric with Precision and Recall at K.

How do AI engineer interviews test document chunking?

AI engineer interviews test chunking by asking how you split documents so that each retrieved piece holds one complete answer. Chunking is a frequent cause of bad retrieval, so interviewers like to hear how you would detect a chunking problem as well as how you would set it up.

How do you choose a chunk size and overlap?

You choose a chunk size by checking what a typical answer looks like in your documents and making chunks large enough to hold one answer with its context, then confirm the choice with retrieval metrics on real questions. Very small chunks lose the surrounding context, and very large chunks mix several topics into one vector and fill the prompt with irrelevant text. Overlap repeats a little text between neighbouring chunks so that a sentence on a boundary appears whole in at least one of them. Splitting on headings, paragraphs and sentences before falling back to a fixed length keeps chunks closer to natural units. Practise with Text Chunking for RAG.

Why should each chunk carry metadata?

Each chunk should carry metadata such as its source document, its position and a stable identifier, so that the application can cite it, filter by it and trace a bad answer back to the text that caused it. Metadata such as product, date or access level also lets retrieval filter out chunks a user should not see or that are out of date. Without identifiers, you cannot check whether the model's citations point to real retrieved text. The RAG from Scratch project assigns a stable identifier to every chunk and uses it for citations.

What RAG pipeline design questions should AI engineers prepare for?

RAG design questions ask you to put retrieval and generation together into a system that answers from documents and admits when it cannot. The basic definition of RAG is covered in the LLM interview questions guide, so this section focuses on design decisions and failures.

How would you design a RAG assistant for a company knowledge base?

You would design a RAG assistant by first defining the questions it must answer and the documents it may use, then building an ingestion pipeline that cleans, chunks, embeds and indexes those documents with their metadata. At query time, the assistant retrieves candidates with hybrid search, reranks them, places the best few in the prompt with their identifiers and asks the model to answer only from them with citations. You add an evaluation set before launch and logging after it, and you plan how documents are updated and removed so the index stays current. Read Retrieval-Augmented Generation (RAG) Explained and LLM serving and RAG, then build one in the RAG from Scratch project.

What are the common ways a RAG system fails?

A RAG system commonly fails because the answer was never retrieved, because it was retrieved but buried among irrelevant chunks, or because the model ignored or contradicted the retrieved text. Retrieval misses often come from bad chunk boundaries, vocabulary the embedding model does not handle, or stale and duplicated documents. Generation failures include answering from the model's own memory, merging two documents into one wrong claim and citing a chunk that does not support the sentence. Logging the retrieved chunks with every answer lets you sort each failure into one of these groups. The LLM interview questions guide explains how to tell retrieval failures from generation failures.

How should a RAG assistant respond when the documents do not contain the answer?

A RAG assistant should say that it cannot answer from the available documents, because a confident guess is worse than a refusal in most support and internal tools. You instruct the model to abstain with a fixed phrase when the context does not contain the answer, and you can also abstain before calling the model when the best retrieval score is below a threshold. Your evaluation set should include questions that the documents cannot answer, so that you measure abstention as well as accuracy. The final step of the RAG from Scratch project probes the assistant with a question its knowledge base cannot answer.

How do reranking and hybrid search come up in AI engineer interviews?

Reranking and hybrid search come up as the usual answer to the question of how to make a working retrieval system more accurate. Interviewers expect you to know what each stage adds and what it costs in latency.

What does a reranker add on top of vector search?

A reranker scores each retrieved candidate against the query with a more accurate model, usually a cross-encoder that reads the query and the passage together, and reorders the candidates by that score. Vector search embeds the query and the documents separately, which is fast and lets you search millions of vectors, and the reranker spends more compute on a short list to fix its ordering. The usual design retrieves a few dozen candidates and keeps only the top few after reranking, which adds some latency per query. Read the retrieval and ranking lesson for this two-stage pattern.

How do you combine keyword search and vector search results?

You combine keyword and vector results most simply with reciprocal rank fusion, which gives each document the sum of one divided by a constant plus its rank in each list and sorts by that total. Fusing by rank avoids comparing BM25 scores with cosine similarities, which sit on different scales. Hybrid search helps most when queries contain exact terms such as product names, error codes or IDs, which embeddings often match poorly. Practise the keyword side with BM25 Ranking and TF-IDF Vectorizer, and the vector side with Cosine Similarity.

How do you evaluate a RAG system or LLM application in an interview answer?

You evaluate an LLM application with a fixed set of realistic test cases, separate metrics for retrieval and for answers, and a check that automatic scores agree with human judgement. Evaluation is the topic that most often separates strong AI engineer candidates from the rest.

How do you build an evaluation set for an LLM application?

You build an evaluation set from real or realistic user questions, each paired with the expected answer or the facts it must contain and, for RAG, the passages that hold those facts. The set should include easy questions, hard ones, questions the documents cannot answer and inputs that try to break the system. Start with a few dozen cases you wrote and checked by hand, and add every production failure you find as a new case. Read Evaluating generated answers for how to grade outputs that have no single correct string.

Which retrieval metrics would you report for a RAG system?

You would report recall at k, which checks whether the relevant passage appears in the top k results, along with a rank-aware metric such as mean reciprocal rank or NDCG that rewards placing it near the top. Recall at k matters most for RAG because the model can only use what it receives. Precision at k matters when the context window is small or irrelevant chunks distract the model. Practise with Precision and Recall at K and the NDCG problem, and read the ranking metrics lesson.

How do you use an LLM as a judge without trusting it blindly?

You use an LLM judge with a specific rubric, ask it to give a reason before its score, and measure how often it agrees with human ratings on a sample before you rely on it. Judges tend to favour longer answers and the first answer in a pair, so you randomize order and check whether scores track length. You also keep the judge model and prompt fixed between runs, since changing either one changes the scores. Measure judge agreement with Cohen's Kappa and turn pairwise judgements into a ranking with Fit Elo Ratings from Pairwise Votes.

What hallucination and grounding questions do AI engineer interviews ask?

Hallucination questions ask why a model states things that are false and how an application keeps its answers tied to sources. Interviewers want specific mechanisms and ways to measure them.

Why do LLMs hallucinate, and how do you reduce it in an application?

LLMs hallucinate because they are trained to produce likely text, and a fluent but false continuation can be likely when the model lacks the fact or the prompt does not supply it. In an application, you reduce hallucination by retrieving the relevant facts, instructing the model to answer only from them, allowing and rewarding abstention, and keeping the context short and relevant. Lower sampling temperature reduces variation between runs, although it does not stop a model from being confidently wrong. Practise how temperature changes sampling with Temperature, Top-k, and Top-p Sampling.

How do you check that an answer is supported by its citations?

You check citations in two steps, first confirming that every cited identifier belongs to a chunk that was actually retrieved, and then checking that each cited chunk supports the sentence that cites it. The first check is plain code, and the second is usually done with an entailment model or an LLM judge prompted to compare the claim with the passage. Answers with claims that no chunk supports can be flagged, regenerated or shown with a warning. The RAG from Scratch project returns the retrieved source identifiers with every answer so that this check is possible.

What agent and ReAct loop questions come up in AI engineer interviews?

Agent questions ask how a model can take several actions in a row to finish a task and how you keep that loop safe and bounded. Expect to sketch the loop and say exactly when it stops.

What is an LLM agent, and how does the ReAct loop work?

An LLM agent is a loop in which the model decides on an action, the application runs it and returns the result, and the model decides again until it gives a final answer. In the ReAct pattern, each model turn contains a short thought, then either an action with its input or a final answer, and the application appends each tool result to the history as an observation. The application parses every turn, runs only tools from an allowlist and stops at a final answer or a step limit. Build this loop in the ReAct Agent from Scratch project.

How do you stop an agent from looping forever or taking harmful actions?

You stop runaway agents with a hard limit on steps, tokens and time, plus detection of repeated identical actions, and you end the run with a clear failure message when a limit is reached. Harmful actions are limited by giving the agent only the tools it needs, making most tools read-only, and requiring human approval for actions that change data or spend money. Every tool call and its result should be logged so that a bad run can be replayed. The Agentic Systems project sheet groups the agent projects that cover these controls.

How do you evaluate an agent?

You evaluate an agent on the final outcome and on the trajectory, which is the sequence of steps it took to get there. Outcome checks ask whether the task was completed correctly, and trajectory checks ask whether the agent called the right tools with valid arguments, stayed within its step budget and avoided unnecessary or unsafe calls. Because agent runs vary, you run each task several times and report a success rate. The last step of the ReAct Agent from Scratch project evaluates a trajectory for completion, real tool use and a finite step count.

What tool calling and MCP questions should you expect?

Tool calling questions ask how a model asks the application to run code and how the application protects itself from bad requests. MCP questions ask how tools and context are exposed to models through a shared protocol.

How does tool calling work between a model and an application?

Tool calling works by giving the model a list of tools, each with a name, a description and a JSON Schema for its arguments, and letting the model reply with a structured request to call one of them. The application validates the request, runs the tool and sends the result back as a new message, and the model then continues with that result in its context. The model never runs code itself, so the application decides what is allowed. Clear tool names and descriptions matter as much as the schema, because the model chooses tools by reading them. Build a tool catalog in the MCP Server from Scratch project.

What is MCP, and what problem does it solve?

MCP, the Model Context Protocol, is an open protocol that lets an application expose tools, resources and prompts to any compatible AI client through a standard set of JSON-RPC requests. Without a shared protocol, every integration between a client and a service needs its own custom code. With MCP, a server publishes its catalog once and any compatible client can discover the tools, read their schemas and call them. Build the protocol core of a server, from discovery to tool dispatch and resource reads, in the MCP Server from Scratch project.

How should a tool server handle invalid arguments from a model?

A tool server should validate every argument against the tool's schema before running any application code, and return an error that names each missing field, unknown field or wrong type. In MCP, a tool that fails returns a normal result marked as an error so the model can read the message and correct its call, while problems with the request itself, such as an unknown method, are returned as protocol errors. Keeping the two kinds separate lets the model recover from its own mistakes without hiding real protocol bugs. The MCP Server from Scratch project has a step for each of these checks.

How do AI engineer interviews ask about memory and context management?

AI engineer interviews ask about memory and context by giving you a conversation or an agent run that grows longer than the model can read at once. Good answers explain what to keep, what to compress and what to store outside the prompt.

How do you handle conversation history that grows past the context window?

You handle long history by keeping the system prompt and the most recent turns in full, summarizing older turns into a short running summary, and retrieving specific older details only when they become relevant. Dropping the oldest turns is simple but loses facts the user stated early on. Long prompts also cost more and are slower, and models often use information in the middle of a long context less reliably than information near the start or end. Read KV Cache in LLMs: How It Makes Inference Fast for why every extra context token adds memory and compute.

What is the difference between short-term and long-term memory for an agent?

Short-term memory is the current context window, which holds the task, recent steps and tool results, and it disappears when the run ends. Long-term memory is stored outside the model, usually as records or embeddings in a database, and the agent writes to it and retrieves from it with the same tools used for RAG. Long-term memory needs rules for what is saved, how it is updated and when it expires, or old and wrong facts keep coming back into the prompt. Practise the retrieval side with Cosine Similarity and the RAG from Scratch project.

What guardrail and safety questions come up in AI engineer interviews?

Guardrail questions ask how an LLM application stays safe when users, documents and tool results can all contain hostile text. Interviewers look for layered defences, since no single check stops every attack.

What is prompt injection, and how do you defend against it?

Prompt injection is an attack in which text from a user, a document or a tool result contains instructions that try to override the application's own instructions. Direct injection comes from the user, and indirect injection hides inside content the application retrieves, such as a web page or an email. Defences include treating all retrieved and tool content as data, marking it clearly in the prompt, giving the model only the tools and permissions the task needs, and requiring human approval before any action that sends, deletes or pays. Read the adversaries lesson in ML Design for how to plan for users who attack the system.

What guardrails would you put around an LLM application?

You would put checks on the input, on the output and on every action, including input filters for abuse and personal data, output validation against a schema, checks for unsupported claims and policy violations, and permission checks before tool calls. Rate limits and per-user quotas protect against abuse and runaway cost. Each guardrail should have a defined fallback, such as a safe refusal or a hand-off to a person, and its trigger rate should be monitored so that you notice when it blocks too much or too little. Read the fallbacks lesson for how to design what happens when a check fails.

What latency and cost questions should AI engineers prepare for?

Latency and cost questions ask what makes an LLM feature slow or expensive and what you would change first. The internals of prefill, decode and the KV cache are covered in the LLM interview questions guide, so this section stays at the level of the application.

What drives the latency and cost of an LLM call?

The latency and cost of an LLM call are driven mainly by the number of input tokens, the number of output tokens, the size of the model and the time spent waiting in a queue. Input tokens are processed together in one pass and mostly affect the time to the first token, while output tokens are generated one at a time and set the rest of the response time. That is why long answers and long reasoning are usually the largest cost in a chain of calls. Practise the metrics with Compute TTFT, TPOT, ITL, and Token Throughput, and read Latency, throughput, and cost.

How would you reduce the latency of a slow LLM feature?

You would first measure where the time goes, and then stream the response so the user sees output early, shorten the prompt and the output, and run independent calls in parallel instead of one after another. You can route easy requests to a smaller model and keep the large model for hard ones, and cache repeated work. If you serve the model yourself, quantization, batching and speculative decoding make each token cheaper or faster. Read Speculative Decoding in LLMs Explained and LLM Quantization Explained: INT8, INT4, GPTQ and AWQ.

When would you serve an open model yourself instead of calling a hosted API?

You would serve an open model yourself when data cannot leave your infrastructure, when your traffic is large and steady enough that dedicated GPUs cost less than paying per token, or when you need a fine-tuned model or a latency guarantee that a hosted service cannot give. Hosted APIs are usually the better start, because they need no GPU operations and give access to the strongest models. Self-hosting means you own batching, scaling, monitoring and model updates. Build a streaming server with a request queue in the Build an Inference Server project, and read the inference engineering guide.

What caching questions come up in AI engineer interviews?

Caching questions ask which parts of an LLM request can be reused and what can go wrong when they are. Caching is often the largest single saving in a production LLM system.

What can you cache in an LLM application?

You can cache full responses for exact repeated requests, responses for semantically similar requests, embeddings of documents and queries, retrieval results, and the KV cache of a shared prompt prefix. Prefix caching works when many requests start with the same system prompt, examples or documents, so the server can skip reprocessing that prefix, which is why stable content should come first in the prompt. Exact response caches are safe but rarely hit for free-text input. Practise prefix reuse with Find Reusable Prefixes in a KV Trie.

What are the risks of a semantic cache?

The main risk of a semantic cache is returning a stored answer to a question that looks similar but needs a different answer, such as the same question about a different account or date. A threshold that is too loose returns wrong answers, and one that is too strict rarely hits, so you tune it on labelled pairs of similar and different questions. Cached answers also go stale when the underlying documents change and can leak data across users if the cache key ignores who is asking. Practise the similarity score that the threshold is applied to with Cosine Similarity.

How do you choose between prompting, RAG and fine-tuning in an interview?

You choose between prompting, RAG and fine-tuning by starting with prompting, adding retrieval when the model lacks knowledge, and fine-tuning only when behaviour still falls short. Interviewers want the order and the reason for each step.

A team wants to fine-tune a model so it can answer questions about their documents. What would you recommend?

You would usually recommend RAG first, because documents change, answers need citations and fine-tuning is an unreliable way to add new facts. Fine-tuning fits when the problem is behaviour, such as a fixed output format, a house style or a narrow task where prompting has already been tried and measured. Many systems use both, with retrieval for knowledge and a fine-tuned model for format or tone. Read What Is LLM Fine-Tuning?, then train and evaluate an adapter in the Fine-Tune an LLM project.

What observability questions do AI engineer interviews ask?

Observability questions ask how you would know that an LLM application is failing after launch and how you would find the cause. The answer needs both traces of individual requests and metrics across all of them.

What would you log and trace for an LLM application in production?

You would trace every request as a tree of steps, recording the prompt version, the model, the retrieved chunks with their scores, each tool call with its arguments and result, the final output, token counts, latency for each step and any guardrail that fired. That trace lets you replay a bad answer and see which step caused it. Logs that contain user text need access controls and retention limits like any other personal data. Read the logging lesson in ML Design.

How do you monitor answer quality after launch?

You monitor quality by tracking user signals such as thumbs-down, retries and hand-offs to a person, running an automatic judge on a sample of live traffic, and reviewing a small sample by hand every week. You also watch operational metrics such as latency, error rate, refusal rate and cost per request, because a sudden change in any of them often signals a quality problem. Every confirmed failure becomes a new case in the evaluation set, so the next prompt or model change is tested against it. Read the monitoring lesson and the error analysis lesson.

Which projects should you build to prepare for an AI engineer interview?

The projects to build are a retrieval assistant, a tool-using agent, a tool server, an inference server and a small fine-tuning run, because together they cover nearly every topic in this guide. Each one gives you a system you built yourself to talk about in the project discussion, with real failures and the fixes you made. The table lists what you build in each TensorTonic project and the interview topics it prepares you for.

TensorTonic projects to build for an AI engineer interview, what each one builds and the topics it prepares
Project	What you build	Interview topics
RAG from Scratch	Chunking, embeddings, retrieval, grounded context, cited answers and an evaluation of retrieval and abstention	Embeddings, chunking, RAG design, grounding, evaluation
ReAct Agent from Scratch	A tool registry, a parser for model decisions, an agent contract, a bounded loop and a trajectory evaluation	Prompting, agents, tool calling, agent evaluation
MCP Server from Scratch	A tool catalog with JSON Schemas, a resource, argument validation, tool dispatch and a JSON-RPC request handler	Structured outputs, tool calling, MCP, error handling
Build an Inference Server	A server that loads a model once, accepts requests, streams responses and schedules requests through a queue	Latency, streaming, serving cost, self-hosting
Fine-Tune an LLM	A prepared dataset, a QLoRA adapter and a before and after evaluation on held-out examples	Choosing between prompting, RAG and fine-tuning

The Agentic Systems project sheet groups the retrieval, agent and MCP projects in one place, and the LLM Engineering project sheet groups the fine-tuning and inference projects. For advice on presenting a project in an interview, read how to explain an ML project in an interview.

How should you practise AI engineer interview questions?

You should practise AI engineer interview questions by building each part of an LLM application yourself, answering the questions about that part out loud in about two minutes, and then defending your choices in a mock design round. Interviewers usually follow up with a failure case or a cost question, so check that every answer says how you would measure the result and what it would cost.

Solve the retrieval problems first, starting with chunking, BM25, cosine similarity and precision and recall at k, so that you can write each piece from a blank file.
Build the RAG from Scratch project and keep a note of every retrieval miss and wrong answer you see while working on it.
Build the ReAct agent and the MCP server projects so that you can explain tool calling from the model's side and from the server's side.
Read the LLM Internals lessons on RAG, the KV cache, quantization and speculative decoding so that your latency and cost answers rest on how the model actually runs.
Run a timed mock design round for a document assistant or a support agent, and use the ML Design interview framework to structure it.

If serving is a large part of the role, follow the Inference Engineering study plan after the projects, since it goes deeper into the latency and cost questions above.

What mistakes do candidates make in AI engineer interviews?

The most common mistakes in AI engineer interviews are designs with no way to measure quality, answers that blame the model for retrieval problems, and agents with no limits on what they can do. Each one is easy to avoid once you have built and debugged a small application yourself.

Designing a RAG system without an evaluation set leaves you unable to say whether any change helped, so name your test cases and metrics early in every design answer.
Blaming the model for every bad answer hides retrieval problems, so check whether the right passage was retrieved before changing the prompt.
Proposing fine-tuning to add knowledge from documents leads to a weak design, since retrieval handles changing facts and citations better.
Describing an agent without a step limit, an allowlist of tools or an approval step for risky actions suggests you have not run one in production.
Trusting an LLM judge without checking it against human ratings makes your quality numbers hard to defend.
Ignoring prompt injection in a design that reads emails, web pages or documents misses the most common attack on LLM applications.
Talking about latency without separating time to first token from total response time makes it hard to pick the right fix.
Listing tools and frameworks instead of explaining the mechanism behind each step makes answers sound memorized.

The guide to the mistakes that lose ML interviews covers the broader mistakes that apply to every round.

Practice this

Text Chunking for RAG covers the first step of every retrieval pipeline and the chunk-size trade-off interviewers ask about.
BM25 Ranking makes you write the keyword half of hybrid search that embeddings alone often miss.
Precision and Recall at K turns the retrieval evaluation questions into a metric you can compute and explain.

Then put the pieces together in the RAG from Scratch project, or continue with the Agentic Systems project sheet.

How should you prepare for an AI engineer interview?

Prepare for an AI engineer interview by building one small LLM application end to end and then answering the common questions about each part of it out loud. A retrieval assistant with citations, an agent that calls tools and a simple evaluation set cover most of what interviewers ask about. Practise explaining every design choice in terms of quality, latency and cost, and finish with a mock system design round for a document assistant or a support agent.

Do AI engineer interviews include coding questions?

Most AI engineer interviews include at least one coding round, and the tasks are usually closer to application code than to model code. Common tasks are chunking a document, ranking passages with BM25 or cosine similarity, computing recall at k, parsing a tool call from model output and writing a bounded agent loop. Some companies also give a take-home where you build a small retrieval or agent application and explain how you evaluated it.

Do you need to know transformer internals for an AI engineer interview?

You need enough knowledge of transformer internals to reason about context length, token cost, latency and why a model makes things up, and you rarely need to derive training losses. Interviewers expect you to know what tokens are, what the KV cache does, why output tokens are slower than input tokens and what quantization trades away. Research and model training roles go much deeper into attention, training and alignment.

Is an LLM application engineer the same as an AI engineer?

An LLM application engineer and an AI engineer are usually the same role under two job titles, and both build products on top of pretrained language models. Some companies use AI engineer more broadly to include classic ML work, so read the job description for words such as retrieval, agents, evaluation and serving. The interview questions in this guide apply to either title when the role centres on LLM applications.

Which projects help most in an AI engineer interview?

The projects that help most are small applications you built yourself and measured, such as a retrieval assistant that cites its sources, an agent that uses tools within a step limit and a server that exposes tools through a structured protocol. Interviewers ask what broke, how you found out and what you changed, so keep notes on your failures and on the numbers before and after each fix. A project with an evaluation set is worth more than a larger project with none.