Reinforcement Learning Interview Questions and How to Answer Them
The reinforcement learning interview questions that come up most, grouped by topic from MDPs and the Bellman equation to Q-learning, DQN, policy gradients and PPO, each with a short model answer and a coding problem to practise it.
What reinforcement learning interview questions should you expect?
Reinforcement learning interview questions test whether you can explain how an agent learns from rewards and why each algorithm is built the way it is, across MDPs and the Bellman equations, dynamic programming, Monte Carlo and TD learning, Q-learning and SARSA, exploration, DQN, policy gradients, actor-critic methods and PPO. Most are spoken questions that the interviewer follows with a why or a what if, and some ask you to write an update rule or a loss from scratch. TensorTonic has a tested coding problem for the questions in this guide, and its Cracking Reinforcement Learning study plan collects 30 reinforcement learning problems in six sections, from discounted returns to PPO, DPO and GRPO.
This guide groups the questions by topic and gives a short model answer for each, followed by a problem to solve or a lesson to read. Questions about RLHF, DPO and GRPO for language models are covered in more depth in LLM interview questions, and the deep learning background that DQN and PPO rely on is in deep learning interview questions.
What MDP and Bellman equation questions do interviewers ask?
Interviewers ask what a Markov decision process is, why rewards are discounted and how the Bellman equations relate the value of a state to the values of the states after it. These questions come first because every algorithm later in the interview is a way of solving or approximating the Bellman equations.
What is a Markov decision process?
A Markov decision process is a model of sequential decision making made of a set of states, a set of actions, a transition probability p(s′ | s, a), a reward function and a discount factor gamma. It is Markov because the next state and reward depend only on the current state and action, and not on the earlier history. The goal is a policy, a rule for choosing actions in each state, that maximizes the expected discounted return.
The Markov chains lesson covers the Markov property and transition matrices that an MDP builds on.
What is the discounted return, and why discount future rewards?
The discounted return from time t is G_t = r_{t+1} + gamma r_{t+2} + gamma squared r_{t+3} and so on, with a discount factor gamma between 0 and 1. Discounting keeps the return finite in tasks that never end, makes rewards that arrive sooner worth more, and reduces the variance of long-horizon estimates. A gamma close to 0 gives a short-sighted agent, and a gamma close to 1 makes the agent weigh distant rewards almost as much as immediate ones.
Compute returns for whole episodes in the Discounted Returns problem.
What does the Bellman expectation equation say?
The Bellman expectation equation says that the value of a state under a policy pi equals the expected immediate reward plus gamma times the value of the next state, averaged over the actions pi takes and the transitions that follow. Written out, V^pi(s) is the sum over actions of pi(a | s) times the sum over next states and rewards of p(s′, r | s, a) times (r + gamma V^pi(s′)). It gives one linear equation per state, so for a small known MDP you can solve for V^pi exactly.
Apply one full backup in the Bellman Expectation Equation problem.
How is the Bellman optimality equation different?
The Bellman optimality equation replaces the average over the policy with a maximum over actions, so V*(s) is the maximum over a of the expected value of r + gamma V*(s′). The maximum makes the equation nonlinear, so it is usually solved by iteration instead of a linear solve. Once you know Q*, the optimal policy is simply to take the action with the largest Q* value in each state.
Write the optimal backup in the Bellman Optimality Equation problem.
What is the difference between value-based and policy-based methods?
Value-based methods learn how good each action is and act greedily on those estimates, while policy-based methods learn the policy directly as a parameterized distribution over actions. Interviewers ask this early to see whether you can place each later algorithm in the right family.
What is the difference between the state-value function V and the action-value function Q?
The state-value function V^pi(s) is the expected return from state s when you follow pi, and the action-value function Q^pi(s, a) is the expected return when you first take action a in state s and follow pi afterwards. V^pi(s) is the average of Q^pi(s, a) over the actions pi would take. Q is more useful for control without a model, because you can pick the best action by comparing Q values without knowing the transition probabilities.
The Bellman Expectation Equation problem shows how the two functions are tied together.
When would you choose a policy-based method over a value-based one?
You would choose a policy-based method when the action space is continuous or very large, because taking a maximum over Q values is then expensive or impossible, and a policy network can output the parameters of a distribution directly. Policy-based methods can also learn stochastic policies, which helps in partially observed problems. Value-based methods such as DQN are usually more sample efficient on small discrete action spaces, since they can reuse old data through a replay buffer.
Compare Q-Learning Update with REINFORCE Gradient to see the two update styles side by side.
What dynamic programming questions come up?
Dynamic programming questions ask how value iteration and policy iteration work and why they need a full model of the environment. They are the simplest algorithms that turn the Bellman equations into code.
What is the difference between value iteration and policy iteration?
Value iteration repeatedly applies the Bellman optimality backup to every state until the values stop changing, and reads off the greedy policy at the end. Policy iteration alternates between evaluating the current policy fully and improving it by acting greedily on the new values, and it stops when the policy no longer changes. Policy iteration usually needs fewer iterations, but each iteration costs more because it contains a full policy evaluation.
Write both in the Value Iteration and Policy Iteration problems.
Why does value iteration converge?
Value iteration converges because the Bellman optimality backup is a contraction in the max norm with factor gamma, so each sweep shrinks the largest error across states by at least a factor of gamma. By the Banach fixed-point theorem the values approach the unique fixed point V* from any starting point. This argument needs gamma below 1, or an episodic task in which every policy eventually terminates.
The Value Iteration problem asks you to stop when the change between sweeps falls below a tolerance, which is this contraction in practice.
Why is dynamic programming not enough for most RL problems?
Dynamic programming is not enough for most RL problems because it needs the full transition probabilities and reward function, and it sweeps over every state on each iteration. Real environments rarely give you the model, and their state spaces are often too large to enumerate. Model-free methods such as Monte Carlo and TD learning estimate the same values from sampled experience instead.
The Cracking Reinforcement Learning plan moves from dynamic programming to model-free methods in the same order.
What Monte Carlo and TD learning questions do interviewers ask?
Monte Carlo and temporal-difference questions ask how each method estimates values from experience and what the bias and variance of each estimate are. The comparison between them is one of the most common conceptual questions in an RL interview.
What is the difference between Monte Carlo and TD learning?
Monte Carlo methods wait until the end of an episode and update each visited state towards the actual return that followed it, while TD methods update after a single step towards the reward plus the discounted estimate of the next state. The TD(0) update is V(s) = V(s) + alpha (r + gamma V(s′) minus V(s)), where the bracketed quantity is the TD error. TD can learn online and in tasks that never end, while Monte Carlo needs complete episodes.
Write both in the Monte Carlo Policy Evaluation and TD(0) Value Update problems, and read the Monte Carlo lesson for why sample averages converge.
How do Monte Carlo and TD compare on bias and variance?
The Monte Carlo return is an unbiased estimate of the value, but it has high variance because it sums many random rewards and transitions. The TD target has lower variance because it depends on only one reward and one transition, but it is biased because it bootstraps from a current estimate that may be wrong. TD usually learns faster in practice, and n-step returns and TD(lambda) sit between the two extremes.
The Monte Carlo lesson shows how the error of a sample average shrinks with more samples.
What Q-learning, SARSA and on-policy versus off-policy questions come up?
These questions ask how Q-learning and SARSA differ, which one is on-policy, and what that changes about the policy each one learns. Interviewers often use the cliff walking example to test the difference.
What is the difference between Q-learning and SARSA?
SARSA updates Q(s, a) towards r + gamma Q(s′, a′), where a′ is the action the agent actually takes next, while Q-learning updates towards r + gamma times the maximum of Q(s′, a′) over all actions. SARSA therefore learns the value of the exploring policy it follows, and Q-learning learns the value of the greedy policy whatever the agent does. In cliff walking with epsilon-greedy exploration, SARSA learns a safer path away from the edge, while Q-learning learns the shortest path along it and falls off more often while it is still exploring.
Write both updates in the SARSA Update and Q-Learning Update problems.
What do on-policy and off-policy mean?
An on-policy method learns about the same policy that generates its data, and an off-policy method learns about a target policy that can differ from the behaviour policy that collected the data. SARSA, REINFORCE and PPO are on-policy, and Q-learning and DQN are off-policy. Off-policy methods can reuse old experience from a replay buffer, which makes them more sample efficient, but they are harder to keep stable when combined with function approximation.
The Experience Replay Buffer problem shows the data reuse that off-policy learning allows.
What is expected SARSA?
Expected SARSA updates towards r + gamma times the expected value of Q(s′, a′) under the policy, which is the sum over actions of pi(a′ | s′) Q(s′, a′), instead of the value of one sampled next action. Averaging over the next action removes that source of variance from the target. When the target policy is greedy, the expectation becomes the maximum and expected SARSA reduces to Q-learning.
Write it in the Expected SARSA problem.
What exploration and bandit questions do interviewers ask?
Exploration questions ask how an agent balances trying new actions against exploiting the best one it knows. They are usually asked through multi-armed bandits, which remove states and transitions so only the exploration problem is left.
What is the exploration and exploitation trade-off, and how does epsilon-greedy handle it?
The trade-off is that an agent must sometimes try actions that look worse to find out whether they are better, at the cost of reward it could have collected by exploiting its current best action. Epsilon-greedy picks a uniformly random action with probability epsilon and the greedy action otherwise. It is simple and works well in practice, especially when epsilon decays over training, but it explores every non-greedy action equally however bad that action has proved to be.
Write it in the Epsilon-Greedy Action Selection problem, and compare it with Softmax (Boltzmann) Exploration, which picks better actions more often.
How does UCB1 choose an arm?
UCB1 chooses the arm with the largest mean reward so far plus an exploration bonus of the square root of 2 ln t divided by n_a, where t is the total number of pulls and n_a is the number of times that arm has been pulled. The bonus is large for arms that have been tried rarely and shrinks as an arm is pulled more, so the algorithm is optimistic under uncertainty. Its regret grows only logarithmically with the number of pulls.
Implement it in the UCB1 for Multi-Armed Bandits problem.
How does Thompson sampling work?
Thompson sampling keeps a posterior distribution over each arm’s reward probability, samples one value from each posterior, and pulls the arm with the largest sample. For rewards of 0 or 1 the posterior is a Beta distribution, and a success adds 1 to its alpha parameter while a failure adds 1 to its beta parameter. Arms with uncertain posteriors sometimes produce high samples, so exploration happens in proportion to how likely each arm is to be the best.
Write the Beta-Bernoulli version in the Thompson Sampling (Beta-Bernoulli) problem.
What is a contextual bandit, and how does LinUCB work?
A contextual bandit observes a feature vector before each choice, so the best arm can depend on the context, as in choosing which article to show a particular user. LinUCB models each arm’s expected reward as a linear function of the context, fitted by ridge regression, and adds a bonus of alpha times the square root of x transposed A inverse x, which is large when the context lies in a direction the arm has seen little data for. It is the same optimism idea as UCB1 applied to a linear model.
Implement it in the LinUCB Contextual Bandit problem.
What function approximation and DQN questions come up?
DQN questions ask why Q-learning with a neural network is unstable and what each of the standard fixes does. Interviewers expect you to name the fix and the specific failure it addresses.
Why is Q-learning with a neural network unstable?
Q-learning with a neural network is unstable because it combines function approximation, bootstrapping and off-policy learning, a combination often called the deadly triad. Updating the network for one state changes its predictions for many others, including the target the update was moving towards, and consecutive samples from one trajectory are strongly correlated. Without corrections the values can oscillate or diverge.
The DQN Loss with Target Network problem shows the two pieces DQN adds to keep this stable.
What do the target network and experience replay do in DQN?
The target network is a copy of the Q-network that is used to compute the bootstrap target r + gamma times the maximum of Q_target(s′, a′) and is only updated every few thousand steps or slowly by averaging, so the target does not move with every gradient step. Experience replay stores transitions in a buffer and trains on random minibatches from it, which breaks the correlation between consecutive samples and lets each transition be used many times. The target is also multiplied by 1 minus the done flag, so terminal states do not bootstrap.
Write the loss in DQN Loss with Target Network and the buffer in Experience Replay Buffer.
What problem does double DQN solve?
Double DQN reduces the overestimation bias of Q-learning, which comes from taking the maximum over noisy Q estimates, since the maximum of noisy values is on average larger than the true maximum. It uses the online network to choose the best next action and the target network to evaluate that action, so the noise that picks the action is not the same noise that scores it. The change is a single line in the target computation.
Write the target in the Double DQN Target problem.
What are dueling DQN and prioritized experience replay?
Dueling DQN splits the network into a state-value stream V(s) and an advantage stream A(s, a), and combines them as Q = V + A minus the mean of A over actions, so the network can learn how good a state is without learning the effect of every action. Prioritized experience replay samples transitions in proportion to their TD error raised to a power alpha, so surprising transitions are replayed more often. Because this changes the sampling distribution, it multiplies each loss by an importance-sampling weight, with an exponent beta that is annealed towards 1 during training.
Write both in the Dueling DQN Decomposition and Prioritized Experience Replay problems.
What policy gradient and REINFORCE questions do interviewers ask?
Policy gradient questions ask how you can take the gradient of an expected return when the environment itself is not differentiable. The answer is the log-derivative trick, and most follow-ups are about the variance of the resulting estimate.
How does the REINFORCE policy gradient work?
REINFORCE estimates the gradient of the expected return as the sum over time steps of the gradient of log pi(a_t | s_t) multiplied by the return G_t that followed. It comes from the log-derivative trick, where the gradient of a probability equals the probability times the gradient of its log, so the gradient becomes an expectation you can estimate from sampled episodes. In code you minimize the negative sum of log probabilities weighted by returns and let autograd compute the gradient.
Write it in the REINFORCE Gradient problem.
Why does REINFORCE have high variance?
REINFORCE has high variance because each gradient is weighted by a full Monte Carlo return, which depends on every random action and transition until the end of the episode. If all returns are positive, every sampled action is pushed up, and the useful signal is only the small difference between actions that did better and those that did worse. Baselines, advantage estimates and larger batches are the standard ways to reduce this variance.
Compare the plain estimator with REINFORCE with Baseline to see the effect of subtracting a baseline.
What baseline and advantage questions come up?
Baseline questions ask why you can subtract a quantity from the return without changing the expected gradient. The advantage function follows directly from that answer.
Why does subtracting a baseline not bias the policy gradient?
Subtracting a baseline b(s) that does not depend on the action leaves the gradient unbiased, because the expected value of the gradient of log pi(a | s) times b(s) is b(s) times the gradient of the sum of pi(a | s) over actions. That sum is always 1, so its gradient is zero. The baseline can still reduce variance a great deal, and a learned state value V(s) is the usual choice.
Write it in the REINFORCE with Baseline problem.
What is the advantage function?
The advantage A(s, a) is Q(s, a) minus V(s), the amount by which taking action a in state s does better than the policy does on average from that state. Using the advantage as the weight in the policy gradient pushes up actions that beat the average and pushes down those that fall short. The one-step TD error r + gamma V(s′) minus V(s) is an estimate of the advantage, and it is unbiased when V is the true value of the policy.
The Generalized Advantage Estimation (GAE) problem builds advantage estimates from TD errors.
What actor-critic and GAE questions do interviewers ask?
Actor-critic questions ask how a learned value function replaces the Monte Carlo return, what the combined loss looks like, and how GAE trades bias against variance. These ideas lead straight into PPO.
How does an actor-critic method work, and what is in the A2C loss?
An actor-critic method trains a policy, the actor, together with a value function, the critic, and uses the critic to estimate the advantage that weights the policy gradient. The A2C loss has three terms, the policy loss of minus log pi(a | s) times the advantage, a value loss that is the squared error between V(s) and its return target, and an entropy bonus that is subtracted to discourage the policy from becoming deterministic too early. The advantage is treated as a constant in the policy term, so the policy loss does not send gradients into the critic.
Write the full loss in the Actor-Critic (A2C) Loss with Entropy problem.
What is generalized advantage estimation, and what does lambda control?
Generalized advantage estimation computes the advantage as a discounted sum of TD errors, A_t = delta_t + gamma lambda delta_{t+1} + (gamma lambda) squared delta_{t+2} and so on, with lambda between 0 and 1. Lambda of 0 gives the one-step TD error, which has low variance but is biased by errors in V, and lambda of 1 gives the Monte Carlo return minus V, which is unbiased but has high variance. In code you compute it backwards with A_t = delta_t + gamma lambda (1 minus done) A_{t+1}, and values around 0.95 are common.
Write it in the Generalized Advantage Estimation (GAE) problem.
What PPO clipping questions come up?
PPO questions ask what the clipped objective is, why the minimum is taken, and how you can tell that training is going wrong. PPO is the most common policy gradient method in interviews because it is also used to fine-tune language models.
What is the PPO clipped surrogate objective?
The PPO clipped objective is the average of the minimum of r_t A_t and clip(r_t, 1 minus epsilon, 1 plus epsilon) A_t, where r_t is the ratio of the new policy probability to the old policy probability of the action taken and epsilon is often 0.2. When the advantage is positive the objective stops rising once the ratio passes 1 plus epsilon, and when it is negative it stops rising once the ratio falls below 1 minus epsilon. This removes the incentive to move the policy far from the one that collected the data, which lets PPO take several epochs of minibatch updates on the same batch.
Write it in the PPO Clipped Surrogate Objective problem, and read the PPO lesson for how advantages, ratios and clipping fit together.
Why does PPO take the minimum of the clipped and unclipped terms?
PPO takes the minimum so that the objective is a pessimistic lower bound on the unclipped one. Clipping alone would also cap the objective in cases where the policy has moved in a harmful direction, for example a ratio far above 1 on an action with a negative advantage, and the minimum keeps the full unclipped penalty in those cases. The result is that the policy is never rewarded for large changes, but it is still corrected when a change makes things worse.
The PPO Clipped Surrogate Objective problem includes both signs of the advantage, which is the quickest way to check this behaviour.
How do you tell that PPO training is going wrong?
You watch the approximate KL divergence between the old and new policies and the clip fraction, the share of samples whose ratio fell outside the clipping range. A KL or clip fraction that keeps growing means each update moves the policy too far, which often comes before a collapse in reward, and common fixes are a lower learning rate, fewer epochs per batch or early stopping when the KL passes a threshold. Policy entropy that drops quickly towards zero is another warning sign, since the policy has stopped exploring.
The Train an Agent with PPO project has you write these health checks and then train an agent to balance a pole, including a run with the clipping removed.
What model-based RL and MCTS questions come up?
Model-based questions are usually brief in a general RL interview, and they ask what changes when the agent learns or is given a model of the environment. Monte Carlo tree search in AlphaGo Zero is the example most interviewers reach for.
What is the difference between model-based and model-free RL?
Model-based RL uses a model of the transitions and rewards, either given or learned, to plan or to generate extra experience, while model-free RL learns values or a policy directly from real experience. Model-based methods are usually far more sample efficient, but errors in a learned model can compound over a long plan and lead the agent to exploit mistakes in the model. Dynamic programming is model-based, and Q-learning, DQN and PPO are model-free.
The Cracking Reinforcement Learning plan starts with the model-based dynamic programming methods before the model-free ones.
How does Monte Carlo tree search work in AlphaGo Zero?
In AlphaGo Zero each simulation walks down the tree by choosing the move that maximizes Q plus an exploration term from the PUCT rule, which grows with the policy network prior and the parent visit count and shrinks with the visit count of the move itself. At a new leaf the network returns move priors and a value, and that value is backed up along the path, with its sign flipped at each level because the players alternate. After many simulations the visit counts at the root form an improved policy, which becomes the training target for the policy head, while the game result becomes the target for the value head.
The Build AlphaGo from Scratch plan has you write PUCT selection, the backup, root Dirichlet noise and the policy-value loss in 18 problems.
What reward design and RL for LLMs questions come up?
Reward design questions ask what happens when the reward does not say exactly what you want, and RL for LLMs questions ask how the same policy gradient methods are applied to language models. Our LLM interview questions guide covers RLHF, DPO and GRPO in more depth, so this section stays short.
What is reward hacking, and how do you design a reward that avoids it?
Reward hacking is when an agent finds a way to score highly on the reward as written without doing the task you intended, such as circling to collect a bonus repeatedly instead of reaching the goal. You reduce it by rewarding the outcome you care about instead of a proxy, testing the environment for loopholes before training, and watching the behaviour of trained agents as well as their scores. If you add shaping rewards to speed up learning, potential-based shaping, which adds gamma times Phi(s′) minus Phi(s) for some function Phi of the state, leaves the optimal policy unchanged.
The Build an RL Environment project has you build a grid world, watch an agent exploit a badly designed reward and then fix the reward.
How is reinforcement learning used to train language models?
Reinforcement learning from human feedback trains a reward model on pairs of responses that people compared, then optimizes the language model against that reward with PPO and a KL penalty that keeps it close to a reference model. DPO removes the reward model and the sampling by turning the same objective into a loss on preference pairs, and GRPO keeps sampling but replaces the learned critic with the mean reward of a group of responses to the same prompt. The LLM interview questions guide answers the follow-up questions on each method, and RLHF Explained: Reward Models and PPO and GRPO: Group Relative Policy Optimization walk through the objectives.
Practise the losses in Bradley-Terry Reward Model Loss, PPO with Reference-Policy KL, DPO Closed-Form Loss, GRPO Group-Relative Advantage and RLOO (REINFORCE Leave-One-Out).
Which reinforcement learning interview questions ask you to write code?
The reinforcement learning questions most often asked as code are the ones whose answer is a short update rule or loss, such as a Bellman backup, a Q-learning step, a bandit policy, the DQN loss, a return or advantage computation and the PPO objective. In these rounds the interviewer checks your targets, your handling of terminal states, your signs and which quantities you treat as constants, as well as the final output.
| Question | What you write | Practice |
|---|---|---|
| Value iteration | Bellman optimality backups over every state until the largest change falls below a tolerance | Value Iteration |
| A tabular Q-learning step | The TD target with a maximum over next actions and the update towards it | Q-Learning Update |
| Epsilon-greedy action selection | A random action with probability epsilon and the greedy action otherwise | Epsilon-Greedy Action Selection |
| A UCB1 bandit | Mean rewards, pull counts and the exploration bonus for each arm | UCB1 for Multi-Armed Bandits |
| The DQN loss | The bootstrap target from a target network, the done mask and the loss on the taken actions | DQN Loss with Target Network |
| A replay buffer | A fixed-size store of transitions and uniform random minibatch sampling | Experience Replay Buffer |
| The REINFORCE gradient | Discounted returns and the loss of log probabilities weighted by those returns | REINFORCE Gradient |
| Generalized advantage estimation | TD errors and the backward recursion with gamma, lambda and the done mask | Generalized Advantage Estimation (GAE) |
| The PPO clipped objective | Probability ratios from log probabilities, clipping and the minimum of the two terms | PPO Clipped Surrogate Objective |
To rehearse the policy optimization losses under time pressure, the Post-Training and Alignment assessment gives you 45 minutes for three problems on PPO, DPO, GRPO and RLHF objectives.
How should you practise reinforcement learning interview questions?
Practise reinforcement learning interview questions by answering each one out loud first and then writing the update or loss in code, because the code exposes mistakes in targets and signs that a spoken answer hides. Work through the topics in the order of this guide, since policy gradients, actor-critic methods and PPO each build on the value functions and TD errors that come before them.
| Study plan | Size | What it covers |
|---|---|---|
| Cracking Reinforcement Learning | 30 problems in six sections | MDPs and dynamic programming, tabular model-free RL, bandits, deep value-based RL, policy gradients and PPO, and RLHF and preference optimization |
| Build AlphaGo from Scratch | 18 problems in five sections | Go rules, a policy-value network, neural-guided MCTS, self-play training targets and the integrated pipeline |
| Probability and Statistics | 30 problems in six sections | Descriptive statistics and distributions through sampling, hypothesis testing and Bayesian updating |
Start with Cracking Reinforcement Learning if you have limited time, since it covers every topic in this guide in one plan. Then train a full agent in the Train an Agent with PPO project, and build the other side of the loop in the Build an RL Environment project. Use the Markov chains and Monte Carlo lessons when an MDP or sampling question needs more background.
What mistakes do candidates make on reinforcement learning questions?
- Mixing up the Q-learning and SARSA targets is the most common slip, so say which action the target uses and which policy that makes each method learn.
- Forgetting the done mask in a bootstrap target lets terminal states borrow value from the next episode, which is also a common real bug in DQN and GAE code.
- Saying that a baseline reduces bias is wrong, because a baseline that does not depend on the action leaves the gradient unbiased and only reduces variance.
- Describing PPO clipping without the minimum leaves out the part that keeps the full penalty when an update makes things worse.
- Explaining DQN without naming the failure each fix addresses sounds memorised, so tie the target network to moving targets, replay to correlated samples and double DQN to overestimation.
- Treating RL results from a single random seed as conclusive overstates them, since RL training is noisy and interviewers often ask how you would compare two runs fairly.
The guide to ML interview mistakes covers the mistakes that cost candidates across the other rounds of the loop.
Practice this
- Q-Learning Update covers the TD target, the maximum over next actions and the off-policy idea that most tabular and DQN questions build on.
- Generalized Advantage Estimation (GAE) makes you handle TD errors, the backward recursion and the done mask, which is where advantage questions usually go wrong.
- PPO Clipped Surrogate Objective has you write the ratio, the clip and the minimum, which is the clearest way to explain why PPO updates stay small.
Then continue with the Cracking Reinforcement Learning study plan, or train an agent end to end in the Train an Agent with PPO project.
Do you need deep learning to answer reinforcement learning interview questions?
You can answer the foundational reinforcement learning questions without deep learning, because MDPs, the Bellman equations, dynamic programming, Monte Carlo and TD methods, Q-learning and bandits all work with tables of values. Deep learning becomes necessary once the questions move to DQN, policy gradient networks, actor-critic methods and PPO, where you need to know how a loss is built and how gradients flow through a network.
Which reinforcement learning algorithms should you be able to implement in an interview?
You should be able to implement the tabular Q-learning and SARSA updates, epsilon-greedy action selection, the DQN loss with a target network, the REINFORCE gradient, generalized advantage estimation and the PPO clipped objective. Each one is a short function, which is why interviewers can ask for it as code in a single round. Value iteration and a UCB1 bandit are also common, since they test whether you can turn an equation into a loop.
How much math do reinforcement learning interviews require?
Reinforcement learning interviews mostly require expected values, conditional probability and the chain rule for gradients. You should be able to write the Bellman equations as expectations, explain why a baseline leaves the policy gradient unbiased, and show where the log-derivative trick comes from. Measure theory and convergence proofs are rarely asked outside research roles.
How do reinforcement learning questions differ for LLM post-training roles?
LLM post-training roles ask about the same policy gradient ideas, but they apply them to a language model that generates a whole response as one long sequence of actions. The questions focus on reward models trained from preference pairs, the KL penalty to a reference model, PPO, DPO and GRPO, and reward hacking. The classic tabular and DQN questions appear less often in these interviews.
Should you practise reinforcement learning by training agents or by writing the updates?
You should do both, starting with the individual updates and then training a small agent end to end. Writing each update as a tested function teaches you the exact targets, signs and shapes that interviewers check. Training an agent then shows you how those pieces behave together, including how sensitive the results are to the learning rate, the reward and the random seed.