Machine Learning Interview Questions and How to Answer Them

Name: TensorTonic
Author: TensorTonic

The classic machine learning interview questions grouped by topic, from bias and variance to trees, ensembles, SVMs, clustering and PCA, each with a short model answer and a lesson or problem to practise it.

By TensorTonic's founderLast updated November 11, 202628 min read

What machine learning interview questions should you expect?

Machine learning interview questions test whether you can explain how the classic algorithms learn, what they assume and when they fail, across the bias-variance tradeoff, validation, regularization, linear models, gradient descent, trees and ensembles, SVMs, k-NN, Naive Bayes, clustering, PCA, feature engineering, metrics and interpretability. Most are spoken questions that the interviewer follows with a why or a when, and some ask you to implement the algorithm from scratch. TensorTonic links every question in this guide to a lesson or a tested coding problem, and its Cracking ML study plan collects 35 classic machine learning problems in six sections.

This guide groups the questions by topic and gives a short model answer for each, followed by a lesson to read or a problem to solve. Where another TensorTonic guide already answers a question in depth, you get a one-sentence answer and a link instead. Neural network topics are in deep learning interview questions, statistics and A/B testing are in statistics interview questions, and the twenty functions most often asked as code are in ML coding interview questions.

What supervised and unsupervised learning questions do interviewers ask?

Interviewers ask what separates supervised from unsupervised learning, how parametric and non-parametric models differ, and where the two kinds of learning meet in one project. These questions often open a breadth round, because the answer shows whether you can frame a task before you choose a model.

What is the difference between supervised and unsupervised learning?

Supervised learning trains a model on examples that come with a target label, so the model learns a mapping from features to that label, as in classification and regression. Unsupervised learning has no labels and looks for structure in the features themselves, as in clustering, dimensionality reduction and anomaly detection. Semi-supervised and self-supervised learning sit between the two, using a small labelled set or labels derived from the data itself.

The Cracking ML study plan covers both sides, and Linear Regression from Scratch and K-Means Clustering are a good first problem for each.

What is the difference between a parametric and a non-parametric model?

A parametric model has a fixed number of parameters however much training data it sees, as linear regression, logistic regression and Gaussian Naive Bayes do. A non-parametric model grows with the data, as k-nearest neighbours does by keeping every training point and a decision tree does by adding splits as the data allows. Parametric models are cheaper and need less data when their assumptions hold, and non-parametric models are more flexible but slower to predict and easier to overfit.

Compare Gaussian Naive Bayes with the KNN Classifier, which classify the same kind of data in two very different ways.

How can unsupervised learning help a supervised model?

Unsupervised methods often prepare the inputs for a supervised model. PCA can compress many correlated features into a few components, cluster assignments can become a new categorical feature, and an anomaly detector such as an isolation forest can flag records to inspect before training. Each of these steps must be fitted on the training data only, or it leaks information from the validation set into the model.

The PCA from Scratch and Isolation Forest problems cover two of these steps.

What bias-variance and overfitting questions do interviewers ask?

Interviewers ask what the bias-variance tradeoff is, how you would diagnose overfitting or underfitting, and whether more data would help. The guide on how to prepare for an ML engineer interview has a full model answer for the tradeoff itself, so the questions here focus on diagnosis.

What is the bias-variance tradeoff?

Bias is the error from a model that is too simple to capture the pattern, variance is the error from a model flexible enough to fit noise in the training set, and lowering one usually raises the other.

The ML engineer interview guide explains how to tell which one your model suffers from, and the Ridge Regression problem has you add a penalty that accepts a little more bias in exchange for lower variance.

How do you tell whether a model is overfitting or underfitting?

You tell by comparing the training error with the validation error. A model that underfits has high error on both, and a model that overfits has low training error and a much higher validation error. A learning curve, which plots both errors against the size of the training set, shows the same pattern across a range of sizes and also tells you whether more data is likely to help.

The K-Fold Cross-Validation problem gives you a reliable validation error to compare against, and the regularization lesson covers the usual fixes for overfitting.

Does more training data always help?

More training data helps a model that overfits, because a larger sample makes it harder to fit noise, but it does little for a model that underfits. When the training and validation errors have already converged at a high value, the model needs more capacity or better features, and more rows of the same data will barely move either curve.

A decision tree shows both cases clearly, since a shallow tree underfits and a very deep one overfits, and you can build one in the Decision Tree Classifier (CART) problem.

What train, validation and test split questions come up?

Split questions ask why each split exists, how cross-validation works and how the split should follow the structure of the data. Interviewers use them to check that you can measure a model honestly.

Why do you need separate validation and test sets?

You need both because the validation set is used to make decisions, such as choosing hyperparameters, features or the final model, and every decision fits your choices a little more closely to that set. The test set is held back until the end and used once, so it gives an unbiased estimate of how the chosen model will perform on new data.

The data splits and leakage lesson in ML system design covers how the split should match the way the model is used.

How does k-fold cross-validation work?

K-fold cross-validation splits the training data into k folds, trains the model k times with a different fold held out each time, and averages the k validation scores. Every example is used for validation exactly once, so the estimate is less noisy than a single split, at the cost of training k models. Common choices for k are 5 and 10.

Write it yourself in the K-Fold Cross-Validation problem.

When should you use a stratified or a time-based split?

Use a stratified split for classification, especially with rare classes, because it keeps the class proportions the same in every fold. Use a time-based split whenever the data is ordered in time, training on the past and validating on the future, because a random split lets the model see the future and makes the score look better than it will be in production. Data with several rows per user or per group should be split by group, so that the same user never appears on both sides.

Each of these is a version of the data leakage question, which the ML engineer interview guide answers in full.

What do interviewers ask about L1 and L2 regularization?

Interviewers ask how a penalty on the weights reduces overfitting, why L1 and L2 behave so differently and when to combine them. The deep learning version of the question, which compares L2 regularization with weight decay in Adam, is answered in the deep learning interview questions guide.

What is the difference between L1 and L2 regularization?

L1 regularization adds the sum of the absolute values of the weights to the loss, and L2 regularization adds the sum of their squares. L1 drives some weights to exactly zero, so it also selects features, while L2 shrinks all the weights smoothly towards zero without removing any. Linear regression with an L1 penalty is called lasso, and linear regression with an L2 penalty is called ridge regression.

Implement both in Ridge Regression and Lasso Regression, and read the regularization lesson for the geometry behind them.

Why does L1 regularization produce sparse weights?

L1 produces sparse weights because the gradient of the absolute value keeps the same size however small the weight becomes, so the penalty keeps pushing a weight with little predictive value until it reaches exactly zero. The L2 gradient is proportional to the weight and fades as the weight shrinks, so the weight gets small without reaching zero. Geometrically, the L1 constraint region is a diamond with corners on the axes, and the loss contours usually touch it first at a corner, where some weights are zero.

The regularization lesson draws both constraint regions, and the Lasso Regression problem lets you see the zeros appear in your own weights.

What is elastic net, and when would you use it?

Elastic net adds both an L1 and an L2 penalty to the loss, with a mixing weight between them. It is useful when several features are strongly correlated, because lasso tends to keep one of them and drop the others somewhat arbitrarily, while the L2 part of elastic net spreads the weight across the group and the L1 part still allows sparsity.

Once ridge and lasso work, elastic net only needs both penalty terms in the same loss, and the convex optimization lesson explains why each of these problems stays convex.

What linear and logistic regression questions should you prepare for?

Linear and logistic regression questions ask about assumptions, fitting and how to read a coefficient. Interviewers like these models because they test the basics of training and interpretation without the noise of a large model.

What are the assumptions of linear regression?

Linear regression assumes a linear relationship between the features and the target, independent errors, constant error variance and, for the usual confidence intervals and p-values, normally distributed errors.

The statistics interview questions guide explains what correlated features do to the coefficients and how to read each one.

When would you fit linear regression with gradient descent instead of the normal equation?

The normal equation solves for the weights directly as the inverse of X transpose X, times X transpose y, which is exact but costs roughly the cube of the number of features and fails when X transpose X is singular. Gradient descent is the better choice when there are many features or too many rows to hold in memory, and it extends to models with no closed-form solution. Adding an L2 penalty makes the matrix invertible, which is one reason ridge regression is numerically safer.

Try both approaches in Linear Regression from Scratch, and read the least squares lesson to see why the normal equation is a projection.

Why is logistic regression a linear classifier?

Logistic regression is a linear classifier because it models the log-odds of the positive class as a linear function of the features. The decision boundary, where the predicted probability is 0.5, is the set of points where that linear function equals zero, which is a hyperplane. The sigmoid only turns the linear score into a probability, so a curved boundary needs nonlinear features such as polynomial terms.

Build it in Logistic Regression from Scratch, then extend the same idea to several classes in Softmax Regression.

Why is logistic regression trained with log loss and not squared error?

Log loss is convex in the weights of a logistic regression model and gives a large gradient for confident wrong predictions, while squared error on a sigmoid output is not convex and its gradient vanishes when the sigmoid saturates.

The guide on which loss function to use covers the general cross-entropy against mean squared error question, and Log Loss (Binary Cross-Entropy) has you write the loss.

How do you interpret a logistic regression coefficient?

A logistic regression coefficient is the change in the log-odds of the positive class for a one-unit increase in that feature, with the other features held fixed. Raising e to the power of the coefficient gives an odds ratio, so a coefficient of 0.7 multiplies the odds by about 2. Coefficients can only be compared across features when the features are on the same scale.

The maximum likelihood lesson explains how the coefficients are estimated.

What gradient descent questions come up in ML interviews?

Gradient descent questions in a classic ML interview ask about the variants, convergence and the effect of feature scale. Momentum, Adam and learning rate schedules are covered in the guide on which optimizer to use.

What is the difference between batch, stochastic and mini-batch gradient descent?

Batch gradient descent computes the gradient on the whole training set for each step, which is accurate but slow on large data. Stochastic gradient descent uses one example per step, which is cheap and noisy, and it keeps bouncing around the minimum unless the learning rate decays. Mini-batch gradient descent uses a small batch of examples per step, which balances the noise against the cost and makes good use of vectorized hardware, so it is the standard choice.

The Batch vs Stochastic vs Mini-Batch GD problem compares the three on the same task, and the SGD variants lesson explains the trade-off.

Does gradient descent always find the global minimum?

Gradient descent finds the global minimum when the loss is convex and the learning rate is small enough, which holds for linear regression with squared error and for logistic regression with log loss. For non-convex losses, such as those of neural networks, it can stop at a local minimum or slow down near a saddle point, and the result depends on the starting point and the learning rate.

The convex optimization lesson and the local minima and saddle points lesson cover both cases.

Why does feature scaling speed up gradient descent?

Feature scaling speeds up gradient descent because features on very different scales stretch the loss surface into a long, narrow valley, where the gradient points mostly across the valley and the steps zigzag. A single learning rate is then too large for the steep direction and too small for the shallow one. Standardizing the features makes the contours closer to circles, so each step heads more directly towards the minimum.

Practise the transforms in the Feature Scaling problem.

What decision tree questions do interviewers ask?

Decision tree questions ask how a split is chosen, how Gini impurity and entropy differ and how to stop a tree from memorising the data. They also set up the ensemble questions that usually follow.

How does a decision tree choose where to split?

A decision tree chooses each split greedily, by trying every feature and every candidate threshold and keeping the one that gives the largest decrease in impurity from the parent node to the weighted average of the two children. Classification trees usually measure impurity with Gini impurity or entropy, and regression trees use the variance of the target in each node, which is the mean squared error around the node mean. The tree repeats this on each child until a stopping rule is met.

Build both kinds in Decision Tree Classifier (CART) and Decision Tree Regressor.

What is the difference between Gini impurity and entropy?

Gini impurity is 1 minus the sum of the squared class proportions in a node, and entropy is minus the sum of each class proportion times its logarithm. Both are zero for a pure node and largest when the classes are evenly mixed, and in practice they usually choose very similar splits. Gini is slightly cheaper because it needs no logarithm, and the decrease in entropy produced by a split is called information gain.

The entropy lesson and the information gain lesson work through both measures.

How do you stop a decision tree from overfitting?

You stop a decision tree from overfitting by limiting its growth or by pruning it afterwards. Common limits are a maximum depth, a minimum number of samples in each leaf and a minimum impurity decrease for a split. Cost-complexity pruning grows a full tree and then removes branches whose gain does not justify a penalty on the number of leaves, and the right settings for any of these come from cross-validation.

Many teams replace a single deep tree with an ensemble, which the Random Forest from Scratch problem builds on top of your tree code.

What is the difference between bagging and boosting?

Bagging trains many models independently on bootstrap samples and averages them, which mainly reduces variance, and boosting trains models one after another so that each corrects the errors of the ensemble so far, which mainly reduces bias. Interviewers then ask how each method works in detail and how stacking differs from both.

How does bagging reduce variance?

Bagging reduces variance by training each model on a bootstrap sample, drawn with replacement from the training set, and averaging their predictions. Each model overfits in a different way, and averaging cancels part of that noise while the bias stays close to that of a single model. It helps most with high-variance models such as deep decision trees and does little for stable models such as linear regression.

Implement it in the Bagging Classifier problem, and read the resampling lesson for the bootstrap itself.

How does AdaBoost work?

AdaBoost trains a sequence of weak learners, often one-split decision stumps, on weighted data. After each round it raises the weights of the examples that the learner got wrong and lowers the rest, and it gives the learner a vote equal to one half of the log of (1 minus its weighted error) divided by its weighted error. The final prediction is the weighted vote of all the learners, and because misclassified points keep gaining weight, AdaBoost is sensitive to label noise and outliers.

Write the reweighting loop in AdaBoost from Scratch.

How does gradient boosting work?

Gradient boosting builds an additive model one tree at a time, where each new tree is fitted to the negative gradient of the loss with respect to the current predictions. For squared error the negative gradient is the residual, so each tree learns what the ensemble still gets wrong. Each tree is scaled by a learning rate before it is added, and a smaller learning rate with more trees usually generalizes better.

Build it in the Gradient Boosted Regressor problem.

What is stacking?

Stacking trains several different base models and then a meta-model that takes their predictions as input features. The meta-model must be trained on out-of-fold predictions, made by base models that did not see those rows, because predictions on their own training data are overconfident and would teach the meta-model to trust them too much.

Practise it in the Stacking Ensemble problem.

How do random forests and gradient boosting compare?

Random forests average many deep trees grown in parallel, while gradient boosting adds shallow trees in sequence, and the two are the ensembles most used on tabular data. Interviewers ask you to compare them, explain the feature sampling in a random forest and describe out-of-bag error.

When would you choose a random forest over gradient boosting?

Choose a random forest when you want a strong model with little tuning, because it is robust to its settings and adding more trees does not make it overfit. Gradient boosting usually reaches higher accuracy when it is tuned well, but it can overfit as trees are added, so it needs a small learning rate and early stopping on a validation set. Random forest trees can also be trained in parallel, while boosting trees must be built one after another.

Build both in Random Forest from Scratch and Gradient Boosted Regressor and compare them on the same data.

Why does a random forest sample features at each split?

A random forest samples a random subset of features at each split to make its trees less correlated. If one feature is very strong, every bagged tree would split on it first and the trees would make similar errors, and averaging correlated trees removes much less variance than averaging independent ones. A common default is the square root of the number of features for classification.

The Random Forest from Scratch problem puts the bootstrap and the feature sampling together.

What is out-of-bag error?

Out-of-bag error is a validation estimate that a bagged ensemble gets from its own bootstrap samples. Each bootstrap sample leaves out about 37 percent of the training rows, since the chance that a row is never drawn in n draws approaches 1 divided by e, so each row can be predicted by the trees that never saw it. The error on those predictions is close to a cross-validation estimate without training any extra models.

The Bagging Classifier problem has you draw the bootstrap samples that this estimate relies on.

What SVM and kernel questions do interviewers ask?

SVM questions ask what the margin is, what the C parameter controls and how the kernel trick lets a linear method draw curved boundaries. A clear picture of the margin makes every follow-up easier.

How does a support vector machine work?

A support vector machine finds the hyperplane that separates two classes with the largest margin, where the margin is the distance from the hyperplane to the nearest training points. Those nearest points are the support vectors, and they alone determine the boundary. When the classes cannot be separated perfectly, a soft-margin SVM lets some points fall inside the margin and minimizes the hinge loss plus a penalty on the size of the weights.

Train a linear SVM in SVM with Hinge Loss (SGD), and read the Lagrange multipliers lesson for the constrained optimization behind the dual form.

What does the C parameter control in an SVM?

The C parameter controls how heavily the SVM penalizes points that fall inside the margin or on the wrong side of it. A large C tolerates few violations and gives a narrower margin that fits the training data closely, with lower bias and higher variance. A small C allows more violations and gives a wider, smoother margin.

C acts as the inverse of a regularization strength, which the regularization lesson explains in the general case.

What is the kernel trick?

The kernel trick replaces every dot product between two examples with a kernel function that equals a dot product in a higher-dimensional feature space, without ever computing that space. It works for SVMs because the dual form of the problem depends on the data only through dot products. The RBF kernel, e to the power of minus gamma times the squared distance, is the common default, and a larger gamma gives a more flexible boundary that is easier to overfit.

The dot product and norm lesson covers the similarity measure that every kernel generalizes.

When would you choose logistic regression over an SVM?

Choose logistic regression when you need calibrated probabilities or coefficients you can interpret, since an SVM outputs a signed distance from the boundary and needs a separate calibration step, such as Platt scaling, to give probabilities. A kernel SVM can fit nonlinear boundaries on small and medium datasets, but its training cost grows faster than linearly with the number of examples, so a linear model or a tree ensemble is usually more practical on large data.

Train both on the same data with Logistic Regression from Scratch and SVM with Hinge Loss (SGD).

What k-nearest neighbours questions come up?

k-NN questions ask how the choice of k changes the model, why the method weakens with many features and what it costs at prediction time. Interviewers use them to test your sense of distance, scale and dimensionality.

How does k-NN work, and how does the choice of k affect it?

k-NN stores the training set and predicts a new point from its k closest training points, by majority vote for classification or by averaging for regression. A small k follows the training data closely, with low bias and high variance, and a large k smooths the boundary, with higher bias and lower variance. You choose k with cross-validation, and an odd k avoids ties in binary classification.

Implement it in the KNN Classifier problem, and try other distances in Distance Metrics.

Why does k-NN perform poorly in high dimensions?

k-NN performs poorly in high dimensions because of the curse of dimensionality. As the number of features grows, the distances between random points become more and more similar, so the nearest neighbour is hardly nearer than the farthest one and the vote carries little information. Feature selection and dimensionality reduction such as PCA help, and feature scaling matters too, since an unscaled feature with a large range would otherwise dominate the distance.

The Feature Scaling problem covers the transforms that keep one feature from dominating.

What does k-NN cost at prediction time?

k-NN has almost no training cost, but a brute-force prediction compares the query with every training point, which takes time proportional to the number of training points times the number of features for each query. Tree structures such as k-d trees speed this up in low dimensions, and approximate nearest neighbour indexes are used for large datasets and embeddings.

The retrieval and ranking lesson shows where nearest neighbour search appears in production systems.

What Naive Bayes questions do interviewers ask?

Naive Bayes questions ask what the independence assumption is, why the model still works when it is wrong and why it needs smoothing. They often come up for text classification, where Naive Bayes is still a strong baseline.

What is naive about Naive Bayes?

Naive Bayes is called naive because it assumes the features are conditionally independent given the class. With that assumption the probability of a class given the features is proportional to the prior of the class times the product of the probability of each feature given the class, which is cheap to estimate even with many features. Gaussian Naive Bayes models each continuous feature with a normal distribution per class, and the multinomial and Bernoulli versions handle counts and binary features.

Implement the Gaussian version in Gaussian Naive Bayes, with the Bayes theorem lesson alongside.

Why does Naive Bayes work when the independence assumption is wrong?

Naive Bayes often classifies well when the assumption is wrong because classification only needs the correct class to receive the highest score, and that ranking can survive errors in the individual probabilities. The probabilities themselves are usually pushed too close to 0 or 1, since correlated features are counted as separate evidence, so they should be calibrated before you use them as confidence scores.

The conditional probability lesson explains what conditional independence does and does not imply.

Why does Naive Bayes need smoothing?

Naive Bayes needs smoothing because a feature value that never appears with a class in the training data gets a probability of zero, and one zero in the product sets the score of that class to zero whatever the other features say. Laplace smoothing adds a small count, usually 1, to every feature value and class pair. Implementations also add log probabilities instead of multiplying probabilities, which avoids numerical underflow.

The probability distributions lesson covers the distributions that each version of Naive Bayes assumes.

How does k-means work, and how do you choose k?

K-means alternates between assigning each point to its nearest centroid and moving each centroid to the mean of its points, and k is usually chosen with the elbow method, the silhouette score and the needs of the task. Interviewers then ask about initialization and the cases where k-means fails.

What does k-means minimize, and does it always find the best clustering?

K-means minimizes the within-cluster sum of squared distances between each point and its centroid, which is also called inertia. Neither the assignment step nor the update step can increase this value, so the algorithm always converges, but it converges to a local minimum that depends on the starting centroids. Running it several times from different starts and keeping the result with the lowest inertia is standard practice.

Write both steps in the K-Means Clustering problem.

What does k-means++ initialization do?

K-means++ picks the first centroid at random from the data and each later centroid with probability proportional to its squared distance from the nearest centroid already chosen. This spreads the starting centroids out, which usually gives faster convergence and a lower final inertia than uniform random starts, and its expected inertia is provably within a factor of order log k of the best clustering.

Implement the seeding in K-Means++ Initialization.

How do you choose the number of clusters?

You choose the number of clusters by comparing several values of k. The elbow method plots inertia against k and looks for the point where adding clusters stops reducing it much, and the silhouette score compares how close each point is to its own cluster with how close it is to the next nearest one, with higher values meaning better separated clusters. The final choice should also make sense for the task, such as the number of customer segments a team can act on.

Once K-Means Clustering works, running it for several values of k and plotting the inertia gives you the elbow plot.

When does k-means fail, and what would you use instead?

K-means fails when clusters are far from spherical, have very different sizes or densities, or contain outliers, because it assigns every point to its nearest centroid by Euclidean distance. DBSCAN finds clusters of any shape from point density and marks sparse points as noise, a Gaussian mixture model allows elliptical clusters and soft assignments, and agglomerative clustering builds a hierarchy you can cut at any level. Features should be scaled before any of these methods.

Try the alternatives in DBSCAN, Gaussian Mixture Models (EM) and Agglomerative Clustering.

What PCA and dimensionality reduction questions do interviewers ask?

PCA questions ask how the components are computed, why the features must be standardized and how many components to keep. Some interviews add t-SNE as a comparison.

How does PCA work?

PCA centres the data and finds the directions of greatest variance, which are the eigenvectors of the covariance matrix ordered by their eigenvalues. Each eigenvalue is the variance of the data along its direction, and projecting onto the top k eigenvectors gives the best k-dimensional linear approximation of the data in squared error. In practice PCA is computed with the singular value decomposition of the centred data matrix, which is more numerically stable than forming the covariance matrix.

The PCA lesson and the SVD lesson explain the derivation, and PCA from Scratch has you write it.

Why should you standardize features before PCA?

You should standardize features before PCA because PCA looks for directions of large variance, and a feature measured in large units has a large variance for that reason alone. Without scaling, the first component often just follows the feature with the largest range. Centering is always required, and scaling to unit variance is the usual choice when the features are in different units.

Practise both steps in the Feature Scaling problem.

How do you choose the number of principal components?

You choose the number of components from the cumulative explained variance ratio, keeping enough components to explain a target share such as 90 or 95 percent of the variance, or from a scree plot where the eigenvalues level off. When PCA feeds a supervised model, the best number is the one that gives the best validation score.

The eigenvalues and eigenvectors lesson explains why each eigenvalue measures explained variance.

What is the difference between PCA and t-SNE?

PCA is a linear projection that preserves global variance and can transform new data, while t-SNE is a nonlinear method that preserves local neighbourhoods and is used mainly for two- or three-dimensional visualization. Distances between clusters and cluster sizes in a t-SNE plot are not reliable, the result depends on the perplexity setting and the random seed, and standard t-SNE cannot place new points without being rerun.

Compute the first stage of t-SNE in the t-SNE Affinity Matrix problem.

What feature engineering and scaling questions come up?

Feature questions ask how you would turn raw columns into inputs a model can use and which models care about scale. Interviewers often hide a leakage trap inside them.

What is the difference between standardization and min-max scaling?

Standardization subtracts the mean and divides by the standard deviation, so each feature has mean 0 and standard deviation 1, and min-max scaling maps each feature to a fixed range, usually 0 to 1. Min-max scaling is sensitive to outliers, because one extreme value squeezes the rest into a narrow band, so standardization or a robust scaler based on the median and interquartile range is the safer default.

Implement the scalers in the Feature Scaling problem.

Which models need feature scaling?

Models that use distances or gradient descent need feature scaling, including k-NN, k-means, SVMs, PCA and regularized linear and logistic regression, where an L1 or L2 penalty treats every weight as if the features shared one scale. Tree-based models such as decision trees, random forests and gradient boosting do not need it, because a split depends only on the order of values within one feature.

The Distance Metrics problem is a quick way to see how one unscaled feature can dominate a distance.

How do you encode categorical features?

Use one-hot encoding for a categorical feature with a small number of unordered values, and ordinal encoding when the values have a natural order. For a feature with many values, such as a postcode or a product ID, target encoding replaces each value with the mean target for that value, and it must be computed out of fold or on the training split only to avoid leakage. Feature hashing is another option, which maps values into a fixed number of columns at the cost of some collisions.

Practise the common encodings in the Categorical Encoding problem.

Why must preprocessing be fitted on the training data only?

Preprocessing must be fitted on the training data only because the statistics it learns, such as the mean for a scaler or the category means for target encoding, would otherwise include information from the validation and test rows. Putting every step in one pipeline that is refitted inside each cross-validation fold avoids the problem.

The lesson on choosing features in ML system design goes further into building features for a production model.

How should you handle missing data?

You should handle missing data by first asking why the values are missing, then choosing between dropping rows, imputing values or letting the model use the missingness itself. Interviewers listen for that first step, since the right fix depends on it.

What are the types of missing data?

Data is missing completely at random when the chance of a value being missing has nothing to do with any data, missing at random when it depends only on other observed features, and missing not at random when it depends on the missing value itself, such as people with high incomes leaving an income field blank. Dropping rows is only safe in the first case, and in the last case the fact that a value is missing carries information of its own.

The Missing Value Imputation problem has you fill the gaps once you have decided why they exist.

Which imputation method should you use?

Mean or median imputation is a quick baseline for numeric features, with the median safer when the feature is skewed, and the most frequent value or a separate missing category works for categorical features. Model-based methods such as k-NN imputation or iterative imputation use the other features and are often more accurate. Adding a binary indicator for each imputed feature lets the model learn from the missingness, some gradient boosting libraries handle missing values natively, and every imputer should be fitted on the training split only.

The basic statistics lesson explains why the median resists skew and outliers better than the mean.

How should you answer class imbalance questions?

You should answer class imbalance questions by fixing the metric first, then the training with class weights or resampling, and finally the decision threshold. The ML engineer interview guide has the full model answer, and the guide to ML evaluation metrics covers which metric to report.

Does resampling change the predicted probabilities?

Resampling changes the predicted probabilities, because oversampling the minority class or undersampling the majority class changes the class ratio the model sees in training. The model then predicts the rare class more often than it really occurs, so if you need calibrated probabilities you should correct them for the original class ratio or recalibrate on untouched validation data. Resampling should only ever be applied to the training split.

The Precision-Recall Curve & Average Precision problem builds the metric that suits a rare positive class.

When should you use class weights instead of resampling?

Class weights multiply the loss of each example by a weight for its class, which has a similar effect to resampling without duplicating or discarding rows. They are the simpler first choice for models whose loss accepts weights, such as logistic regression, SVMs and gradient boosting, and resampling is useful when a model offers no way to weight its loss.

The guide on which loss function to use compares class weights with focal loss.

What evaluation metric questions should you expect?

You should expect questions on precision and recall, ROC AUC and PR AUC, and MAE and RMSE, and the guide to ML evaluation metrics answers each of them in depth, with the formulas. The one-sentence answers below are the ones interviewers usually ask for first.

What is the difference between precision and recall?

Precision is the share of predicted positives that are truly positive, and recall is the share of actual positives that the model finds.

Compute both across every threshold in Precision-Recall Curve & Average Precision.

What does ROC AUC measure?

ROC AUC is the probability that the model scores a randomly chosen positive example above a randomly chosen negative one, so it measures ranking quality across all thresholds.

Write it in AUC-ROC from Scratch, and read the integration and AUC lesson for the area under the curve.

Should you use MAE or RMSE for regression?

Use RMSE when large errors are much worse than small ones, since squaring weights them heavily, and use MAE when every unit of error costs the same or the data contains outliers.

Implement both in the Regression Metrics problem.

What model selection and hyperparameter tuning questions come up?

Model selection questions ask how you would choose between models and tune them without fooling yourself. Interviewers want a process that starts from a baseline and keeps the test set out of every decision.

How do you choose between several models?

You choose between models by starting from a simple baseline, such as predicting the majority class or fitting a regularized linear model, and comparing every candidate against it on the same cross-validation folds and metric. When the scores are close, prefer the simpler, faster or more interpretable model, and weigh practical limits such as prediction latency, training cost and how often the model must be retrained.

The baselines lesson and the model selection lesson in ML system design walk through the same decision for production systems.

What is the difference between grid search, random search and Bayesian optimization?

Grid search tries every combination from a fixed list of values for each hyperparameter, random search samples combinations at random, and Bayesian optimization fits a model of the score as a function of the hyperparameters and uses it to pick the next promising point. Random search usually beats grid search on the same budget, because only a few hyperparameters tend to matter and random sampling tries many more distinct values of each one. Bayesian optimization is most useful when each training run is expensive.

Every search method needs a reliable score for each configuration, which the K-Fold Cross-Validation problem provides.

Why can tuning on the same folds give an optimistic score?

Tuning and reporting on the same cross-validation folds gives an optimistic score, because the best of many configurations is partly the one that got lucky on those folds. Nested cross-validation fixes this with an inner loop that tunes the hyperparameters and an outer loop that scores the tuned model on folds it never saw, and a held-out test set used once at the end serves the same purpose.

This is the same multiple comparisons effect that the statistics interview questions guide describes for A/B tests.

What interpretability questions do interviewers ask?

Interpretability questions ask how you would explain what a model learned and where each explanation method misleads. Expect to name more than one method.

What is the difference between global and local interpretability?

Global interpretability explains how the model behaves across the whole dataset, for example through coefficients, feature importance or partial dependence plots, and local interpretability explains a single prediction, for example through SHAP values or LIME. SHAP values come from Shapley values in game theory and split each prediction into additive contributions from the features, and averaging their absolute size over many rows also gives a global view.

Start with a global method in the Permutation Feature Importance problem.

Why can impurity-based feature importance be misleading?

Impurity-based importance from a tree ensemble is computed on the training data and favours features with many distinct values, such as IDs or continuous columns, because they offer more candidate splits. Permutation importance avoids most of this by shuffling one feature in the validation data and measuring how much the score drops. Both methods understate correlated features, since the model can fall back on a correlated copy when one feature is shuffled.

Compute permutation importance on top of your own forest with Random Forest from Scratch and Permutation Feature Importance.

Are linear model coefficients a reliable measure of importance?

Linear model coefficients measure importance reliably only when the features share a scale and are not strongly correlated. A coefficient depends on the units of its feature, so standardize the features before comparing them, and with correlated features the weight can be split between them or even flip sign while the predictions stay just as good.

The correlation lesson explains how to measure the correlation that makes coefficients unstable.

Which machine learning interview questions ask you to write code?

The machine learning questions most often asked as code are the ones whose answer fits in one short function, such as fitting a linear model, growing a tree, running k-means or computing a metric. In these rounds the interviewer checks your vectorized NumPy, your handling of edge cases such as empty clusters or pure nodes, and whether you can explain each line as you write it.

Machine learning interview questions that are often asked as code, what you write, and a problem to practise each
Question	What you write	Practice
Fit linear regression	The normal equation or a gradient descent loop on the mean squared error	Linear Regression from Scratch
Train logistic regression	The sigmoid, the log loss gradient and the weight update	Logistic Regression from Scratch
Add an L1 or L2 penalty	The penalty term and its effect on the update or the closed form	Lasso Regression
Grow a decision tree	The impurity of each candidate split and the recursive build	Decision Tree Classifier (CART)
Build a random forest	Bootstrap samples, random feature subsets and a majority vote	Random Forest from Scratch
Boost a sequence of trees	Residuals as targets, a learning rate and the additive prediction	Gradient Boosted Regressor
Cluster with k-means	The assignment and update steps, repeated until the centroids settle	K-Means Clustering
Reduce dimensions with PCA	Centering, the eigenvectors or SVD, and the projection onto the top components	PCA from Scratch
Classify with Naive Bayes	Class priors, per-class means and variances, and log-probability scores	Gaussian Naive Bayes
Cross-validate a model	Fold indices, a train and score loop and the averaged result	K-Fold Cross-Validation
Compute ROC AUC	Sorting by score and the area under the curve	AUC-ROC from Scratch

The guide to ML coding interview questions explains how these rounds are graded, and the guide on learning machine learning from scratch explains why building the algorithms yourself still matters. To rehearse under time pressure, the Supervised + Unsupervised Learning from Scratch assessment gives you 45 minutes for three problems on supervised and unsupervised algorithms in pure Python, and the ML Evaluation Metrics Sprint gives you 45 minutes for three metric implementations.

How should you practise machine learning interview questions?

Practise machine learning interview questions by answering each one out loud first and then implementing the algorithm behind it, because writing the code exposes the gaps that a spoken answer hides. Work through the topics in the order of this guide, since ensembles depend on trees, and regularization and feature scaling come back in almost every later topic.

TensorTonic study plans for machine learning interview preparation, with their size and coverage
Study plan	Size	What it covers
Cracking ML	35 problems in six sections	Linear models, classification, trees and ensembles, unsupervised learning, evaluation and feature engineering
Probability and Statistics	30 problems in six sections	Descriptive statistics and distributions through sampling, hypothesis testing and Bayesian updating
Linear Algebra	30 problems in five sections	Vector operations through matrix decompositions and their use in methods such as PCA
Optimization	25 problems in six sections	Gradient descent, momentum, adaptive optimizers, learning rate schedules, regularization and second-order methods
NumPy	25 problems in five sections	Creating arrays, indexing, broadcasting and vectorized operations

Start with Cracking ML if you have limited time, since it covers every algorithm family in this guide in one plan. Add the math plans when a derivation feels uncertain, and use the ML math formula sheet to review the formulas before an interview. When you can answer most of these questions without notes, take the ML Engineer Mock Interview, a 75-minute sitting of five problems across Python, ML coding, data wrangling, evaluation metrics and statistics, or the FAANG ML Phone Screen, which pairs two LeetCode-style medium problems with one ML implementation in 45 minutes. The guide to ML mock interviews explains how to read the score.

What mistakes do candidates make on machine learning questions?

Giving a definition without the reason behind it leaves the interviewer to ask the follow-up, so say why a method works and when it fails in the same answer.
Saying that L1 and L2 both just make weights smaller misses the point of the question, which is that only L1 sets weights to exactly zero.
Fitting a scaler, an imputer or a target encoder on the full dataset before splitting is a leakage error that interviewers listen for in almost every feature question.
Describing random forests and gradient boosting as interchangeable tree ensembles hides the bagging and boosting difference that the comparison is meant to test.
Choosing k for k-means from the elbow plot alone, without the silhouette score or the needs of the task, sounds like a recipe and invites the follow-up.
Reading feature importance from a single method, or comparing coefficients on unscaled features, leads to confident conclusions that the data does not support.

The guide to ML interview mistakes covers the mistakes that cost candidates across the other rounds of the loop.

Practice this

Logistic Regression from Scratch covers the log loss, its gradient and the linear decision boundary that many follow-up questions build on.
Decision Tree Classifier (CART) makes you choose splits by impurity, which prepares you for every bagging, boosting and feature importance question.
K-Means Clustering has you write the assignment and update steps, which is the clearest way to explain convergence and the choice of k.

Then continue with the Cracking ML study plan, or take the Supervised + Unsupervised Learning from Scratch assessment to test yourself against the clock.

Which machine learning topics come up most often in interviews?

The topics that come up most often are the bias-variance tradeoff, regularization, cross-validation and data leakage, logistic regression, decision trees and tree ensembles, and evaluation metrics, because they apply to almost every model. Clustering, PCA, SVMs and Naive Bayes come up less often, but they are common in breadth rounds where the interviewer moves quickly across many topics.

Do you need to know the math behind each algorithm?

You need enough math to explain why each algorithm works and to derive its key step, such as the gradient of the log loss, the update in k-means or the eigenvectors in PCA. Full proofs are rarely asked outside research roles. The linear algebra, probability and optimization behind these answers are covered in the TensorTonic ML Math lessons.

Are classic machine learning questions still asked now that deep learning is common?

Classic machine learning questions are still asked, because many tabular problems such as fraud detection and pricing are solved with linear models and gradient boosted trees. The ideas of bias, variance, regularization and validation also apply to neural networks, so interviewers use the classic models to test them without the noise of a large architecture.

How are machine learning theory questions different from ML coding questions?

Theory questions ask you to explain how an algorithm works, what it assumes and when it fails, while ML coding questions ask you to implement it from scratch in NumPy or plain Python. Interviewers often move from one to the other on the same algorithm, and writing the code for a model is also the most reliable way to make the spoken answer precise.

Should you learn scikit-learn for a machine learning interview?

You should know scikit-learn well enough to use its estimators, pipelines and cross-validation tools, since take-home tasks and practical rounds often allow it. Theory and coding rounds usually expect you to explain or implement the algorithm without the library, so knowing what fit and predict do inside each model matters more than remembering the API.