Machine Learning Glossary: 100 Terms You Should Be Able to Define

Machine learning is the practice of building programs that learn a mapping or a pattern from data, in place of hand-written rules. A model improves at a task as it sees more examples, and its quality is measured by a metric on data it has not seen.

• GuideLearning ML from scratch

Supervised learning trains a model on labeled pairs of inputs and targets so that it can predict the target for new inputs. Predicting a number is called regression, and predicting a category is called classification.

• ProblemLinear Regression Closed Form

Unsupervised learning finds structure in data that has no labels, for example by grouping similar points into clusters or by compressing the data into fewer dimensions. Clustering, dimensionality reduction and density estimation are its main tasks.

• ML MathPrincipal Component Analysis (PCA) Intuition
• ProblemK-Means Assignment Step

Self-supervised learning creates its training labels from the data itself, for example by hiding the next word or a patch of an image and asking the model to predict it. Large language models are pretrained this way on next-token prediction.

• LLM InternalsGPT Pretraining

Reinforcement learning trains an agent to choose actions in an environment so that it maximizes the expected sum of discounted future rewards. The agent learns from the rewards that follow its own actions, with no labeled answers.

• ProblemTabular Q-Learning (Single Update)

A feature is one measurable input variable that a model uses to make a prediction, such as a pixel value, a word count or a user's age. The features of one example together form its feature vector x.

A hyperparameter is a setting chosen before training that gradient descent does not learn, such as the learning rate, the batch size, the number of layers or the regularization strength. Parameters such as weights and biases are learned from data, while hyperparameters are tuned by comparing scores on a validation set.

• ProblemLearning Rate Sweep

The training set is the portion of the data used to fit the model's parameters. The model sees these examples, and their loss drives every gradient step.

• ProblemStratified Train/Test Split

The validation set is held-out data used during development to tune hyperparameters, compare models and decide when to stop training. Because these decisions are fitted to it, its score becomes optimistic, and more so the more choices are tuned on it.

The test set is held-out data used once, at the end, to estimate how the final model will perform on new data. Using it to make modeling decisions turns it into a second validation set and makes the estimate optimistic.

Cross-validation estimates generalization by splitting the data into k folds, training on k minus 1 folds and scoring on the remaining fold, then repeating until each fold has been scored once. The k scores are averaged, and their spread shows how stable the estimate is.

• ML MathResampling Methods
• ProblemK-Fold Split (Indices Only)

Data leakage happens when information that would not be available at prediction time reaches the training process, which inflates offline scores. Common causes are fitting a scaler on the full dataset before splitting, features computed from the target and random splits of time-ordered data.

• GuideKaggle vs implementing from scratch

Feature scaling puts features on comparable ranges so that distance-based models and gradient descent are not dominated by large-valued features. Standardization subtracts the mean and divides by the standard deviation, min-max scaling maps values to the range 0 to 1, and both statistics must be computed on the training set only.

• ProblemImplement z-Score Standardization

One-hot encoding turns a categorical feature with K possible values into a vector of length K that holds a 1 at the index of the observed category and 0 everywhere else. It lets linear models and neural networks use categories without implying an order between them.

• ProblemOne-Hot Encoding (Multi-class)

Class imbalance means one class is much rarer than another, as in fraud or disease detection. Accuracy becomes misleading because always predicting the majority class scores highly, so teams turn to precision, recall, PR-AUC, class weights, resampling or a tuned decision threshold.

• ProblemImplement Focal Loss
• GuideWhich evaluation metric to use

A probability distribution assigns probabilities to the possible values of a random variable. A discrete variable has a probability mass function that sums to 1, a continuous variable has a density that integrates to 1, and the cumulative distribution function gives P(X at most x) for both.

• ML MathProbability Distributions (PDF, PMF, CDF)
• ProblemBinomial PMF and CDF in Python

The expected value of a random variable is its probability-weighted average, which is the long-run mean over many draws.

• ML MathRandom Variables & Expectation
• ProblemExpected Value (Discrete Distribution)

Variance is the expected squared distance of a random variable from its mean, and the standard deviation is its square root, which is in the same units as the data. The sample variance divides by n minus 1 so that it is an unbiased estimate of the population variance.

• ML MathBasic Statistics
• ProblemSample Variance & Standard Deviation

Covariance measures how two variables move together, and it is positive when they tend to sit above their means together and below their means together. A covariance matrix collects the covariance of every pair of features and is the starting point of PCA.

• ML MathCovariance Matrices
• ProblemCompute Covariance Matrix

Pearson correlation is covariance divided by the product of the two standard deviations, which gives a unitless value between minus 1 and 1 that measures linear association. A strong correlation does not show that one variable causes the other, and Spearman correlation applies the same formula to ranks to capture monotonic relationships.

• ML MathCorrelation vs Causation
• ProblemCompute Pearson Correlation Matrix

Conditional probability is the probability of an event A given that event B has occurred. It equals the joint probability of both events divided by the probability of B.

• ML MathConditional Probability
• ProblemConditional Probability

Bayes' theorem turns the probability of the evidence given a hypothesis into the probability of the hypothesis given the evidence. The prior P(A) is updated by the likelihood P(B given A) to give the posterior P(A given B).

• ML MathBayes' Theorem & Inference
• ProblemBayes' Theorem

Maximum likelihood estimation picks the parameters that make the observed data most probable under the model, which in practice means maximizing the sum of log-likelihoods. Minimizing mean squared error is MLE under Gaussian noise, and minimizing cross-entropy is MLE for a Bernoulli or categorical model.

• ML MathMaximum Likelihood Estimation (MLE)
• ProblemMaximum Likelihood Estimation

The central limit theorem says that the mean of n independent draws from the same distribution with finite variance is approximately normal for large n, whatever the shape of the original distribution. The sample mean has mean mu and standard error sigma over the square root of n.

• ML MathCentral Limit Theorem
• ProblemCentral Limit Theorem

A confidence interval is a range computed from a sample that would contain the true parameter in a stated share of repeated samples, such as 95 percent. It does not mean there is a 95 percent probability that this particular interval holds the parameter, which is a common misreading.

• ML MathConfidence Intervals
• ProblemBootstrap Mean & Confidence Interval

A p-value is the probability, assuming the null hypothesis is true, of seeing a test statistic at least as extreme as the one observed. A small p-value is evidence against the null hypothesis, and it is not the probability that the null hypothesis is true.

• ML MathP-Values & Significance
• ProblemOne-Sample t-Test

A tensor is a multi-dimensional array of numbers, where a scalar has 0 dimensions, a vector 1, a matrix 2 and a batch of color images 4. Deep learning code mostly reasons about tensor shapes, such as batch by sequence length by model width.

• ML MathMatrix Multiplication & Tensor Operations
• ProblemMatrix Transpose

The dot product of two vectors is the sum of their element-wise products, and it equals the product of their lengths times the cosine of the angle between them. Every neuron, linear layer and attention score is built from dot products.

• ML MathDot Product & Vector Norms
• ProblemImplement Dot Product

A norm measures the length of a vector. The L2 norm is the square root of the sum of squares, and the L1 norm is the sum of absolute values.

• ML MathDot Product & Vector Norms
• ProblemVector Norms

Cosine similarity is the dot product of two vectors divided by the product of their norms, which gives a score from minus 1 to 1 that ignores vector length. It is the usual way to compare embeddings in search and retrieval.

• ML MathDot Product & Vector Norms
• ProblemImplement Cosine Similarity in Python (NumPy)

Multiplying an m by n matrix A by an n by p matrix B gives an m by p matrix whose entry (i, j) is the dot product of row i of A with column j of B. The inner dimensions must match, and the operation is not commutative.

• ML MathMatrix Multiplication & Tensor Operations
• ProblemMatrix Multiply

An eigenvector of a square matrix A is a non-zero vector whose direction A keeps, so A only scales it, and the scale factor is its eigenvalue. A real symmetric matrix has real eigenvalues and a full set of orthogonal eigenvectors, which is why covariance matrices can be diagonalized for PCA.

• ML MathEigenvalues & Eigenvectors
• ProblemCalculate Eigenvalues of a Matrix

SVD factors any m by n matrix into U times Sigma times V transpose, where U and V have orthonormal columns and Sigma is diagonal with non-negative singular values in decreasing order. Keeping the largest k singular values gives the best rank-k approximation of the matrix.

• ML MathSingular Value Decomposition (SVD) Intuition
• ProblemSVD Components

PCA reduces dimensionality by projecting centered data onto the directions of greatest variance. Those directions are the top eigenvectors of the covariance matrix, and each eigenvalue divided by the sum of all eigenvalues is the share of variance that component explains.

• ML MathPrincipal Component Analysis (PCA) Intuition
• ProblemPCA Projection onto Principal Components

The gradient of a scalar function is the vector of its partial derivatives with respect to each input, and it points in the direction of steepest increase. Training moves the parameters against the gradient of the loss.

• ML MathPartial Derivatives
• ProblemFinite Difference Gradient Checking

The chain rule gives the derivative of a composed function as the product of the derivatives of its parts. Backpropagation is the chain rule applied layer by layer, from the loss back to each weight.

• ML MathChain Rule
• ProblemBackpropagate Through a Scalar Neuron

Gradient descent minimizes a loss by repeatedly moving the parameters a small step in the direction of the negative gradient. The size of that step is set by the learning rate.

• ML MathStochastic Gradient Descent (SGD) Variants
• ProblemImplement Gradient Descent for a 1D Quadratic

Stochastic gradient descent estimates the gradient from one example or a small mini-batch in place of the full dataset, which makes each step much cheaper and adds noise that can help it escape poor regions. In practice, SGD usually means mini-batch SGD.

• ML MathStochastic Gradient Descent (SGD) Variants
• ProblemApply One Full SGD Training Step
• GuideWhich optimizer to use

The batch size is the number of examples used to compute one gradient estimate and one parameter update. Larger batches give less noisy gradients and better hardware use but need more memory, and gradient accumulation can simulate a larger batch.

• LLM InternalsGradient Accumulation
• ProblemBatch Shuffling & Mini-Batch Generator

An epoch is one full pass over the training set. With N examples and batch size B, one epoch is N divided by B update steps, rounded up, and each of those steps is called an iteration.

The learning rate is the multiplier on the gradient in each update and is usually the most important hyperparameter. A rate that is too high makes the loss oscillate or diverge, and a rate that is too low makes training slow or stall.

• ML MathStochastic Gradient Descent (SGD) Variants
• ProblemLearning Rate Sweep
• GuideWhich optimizer to use

A learning rate schedule changes the learning rate during training, commonly with a short linear warmup from near zero followed by cosine or linear decay. Warmup avoids large unstable early steps, and decay lets the model settle into a minimum.

• ML MathStochastic Gradient Descent (SGD) Variants
• ProblemCosine Annealing LR Scheduler
• ProblemWarmup + Linear Decay LR Schedule

Momentum keeps an exponentially decaying running sum of past gradients and steps along it, which damps oscillation and speeds progress along directions that stay consistent. A coefficient of 0.9 is a common default, and Nesterov momentum evaluates the gradient at the look-ahead position.

• ML MathMomentum & Nesterov Acceleration
• ProblemImplement Nesterov Momentum (NAG)

Adam is an adaptive optimizer that keeps running averages of the gradient and of the squared gradient for every parameter, corrects their start-up bias and divides the first by the square root of the second. Common defaults are beta1 of 0.9, beta2 of 0.999 and epsilon of 1e-8.

• ML MathAdaptive Learning Rate Methods (Adam, RMSprop)
• ProblemImplement Adam Optimizer Step
• GuideWhich optimizer to use

Weight decay shrinks every weight a little towards zero at each step, which acts as L2 regularization for plain SGD. With Adam, an L2 term added to the loss gets rescaled by the adaptive denominator, so AdamW applies the decay directly to the weights, decoupled from the gradient.

• ML MathAdaptive Learning Rate Methods (Adam, RMSprop)
• ProblemImplement AdamW (Decoupled Weight Decay)

A critical point is a point where the gradient is zero. It is a strict local minimum when every eigenvalue of the Hessian is positive and a saddle point when the Hessian has both positive and negative eigenvalues, and in high-dimensional networks saddle points are far more common than poor local minima.

• ML MathLocal Minima vs Saddle Points
• ProblemClassify a Critical Point from the Hessian

Linear regression predicts a number as a weighted sum of the features plus a bias and fits the weights by minimizing mean squared error. When X transpose X is invertible, the weights have a closed-form solution called the normal equation.

• ML MathProjections & Least Squares
• ProblemLinear Regression Closed Form

Logistic regression is a linear classifier that passes a weighted sum of the features through the sigmoid to output the probability of the positive class. It is trained by minimizing binary cross-entropy, which is convex in the weights, and its decision boundary is a hyperplane.

• ML MathCross-Entropy Loss
• ProblemLogistic Regression Training Loop

A decision tree predicts by asking a sequence of threshold questions about single features, where each split is chosen to reduce impurity the most. Classification trees usually measure impurity with Gini impurity or entropy, and regression trees use variance.

• ML MathInformation Gain (Decision Trees)
• ProblemDecision Tree Best Split

A random forest trains many decision trees on bootstrap samples of the data and considers only a random subset of features at each split. Averaging or voting across these decorrelated trees reduces variance compared with a single tree.

• ProblemRandom Forest Majority Vote

Gradient boosting builds an ensemble of shallow trees one at a time, fitting each new tree to the negative gradient of the loss with respect to the current predictions. For squared error those negative gradients are the residuals, and each tree's contribution is shrunk by a learning rate.

• ProblemGradient Boosted Regressor

k-nearest neighbors predicts a new point from the k training points closest to it, using a majority vote for classification or an average for regression. It has no training step beyond storing the data, so prediction cost grows with the dataset and feature scaling matters a great deal.

• ProblemKNN Distance + Neighbor Lookup

k-means splits data into k clusters by alternating two steps, assigning each point to its nearest centroid and moving each centroid to the mean of its points. It minimizes the within-cluster sum of squared distances and only finds a local optimum, so initialization such as k-means++ matters.

• ProblemK-Means Assignment Step
• ProblemK-Means Centroid Update

Naive Bayes is a classifier that applies Bayes' theorem with the assumption that features are conditionally independent given the class. The assumption is rarely true, yet the model is fast and works well for text.

• ML MathBayes' Theorem & Inference
• ProblemGaussian Naive Bayes

A support vector machine finds the separating hyperplane with the largest margin to the nearest training points, which are called support vectors. The soft-margin version minimizes hinge loss plus an L2 penalty, and kernels let it learn non-linear boundaries.

• ProblemImplement Hinge Loss (Binary SVM)

A neural network is a stack of layers, each applying a linear map followed by a non-linear activation. A multilayer perceptron, or MLP, is the fully connected version in which every unit in a layer connects to every unit in the next.

• ML MathBackpropagation & Gradient Descent
• ProblemEvaluate a Multi-Layer Perceptron

An activation function is the non-linear function applied after a layer's linear map. Without it, any stack of linear layers collapses into a single linear map, so the network could only learn linear functions.

• ProblemActivation Functions

ReLU, the rectified linear unit, outputs its input when it is positive and zero otherwise. Its gradient is 1 for positive inputs, which helps deep networks train, while units that only ever see negative inputs stop learning, a problem called dead ReLUs.

• ProblemImplement ReLU Activation in NumPy

The sigmoid function squashes any real number into the range 0 to 1, which makes it the standard output for a binary probability. Its derivative is at most 0.25, so stacking many sigmoid layers shrinks gradients.

• ProblemImplement Sigmoid in NumPy

Softmax turns a vector of scores, called logits, into a probability distribution by exponentiating each score and dividing by the sum. Subtracting the maximum logit first leaves the result unchanged and prevents overflow.

• ML MathCross-Entropy Loss
• ProblemImplement Numerically Stable Softmax

Backpropagation computes the gradient of the loss with respect to every weight by applying the chain rule backwards through the network's computational graph. It reuses each intermediate gradient, so one backward pass costs only a small constant multiple of the forward pass.

• ML MathBackpropagation & Gradient Descent
• ML MathComputational Graphs & Backpropagation
• ProblemTrain a Tiny Micrograd MLP

Vanishing and exploding gradients happen when gradients are multiplied through many layers or time steps and shrink towards zero or grow without bound. ReLU-family activations, residual connections, normalization layers, careful initialization and gradient clipping are the usual fixes.

• ML MathChain Rule
• ProblemGradient Clipping (Global Norm)
• GuideTraining loss not decreasing

Weight initialization sets the starting weights so that activations and gradients keep a stable variance from layer to layer. Xavier (Glorot) initialization uses a variance of 2 over fan-in plus fan-out for tanh-like activations, and He initialization uses a variance of 2 over fan-in for ReLU.

• ProblemXavier Initialization

Dropout is a regularizer that sets each activation to zero with probability p during training and scales the surviving activations by 1 over 1 minus p. It is switched off at inference time, and the scaling keeps the expected activation the same in both modes.

• ML MathRegularization (L1, L2, Dropout)
• ProblemImplement Dropout (Training Mode)

Batch normalization standardizes each feature using the mean and variance of the current mini-batch, then applies a learned scale gamma and shift beta. At inference it uses running averages collected during training, so its behavior differs between the two modes.

• ML MathBatch Normalization
• ProblemBatch Normalization (Forward)

Layer normalization standardizes across the features of each single example, then applies a learned scale and shift. It behaves the same at training and inference and does not depend on batch size, which is why transformers use it.

• ML MathBatch Normalization
• ProblemLayer Normalization

A residual connection adds a block's input to its output, so the block learns a correction to the identity. The identity path lets gradients flow straight through very deep networks, which made networks with hundreds of layers trainable.

• ProblemApply an AlphaGo Residual Block

A convolutional neural network slides small learned filters across an input such as an image, reusing the same weights at every position. Weight sharing and local connections make CNNs efficient and translation-equivariant for grid-like data.

• Problem2D Convolution (Image Filtering)

A loss function scores how wrong a prediction is with a single number, and training minimizes its average over the data. The right loss follows from the task and the noise you expect, for example cross-entropy for classification and mean squared error for regression.

• ProblemMean Squared Error (MSE)
• GuideWhich loss function to use

Mean squared error is the average of the squared differences between predictions and targets. Squaring punishes large errors heavily, so MSE is sensitive to outliers, and minimizing it is maximum likelihood under Gaussian noise.

• ProblemMean Squared Error (MSE)

Cross-entropy measures how well a predicted distribution q matches a true distribution p. With one-hot labels it reduces to minus the log probability the model gave the correct class, and binary cross-entropy is the two-class case.

• ML MathCross-Entropy Loss
• ProblemImplement Cross-Entropy Loss in Python

Entropy measures the average uncertainty of a distribution, in bits when the log is base 2. It is zero for a certain outcome and reaches its maximum, log K, for a uniform distribution over K outcomes.

• ML MathShannon Entropy
• ProblemCompute Entropy for a Node

KL divergence measures how much a distribution Q differs from a reference distribution P, as the extra cost of using Q in place of P. It is never negative, equals zero only when the two match and is not symmetric, and cross-entropy equals entropy plus KL divergence.

• ML MathKullback-Leibler (KL) Divergence
• ProblemImplement KL Divergence in Python (NumPy)

Overfitting is when a model learns patterns specific to its training data, including noise, so training error keeps falling while validation error rises. More data, regularization, augmentation, a simpler model or early stopping reduce it.

• ML MathRegularization (L1, L2, Dropout)

Underfitting is when a model is too simple or too briefly trained to capture the real pattern, so both training and validation error stay high. A larger model, better features or longer training are the usual fixes.

• GuideTraining loss not decreasing

The bias-variance tradeoff splits expected squared error into bias squared, variance and irreducible noise. Simple models tend to have high bias and complex models high variance, so model complexity is tuned to minimize their sum.

• ML MathRegularization (L1, L2, Dropout)

Regularization adds a penalty on weight size to the loss to discourage overfitting. An L1 penalty (lasso) drives many weights exactly to zero and so selects features, while an L2 penalty (ridge) shrinks all weights smoothly towards zero.

• ML MathRegularization (L1, L2, Dropout)
• ProblemRidge Regression With Gradient Descent

Early stopping ends training when the validation metric has not improved for a set number of evaluations, called the patience, and keeps the best checkpoint. It limits overfitting and usually shortens training, at the small cost of evaluating on the validation set as training runs.

• ML MathRegularization (L1, L2, Dropout)
• ProblemEarly Stopping

A confusion matrix tabulates predictions against true labels, which for a binary task gives true positives, false positives, false negatives and true negatives. Every threshold-based classification metric is computed from these four counts.

• ProblemCompute Confusion Matrix with Normalization
• GuideWhich evaluation metric to use

Accuracy is the share of predictions that are correct. It is misleading on imbalanced data, because a model that always predicts the majority class scores the majority share.

• ProblemCompute Accuracy, Precision, Recall, F1

Precision is the share of predicted positives that are truly positive. It matters when false positives are costly, such as flagging legitimate transactions as fraud.

• ProblemCompute Accuracy, Precision, Recall, F1

Recall, also called sensitivity or the true positive rate, is the share of actual positives the model finds. It matters when missing a positive is costly, such as a missed diagnosis.

• ProblemCompute Accuracy, Precision, Recall, F1

The F1 score is the harmonic mean of precision and recall, so it is high only when both are high. Macro F1 averages the per-class F1 scores, while micro F1 pools the counts across classes first.

• ProblemImplement Micro-F1

The ROC curve plots the true positive rate against the false positive rate as the decision threshold moves. The area under it, ROC-AUC, equals the probability that a random positive gets a higher score than a random negative, so 0.5 is random ranking and 1.0 is perfect.

• ML MathIntegration & AUC
• ProblemCompute ROC Curve from Scores
• ProblemCompute AUC (Area Under ROC)

PR-AUC is the area under the precision-recall curve, usually computed as average precision. Its random baseline equals the positive rate, which makes it more informative than ROC-AUC when positives are rare.

• ProblemPrecision-Recall Curve & Average Precision

A model is calibrated when its predicted probabilities match observed frequencies, so events given 70 percent happen about 70 percent of the time. Expected calibration error bins predictions by confidence and averages the gap between accuracy and confidence, weighted by bin size.

• ProblemExpected Calibration Error

R-squared is the share of the target's variance that a regression model explains, computed as one minus the residual sum of squares over the total sum of squares. It is 1 for perfect predictions and 0 for always predicting the mean, and it can be negative on new data.

• ProblemImplement R² Score (Coefficient of Determination)

Tokenization splits text into the units a language model reads, which are usually subwords produced by an algorithm such as byte pair encoding. Each token maps to an integer ID, and sequence length, cost and context limits are all counted in tokens.

• LLM InternalsGPT Pretraining
• ProblemTrain a BPE Tokenizer From Scratch in Python

An embedding is a learned dense vector that represents a discrete item such as a token, a product or a user. Items that behave similarly end up close together, so embeddings are compared with cosine similarity for search and retrieval.

• LLM InternalsRetrieval-Augmented Generation (RAG)
• ProblemEmbedding Similarity

Attention lets each position build its output as a weighted average of value vectors, where the weights come from a softmax over query-key dot products. Dividing by the square root of the key dimension keeps the scores in a range where softmax still has useful gradients, and multi-head attention runs several of these in parallel on smaller projections.

• LLM InternalsAttention Mechanism
• ProblemScaled Dot-Product Attention in Python

A transformer is a neural network built from repeated blocks of multi-head self-attention and a position-wise MLP, each wrapped in a residual connection and layer normalization. Decoder-only transformers, which most LLMs use, add a causal mask so each token can attend only to earlier tokens.

• LLM InternalsAttention Mechanism
• LLM InternalsGPT Pretraining
• ProblemTransformer Decoder Block

Attention on its own ignores word order, so a positional encoding injects each token's position. The original transformer adds fixed sinusoids to the embeddings, and rotary position embedding (RoPE) rotates queries and keys so that their dot product depends on relative distance.

• LLM InternalsRoPE (Rotary Position Embedding)
• ProblemImplement Sinusoidal Positional Encoding
• ProblemRoPE: Rotary Query and Key Embeddings

A KV cache stores the key and value vectors of tokens already processed during generation, so each new token computes attention without recomputing the past. It trades memory for speed, and its size grows with the number of layers, KV heads, head dimension and sequence length.

• LLM InternalsKV Cache
• LLM InternalsPagedAttention
• ProblemDecode Attention with a KV Cache

Pretraining trains a model from scratch on a very large unlabeled corpus with a self-supervised objective, which for GPT-style models is predicting the next token with cross-entropy loss. The pretrained model is then adapted to specific tasks by fine-tuning.

• LLM InternalsGPT Pretraining
• ProblemTransformer Training FLOP Estimator

Fine-tuning continues training a pretrained model on a smaller task-specific dataset, usually with a lower learning rate. Full fine-tuning updates every weight, while parameter-efficient methods such as LoRA update only a small number of added parameters.

• LLM InternalsLoRA (Low-Rank Adaptation)
• LLM InternalsQLoRA

LoRA, or low-rank adaptation, freezes a pretrained weight matrix and learns a low-rank update as the product of two small matrices B and A. B starts at zero so training begins exactly at the pretrained model, and the update is scaled by alpha over r.

• LLM InternalsLoRA (Low-Rank Adaptation)

Quantization stores weights or activations in fewer bits, such as INT8 or INT4, by mapping real values to a small integer grid with a scale factor. It cuts memory and can speed up inference at the cost of rounding error.

• LLM InternalsQuantization
• LLM InternalsQLoRA
• ProblemImplement Symmetric INT8 Quantization

Reinforcement learning from human feedback fine-tunes a language model on human preferences in three stages. A supervised fine-tuned model is trained first, a reward model then learns from pairs of preferred and rejected answers, and finally the policy is optimized against that reward with a KL penalty that keeps it close to a reference model, often with PPO.

• LLM InternalsRLHF
• LLM InternalsPPO (Proximal Policy Optimization)
• LLM InternalsDirect Preference Optimization (DPO)
• ProblemBradley-Terry Reward Model Loss
• ProblemImplement the DPO Loss

Retrieval-augmented generation answers a question by first retrieving relevant document chunks, usually by embedding similarity, and then giving them to the language model as context. It lets the model ground its answers in documents outside its training data, such as private or recent ones, without changing its weights.

• LLM InternalsRetrieval-Augmented Generation (RAG)
• ProblemImplement Cosine Similarity in Python (NumPy)

Perplexity is the exponential of the average negative log-likelihood per token, which you can read as the number of equally likely choices the model is effectively picking between. Lower perplexity means the model predicts the text better, and comparisons only make sense under the same tokenizer.

• ML MathPerplexity
• ProblemPerplexity Computation