Ridge Regression
Ridge Regression
Ridge regression (L2 regularization) adds a penalty term to the ordinary least squares objective to prevent overfitting. The regularization term shrinks the weights toward zero, which is especially useful when features are correlated or when the number of features is large relative to the number of samples.
Given a feature matrix X, a target vector y, and a regularization parameter lambda, compute the ridge regression weights using the closed-form solution.
Formula
w=(XTX+λI)−1XTyWhere I is the d x d identity matrix and lambda controls the regularization strength.
Examples
Input:
X = [[1, 0], [0, 1]], y = [3, 5], lam = 1.0
Output:
[1.5, 2.5]
X^T X = [[1,0],[0,1]]. Adding lambda*I gives [[2,0],[0,2]]. Inverse is [[0.5,0],[0,0.5]]. X^T y = [3,5]. Result: [1.5, 2.5]. The weights are shrunk from the OLS solution [3, 5] toward zero.
Input:
X = [[1, 1], [1, 2], [1, 3]], y = [3, 5, 7], lam = 0.0
Output:
[1.0, 2.0]
With lambda = 0, ridge regression reduces to ordinary least squares. The perfect fit y = 1 + 2x gives weights [1, 2].
Hint 1
Build a d x d identity matrix I = [[1 if i==j else 0 for j in range(d)] for i in range(d)]. Scale it by lambda and add to X^T X. Then invert and multiply by X^T y, just like ordinary least squares.
Hint 2
The only difference from ordinary least squares is adding lambda * I to X^T X before inverting. This one extra step provides regularization.
Requirements
- Compute X^T X and add lambda * I (identity matrix) to get the regularized matrix
- Compute the inverse of the regularized matrix
- Multiply by X^T y to get the weight vector
- Return a list of floats
Constraints
- X has at least one row and one column
- lambda >= 0
- Return a list of floats with length equal to the number of columns in X
- Time limit: 300 ms
Log in to take notes on this problem
Accepts: array
Accepts: array
Accepts: number
Ridge Regression
Ridge Regression
Ridge regression (L2 regularization) adds a penalty term to the ordinary least squares objective to prevent overfitting. The regularization term shrinks the weights toward zero, which is especially useful when features are correlated or when the number of features is large relative to the number of samples.
Given a feature matrix X, a target vector y, and a regularization parameter lambda, compute the ridge regression weights using the closed-form solution.
Formula
w=(XTX+λI)−1XTyWhere I is the d x d identity matrix and lambda controls the regularization strength.
Examples
Input:
X = [[1, 0], [0, 1]], y = [3, 5], lam = 1.0
Output:
[1.5, 2.5]
X^T X = [[1,0],[0,1]]. Adding lambda*I gives [[2,0],[0,2]]. Inverse is [[0.5,0],[0,0.5]]. X^T y = [3,5]. Result: [1.5, 2.5]. The weights are shrunk from the OLS solution [3, 5] toward zero.
Input:
X = [[1, 1], [1, 2], [1, 3]], y = [3, 5, 7], lam = 0.0
Output:
[1.0, 2.0]
With lambda = 0, ridge regression reduces to ordinary least squares. The perfect fit y = 1 + 2x gives weights [1, 2].
Hint 1
Build a d x d identity matrix I = [[1 if i==j else 0 for j in range(d)] for i in range(d)]. Scale it by lambda and add to X^T X. Then invert and multiply by X^T y, just like ordinary least squares.
Hint 2
The only difference from ordinary least squares is adding lambda * I to X^T X before inverting. This one extra step provides regularization.
Requirements
- Compute X^T X and add lambda * I (identity matrix) to get the regularized matrix
- Compute the inverse of the regularized matrix
- Multiply by X^T y to get the weight vector
- Return a list of floats
Constraints
- X has at least one row and one column
- lambda >= 0
- Return a list of floats with length equal to the number of columns in X
- Time limit: 300 ms
Log in to take notes on this problem
Accepts: array
Accepts: array
Accepts: number