Principal Component Analysis (PCA) finds the directions of maximum variance in the data and projects it onto a lower-dimensional subspace. This is done by computing the eigenvectors of the covariance matrix and projecting onto the top-k eigenvectors (principal components).
Given a data matrix X (n samples, d features) and the number of components k, project the data onto its top-k principal components.
Center the data by subtracting the mean of each feature
Compute the d x d covariance matrix using sample covariance (divide by n-1)
Find the top-k eigenvectors of C sorted by eigenvalue in descending order (e.g., using power iteration with deflation)
Project the centered data onto these eigenvectors
Where W is the d x k matrix whose columns are the top-k eigenvectors.
Return an n by k list of projected values.
Input: X = [[1, 0], [2, 0], [3, 0], [4, 0], [5, 0]], k = 1
Output: [[-2.0], [-1.0], [0.0], [1.0], [2.0]]
Explanation: Centering leaves all variance along the first feature.
Input: X = [[1, 1], [2, 2], [3, 3], [4, 4], [5, 5]], k = 1
Output: [[-2.8284], [-1.4142], [0.0], [1.4142], [2.8284]]
Center each feature before forming the sample covariance matrix.
Use np.linalg.eigh, sort eigenvectors by descending eigenvalue, then project onto the first k columns.
Accepts: array
Accepts: number
Principal Component Analysis (PCA) finds the directions of maximum variance in the data and projects it onto a lower-dimensional subspace. This is done by computing the eigenvectors of the covariance matrix and projecting onto the top-k eigenvectors (principal components).
Given a data matrix X (n samples, d features) and the number of components k, project the data onto its top-k principal components.
Center the data by subtracting the mean of each feature
Compute the d x d covariance matrix using sample covariance (divide by n-1)
Find the top-k eigenvectors of C sorted by eigenvalue in descending order (e.g., using power iteration with deflation)
Project the centered data onto these eigenvectors
Where W is the d x k matrix whose columns are the top-k eigenvectors.
Return an n by k list of projected values.
Input: X = [[1, 0], [2, 0], [3, 0], [4, 0], [5, 0]], k = 1
Output: [[-2.0], [-1.0], [0.0], [1.0], [2.0]]
Explanation: Centering leaves all variance along the first feature.
Input: X = [[1, 1], [2, 2], [3, 3], [4, 4], [5, 5]], k = 1
Output: [[-2.8284], [-1.4142], [0.0], [1.4142], [2.8284]]
Center each feature before forming the sample covariance matrix.
Use np.linalg.eigh, sort eigenvectors by descending eigenvalue, then project onto the first k columns.
Accepts: array
Accepts: number