A linear (fully connected) layer is the most fundamental building block in neural networks. It transforms an input by multiplying with a weight matrix and adding a bias vector. This operation is also called a dense layer or an affine transformation.
Given an input matrix X (n samples, d_in features), a weight matrix W (d_in x d_out), and a bias vector b (d_out), compute the linear layer output.
Where b is broadcast (added to every row of XW).
Input:
X = [[1, 2], [3, 4]], W = [[1, 0], [0, 1]], b = [0, 0]
Output:
[[1, 2], [3, 4]]
With W as the identity matrix and zero bias, the output equals the input.
Input:
X = [[1, 2]], W = [[1], [2]], b = [3]
Output:
[[8]]
11 + 22 = 5, plus bias 3 gives 8. The layer reduces from 2 features to 1 output.
For each output element Y[i][j], compute the dot product of row i of X with column j of W, then add b[j]. Use nested loops: outer over samples, inner over output dimensions, innermost over input dimensions.
The matrix multiplication X @ W produces an n x d_out matrix. Then add b[j] to every element in column j. In Python: Y[i][j] = sum(X[i][k] * W[k][j] for k in range(d_in)) + b[j].
Sign in to take notes on this problem
Accepts: array
Accepts: array
Accepts: array
A linear (fully connected) layer is the most fundamental building block in neural networks. It transforms an input by multiplying with a weight matrix and adding a bias vector. This operation is also called a dense layer or an affine transformation.
Given an input matrix X (n samples, d_in features), a weight matrix W (d_in x d_out), and a bias vector b (d_out), compute the linear layer output.
Where b is broadcast (added to every row of XW).
Input:
X = [[1, 2], [3, 4]], W = [[1, 0], [0, 1]], b = [0, 0]
Output:
[[1, 2], [3, 4]]
With W as the identity matrix and zero bias, the output equals the input.
Input:
X = [[1, 2]], W = [[1], [2]], b = [3]
Output:
[[8]]
11 + 22 = 5, plus bias 3 gives 8. The layer reduces from 2 features to 1 output.
For each output element Y[i][j], compute the dot product of row i of X with column j of W, then add b[j]. Use nested loops: outer over samples, inner over output dimensions, innermost over input dimensions.
The matrix multiplication X @ W produces an n x d_out matrix. Then add b[j] to every element in column j. In Python: Y[i][j] = sum(X[i][k] * W[k][j] for k in range(d_in)) + b[j].
Sign in to take notes on this problem
Accepts: array
Accepts: array
Accepts: array