1 / 1

Matrix Multiplication as a CUDA Kernel

Linear AlgebraGPU Programming

Medium

Implement a CUDA kernel that computes the matrix product $C = A \cdot B$ , where $A$ has shape $(M, K)$ and $B$ has shape $(K, N)$ , producing $C$ of shape $(M, N)$ . For each output element:

C[i, j] = \sum_{k=0}^{K-1} A[i, k] \cdot B[k, j]

All buffers are stored in row-major layout: $\texttt{A}$ has $M \cdot K$ floats, $\texttt{B}$ has $K \cdot N$ floats, and $\texttt{C}$ has $M \cdot N$ floats. The kernel writes one output element per thread into the preallocated $\texttt{C}$ buffer.

Loading visualization...

Examples

Input: M = 2, K = 3, N = 2, A = [[1, 2, 3], [4, 5, 6]], B = [[7, 8], [9, 10], [11, 12]]

Output: C = [[58, 64], [139, 154]]

Each output cell is a dot product of an A row and a B column.

Input: M = 3, K = 2, N = 3, A = [[1, 0], [0, 1], [1, 1]], B = [[2, 3, 4], [5, 6, 7]]

Output: C = [[2, 3, 4], [5, 6, 7], [7, 9, 11]]

Rectangular product where M, K, N all differ.

Hint 1

The inner loop runs over the shared dimension $K$ , which appears in both $\texttt{A}$ and $\texttt{B}$ but not in $\texttt{C}$ .

Hint 2

Each output cell $C[i, j]$ is independent of every other output cell, so one thread can own one $(i, j)$ pair.

Requirements

Use one thread per output element $C[i, j]$
Accumulate the inner product over the shared dimension $K$
Respect row-major layout: $\texttt{A[i, k] = A[i*K + k]}$ and $\texttt{B[k, j] = B[k*N + j]}$
The $\texttt{solve}$ signature must not change

Constraints

$1 \le M, N, K \le 1024$
Input and output buffers are device pointers
Use a 16 by 16 thread block
Values fit in 32-bit float

Try Similar Problems

RMS Normalization as a CUDA KernelMedium Layer Normalization as a CUDA KernelMedium Matrix-Vector Multiplication as a CUDA KernelMedium Outer Product as a CUDA KernelEasy Matrix Addition as a CUDA KernelEasy Hadamard Product as a CUDA KernelEasy Matrix Transpose as a CUDA KernelEasy L1 Normalization as a CUDA KernelMedium Cosine Similarity as a CUDA KernelMedium CUDA Dot Product: Multi-Block ReductionMedium L2 Normalization as a CUDA KernelMedium Tiled Matrix Multiplication as a CUDA KernelHard

Matrix Multiplication as a CUDA Kernel

Linear AlgebraGPU Programming

Medium

Implement a CUDA kernel that computes the matrix product $C = A \cdot B$ , where $A$ has shape $(M, K)$ and $B$ has shape $(K, N)$ , producing $C$ of shape $(M, N)$ . For each output element:

C[i, j] = \sum_{k=0}^{K-1} A[i, k] \cdot B[k, j]

Loading visualization...

Examples

Input: M = 2, K = 3, N = 2, A = [[1, 2, 3], [4, 5, 6]], B = [[7, 8], [9, 10], [11, 12]]

Output: C = [[58, 64], [139, 154]]

Each output cell is a dot product of an A row and a B column.

Input: M = 3, K = 2, N = 3, A = [[1, 0], [0, 1], [1, 1]], B = [[2, 3, 4], [5, 6, 7]]

Output: C = [[2, 3, 4], [5, 6, 7], [7, 9, 11]]

Rectangular product where M, K, N all differ.

Hint 1

The inner loop runs over the shared dimension $K$ , which appears in both $\texttt{A}$ and $\texttt{B}$ but not in $\texttt{C}$ .

Hint 2

Each output cell $C[i, j]$ is independent of every other output cell, so one thread can own one $(i, j)$ pair.

Requirements

Use one thread per output element $C[i, j]$
Accumulate the inner product over the shared dimension $K$
Respect row-major layout: $\texttt{A[i, k] = A[i*K + k]}$ and $\texttt{B[k, j] = B[k*N + j]}$
The $\texttt{solve}$ signature must not change

Constraints

$1 \le M, N, K \le 1024$
Input and output buffers are device pointers
Use a 16 by 16 thread block
Values fit in 32-bit float

Try Similar Problems

A =

Accepts: array

B =

Accepts: array

M =

Accepts: number

N =

Accepts: number

K =

Accepts: number

Examples

Requirements

Constraints

Kernel performance

Matrix Multiplication as a CUDA Kernel

Examples

Requirements

Constraints

Kernel performance