TensorTonic
Problems
Tiled Matrix Multiplication as a CUDA Kernel
Linear AlgebraGPU Programming
Hard

Re-implement matrix multiplication using a shared-memory tiled algorithm. Each thread block cooperatively loads tiles of AAA and BBB into shared memory, then accumulates partial dot products from those tiles. This reduces global-memory traffic by a factor of TILE_DIM\texttt{TILE\_DIM}TILE_DIM compared to the naive version. The operation is the same GEMM as the naive variant: AAA has shape (M,K)(M, K)(M,K), BBB has shape (K,N)(K, N)(K,N), and the kernel writes CCC of shape (M,N)(M, N)(M,N) where:

C[i,j]=∑k=0K−1A[i,k]⋅B[k,j]C[i, j] = \sum_{k=0}^{K-1} A[i, k] \cdot B[k, j]C[i,j]=k=0∑K−1​A[i,k]⋅B[k,j]

All buffers use row-major layout. Each thread block computes a TILE_DIM×TILE_DIM\texttt{TILE\_DIM} \times \texttt{TILE\_DIM}TILE_DIM×TILE_DIM output tile of CCC by sweeping across the shared dimension KKK in tile-sized chunks, loading one AAA tile and one BBB tile per chunk into shared memory and accumulating C[i,j]C[i, j]C[i,j] in a register. The grader checks correctness, but the naive matmul will be too slow for the larger shapes, so a tiled implementation is required.

Loading visualization...

Examples

Input: M = 2, K = 3, N = 2, A = [[1, 2, 3], [4, 5, 6]], B = [[7, 8], [9, 10], [11, 12]]

Output: C = [[58, 64], [139, 154]]

A single 16×1616 \times 1616×16 block computes the entire output and the inner loop performs one tile iteration along KKK.

Input: M = 3, K = 2, N = 3, A = [[1, 2], [3, 4], [5, 6]], B = [[1, 2, 3], [4, 5, 6]]

Output: C = [[9, 12, 15], [19, 26, 33], [29, 40, 51]]

Rectangular product with partial tiles in all three dimensions; out-of-range tile loads must write 000 to shared memory.

Hint 1

In each tile step every thread in the block loads one element of the AAA tile and one element of the BBB tile from global memory into shared memory. The thread that owns output (i,j)(i, j)(i,j) is a natural choice for loading A[i, k0 + threadIdx.x]\texttt{A[i, k0 + threadIdx.x]}A[i, k0 + threadIdx.x] and B[k0 + threadIdx.y, j]\texttt{B[k0 + threadIdx.y, j]}B[k0 + threadIdx.y, j].

Hint 2

Two __syncthreads are needed: one after loading the tile so every thread can safely read it, and one after the inner accumulation so no thread overwrites the tile while another is still reading it.

Hint 3

For partial tiles, write 000 to shared memory when the global index is out of range. Skipping the write leaves stale values from the previous iteration in shared memory and corrupts the accumulation.

Requirements

  • Use a shared-memory tiled algorithm with TILE_DIM=16\texttt{TILE\_DIM} = 16TILE_DIM=16
  • Accumulate over KKK in tile-sized chunks, loading one AAA tile and one BBB tile per iteration into shared memory
  • Call __syncthreads()\texttt{\_\_syncthreads()}__syncthreads() twice per tile iteration: once after the cooperative load, once after the accumulation
  • Handle partial tiles when MMM, NNN, or KKK are not multiples of TILE_DIM\texttt{TILE\_DIM}TILE_DIM by writing 000 to shared memory for out-of-range indices
  • The solve\texttt{solve}solve signature must not change

Constraints

  • 1≤M,N,K≤20481 \le M, N, K \le 20481≤M,N,K≤2048
  • Input and output buffers are device pointers
  • Use a 16×1616 \times 1616×16 thread block (TILE_DIM=16\texttt{TILE\_DIM} = 16TILE_DIM=16)
  • Values fit in 32-bit float
Try Similar Problems
Scaled Dot-Product Attention in CUDAHardRMS Normalization as a CUDA KernelMediumLayer Normalization as a CUDA KernelMediumMatrix Multiplication as a CUDA KernelMediumL1 Normalization as a CUDA KernelMediumCosine Similarity as a CUDA KernelMediumCUDA Dot Product: Multi-Block ReductionMediumL2 Normalization as a CUDA KernelMediumMatrix-Vector Multiplication as a CUDA KernelMediumSplit-K Matmul in TritonHardGrouped Program-ID Matmul in TritonHardFused Tiled Matmul and ReLUHard

Accepts: array

Accepts: array

You must run your code first.

Kernel performance

Submit a correct solution to see your kernel performance.

Tiled Matrix Multiplication as a CUDA Kernel

Tiled Matrix Multiplication as a CUDA Kernel

Linear AlgebraGPU Programming
Hard

Re-implement matrix multiplication using a shared-memory tiled algorithm. Each thread block cooperatively loads tiles of AAA and BBB into shared memory, then accumulates partial dot products from those tiles. This reduces global-memory traffic by a factor of TILE_DIM\texttt{TILE\_DIM}TILE_DIM compared to the naive version. The operation is the same GEMM as the naive variant: AAA has shape (M,K)(M, K)(M,K), BBB has shape (K,N)(K, N)(K,N), and the kernel writes CCC of shape (M,N)(M, N)(M,N) where:

C[i,j]=∑k=0K−1A[i,k]⋅B[k,j]C[i, j] = \sum_{k=0}^{K-1} A[i, k] \cdot B[k, j]C[i,j]=k=0∑K−1​A[i,k]⋅B[k,j]

All buffers use row-major layout. Each thread block computes a TILE_DIM×TILE_DIM\texttt{TILE\_DIM} \times \texttt{TILE\_DIM}TILE_DIM×TILE_DIM output tile of CCC by sweeping across the shared dimension KKK in tile-sized chunks, loading one AAA tile and one BBB tile per chunk into shared memory and accumulating C[i,j]C[i, j]C[i,j] in a register. The grader checks correctness, but the naive matmul will be too slow for the larger shapes, so a tiled implementation is required.

Loading visualization...

Examples

Input: M = 2, K = 3, N = 2, A = [[1, 2, 3], [4, 5, 6]], B = [[7, 8], [9, 10], [11, 12]]

Output: C = [[58, 64], [139, 154]]

A single 16×1616 \times 1616×16 block computes the entire output and the inner loop performs one tile iteration along KKK.

Input: M = 3, K = 2, N = 3, A = [[1, 2], [3, 4], [5, 6]], B = [[1, 2, 3], [4, 5, 6]]

Output: C = [[9, 12, 15], [19, 26, 33], [29, 40, 51]]

Rectangular product with partial tiles in all three dimensions; out-of-range tile loads must write 000 to shared memory.

Hint 1

In each tile step every thread in the block loads one element of the AAA tile and one element of the BBB tile from global memory into shared memory. The thread that owns output (i,j)(i, j)(i,j) is a natural choice for loading A[i, k0 + threadIdx.x]\texttt{A[i, k0 + threadIdx.x]}A[i, k0 + threadIdx.x] and B[k0 + threadIdx.y, j]\texttt{B[k0 + threadIdx.y, j]}B[k0 + threadIdx.y, j].

Hint 2

Two __syncthreads are needed: one after loading the tile so every thread can safely read it, and one after the inner accumulation so no thread overwrites the tile while another is still reading it.

Hint 3

For partial tiles, write 000 to shared memory when the global index is out of range. Skipping the write leaves stale values from the previous iteration in shared memory and corrupts the accumulation.

Requirements

  • Use a shared-memory tiled algorithm with TILE_DIM=16\texttt{TILE\_DIM} = 16TILE_DIM=16
  • Accumulate over KKK in tile-sized chunks, loading one AAA tile and one BBB tile per iteration into shared memory
  • Call __syncthreads()\texttt{\_\_syncthreads()}__syncthreads() twice per tile iteration: once after the cooperative load, once after the accumulation
  • Handle partial tiles when MMM, NNN, or KKK are not multiples of TILE_DIM\texttt{TILE\_DIM}TILE_DIM by writing 000 to shared memory for out-of-range indices
  • The solve\texttt{solve}solve signature must not change

Constraints

  • 1≤M,N,K≤20481 \le M, N, K \le 20481≤M,N,K≤2048
  • Input and output buffers are device pointers
  • Use a 16×1616 \times 1616×16 thread block (TILE_DIM=16\texttt{TILE\_DIM} = 16TILE_DIM=16)
  • Values fit in 32-bit float
Try Similar Problems
Scaled Dot-Product Attention in CUDAHardRMS Normalization as a CUDA KernelMediumLayer Normalization as a CUDA KernelMediumMatrix Multiplication as a CUDA KernelMediumL1 Normalization as a CUDA KernelMediumCosine Similarity as a CUDA KernelMediumCUDA Dot Product: Multi-Block ReductionMediumL2 Normalization as a CUDA KernelMediumMatrix-Vector Multiplication as a CUDA KernelMediumSplit-K Matmul in TritonHardGrouped Program-ID Matmul in TritonHardFused Tiled Matmul and ReLUHard

Accepts: array

Accepts: array

You must run your code first.

Kernel performance

Submit a correct solution to see your kernel performance.

Problems
Loading...
1 / 1