TensorTonic
Problems
3D Convolution as a CUDA Kernel
Computer VisionGPU Programming
Hard

Implement a CUDA kernel for naive 3D cross-correlation on a single-channel input volume with a single-channel kernel. There is no kernel flip and no zero padding. Stride is fixed at 1.

For each output position (d,i,j)(d, i, j)(d,i,j):

output[d,i,j]=∑a=0kD−1∑b=0kH−1∑c=0kW−1input[d+a,i+b,j+c]⋅kernel[a,b,c]\text{output}[d, i, j] = \sum_{a=0}^{kD-1} \sum_{b=0}^{kH-1} \sum_{c=0}^{kW-1} \text{input}[d + a, i + b, j + c] \cdot \text{kernel}[a, b, c]output[d,i,j]=a=0∑kD−1​b=0∑kH−1​c=0∑kW−1​input[d+a,i+b,j+c]⋅kernel[a,b,c]

The input volume has shape (D,H,W)(D, H, W)(D,H,W) and the kernel has shape (kD,kH,kW)(kD, kH, kW)(kD,kH,kW). With valid padding and stride 1, the output shape is (D−kD+1)×(H−kH+1)×(W−kW+1)(D - kD + 1) \times (H - kH + 1) \times (W - kW + 1)(D−kD+1)×(H−kH+1)×(W−kW+1).

All tensors are row-major flat. The input is D⋅H⋅WD \cdot H \cdot WD⋅H⋅W floats indexed as d⋅H⋅W+i⋅W+jd \cdot H \cdot W + i \cdot W + jd⋅H⋅W+i⋅W+j. The kernel is kD⋅kH⋅kWkD \cdot kH \cdot kWkD⋅kH⋅kW floats indexed as a⋅kH⋅kW+b⋅kW+ca \cdot kH \cdot kW + b \cdot kW + ca⋅kH⋅kW+b⋅kW+c. The output is outD⋅outH⋅outW\texttt{outD} \cdot \texttt{outH} \cdot \texttt{outW}outD⋅outH⋅outW floats indexed as d⋅outH⋅outW+i⋅outW+jd \cdot \texttt{outH} \cdot \texttt{outW} + i \cdot \texttt{outW} + jd⋅outH⋅outW+i⋅outW+j. The kernel writes one element of output\texttt{output}output per thread.

Loading visualization...

Examples

Input: D = 2, H = 2, W = 2, kD = 2, kH = 2, kW = 2, input = [[[1, 2], [3, 4]], [[5, 6], [7, 8]]], kernel = [[[1, 0], [0, 1]], [[0, 1], [1, 0]]]

Output: [[[18]]]

Single scalar output. Matching the four nonzero kernel positions gives 1⋅1+4⋅1+6⋅1+7⋅1=181{\cdot}1 + 4{\cdot}1 + 6{\cdot}1 + 7{\cdot}1 = 181⋅1+4⋅1+6⋅1+7⋅1=18.

Input: D = 2, H = 3, W = 3, kD = 1, kH = 2, kW = 2, input = [[[1, 2, 3], [4, 5, 6], [7, 8, 9]], [[10, 11, 12], [13, 14, 15], [16, 17, 18]]], kernel = [[[1, 1], [1, 1]]]

Output: [[[12, 16], [24, 28]], [[48, 52], [60, 64]]]

Output shape is 2×2×22 \times 2 \times 22×2×2. Each entry sums a 2×22 \times 22×2 window from one depth slice.

Hint 1

Use a 3D thread block so each thread maps to one (d,i,j)(d, i, j)(d,i,j) in the output volume. Recover indices from blockIdx and threadIdx along all three axes.

Hint 2

Inside the kernel, run a triple-nested loop over (a,b,c)(a, b, c)(a,b,c) in (kD,kH,kW)(kD, kH, kW)(kD,kH,kW) to accumulate input[d+a,i+b,j+c]⋅kernel[a,b,c]\texttt{input}[d+a, i+b, j+c] \cdot \texttt{kernel}[a, b, c]input[d+a,i+b,j+c]⋅kernel[a,b,c] into a local float before writing once to global memory.

Hint 3

Remember row-major flatten: indexing input\texttt{input}input uses H⋅WH \cdot WH⋅W stride along depth and WWW stride along height; output\texttt{output}output uses outH⋅outW\texttt{outH} \cdot \texttt{outW}outH⋅outW and outW\texttt{outW}outW.

Requirements

  • Assign one thread per output element using a 3D grid and a 3D block
  • Accumulate the dot product with a triple-nested loop over (kD,kH,kW)(kD, kH, kW)(kD,kH,kW)
  • Respect row-major flatten with outer dimension DDD for input and outer dimension outD\texttt{outD}outD for output
  • The solve\texttt{solve}solve signature must not change

Constraints

  • 1≤D,H,W≤641 \le D, H, W \le 641≤D,H,W≤64
  • 1≤kD≤D1 \le kD \le D1≤kD≤D, 1≤kH≤H1 \le kH \le H1≤kH≤H, 1≤kW≤W1 \le kW \le W1≤kW≤W
  • Input, kernel, and output buffers are device pointers
  • Launch with dim3 threads(8, 8, 8)\texttt{dim3 threads(8, 8, 8)}dim3 threads(8, 8, 8)
  • Values fit in 32-bit float; operation is cross-correlation, no kernel flip
Try Similar Problems
2D Convolution as a CUDA KernelMediumMax Pool 2D as a CUDA KernelMedium1D Convolution as a CUDA KernelEasyTiled Matrix Multiplication as a CUDA KernelHardScaled Dot-Product Attention in CUDAHardArgmax as a CUDA KernelMediumRMS Normalization as a CUDA KernelMediumMean and Variance as a CUDA KernelMediumLayer Normalization as a CUDA KernelMediumSoftmax as a CUDA KernelMediumMatrix Multiplication as a CUDA KernelMediumL1 Normalization as a CUDA KernelMedium

Accepts: array

Accepts: array

Accepts: number

Accepts: number

Accepts: number

Accepts: number

Accepts: number

Accepts: number

You must run your code first.

Kernel performance

Submit a correct solution to see your kernel performance.

3D Convolution as a CUDA Kernel

3D Convolution as a CUDA Kernel

Computer VisionGPU Programming
Hard

Implement a CUDA kernel for naive 3D cross-correlation on a single-channel input volume with a single-channel kernel. There is no kernel flip and no zero padding. Stride is fixed at 1.

For each output position (d,i,j)(d, i, j)(d,i,j):

output[d,i,j]=∑a=0kD−1∑b=0kH−1∑c=0kW−1input[d+a,i+b,j+c]⋅kernel[a,b,c]\text{output}[d, i, j] = \sum_{a=0}^{kD-1} \sum_{b=0}^{kH-1} \sum_{c=0}^{kW-1} \text{input}[d + a, i + b, j + c] \cdot \text{kernel}[a, b, c]output[d,i,j]=a=0∑kD−1​b=0∑kH−1​c=0∑kW−1​input[d+a,i+b,j+c]⋅kernel[a,b,c]

The input volume has shape (D,H,W)(D, H, W)(D,H,W) and the kernel has shape (kD,kH,kW)(kD, kH, kW)(kD,kH,kW). With valid padding and stride 1, the output shape is (D−kD+1)×(H−kH+1)×(W−kW+1)(D - kD + 1) \times (H - kH + 1) \times (W - kW + 1)(D−kD+1)×(H−kH+1)×(W−kW+1).

All tensors are row-major flat. The input is D⋅H⋅WD \cdot H \cdot WD⋅H⋅W floats indexed as d⋅H⋅W+i⋅W+jd \cdot H \cdot W + i \cdot W + jd⋅H⋅W+i⋅W+j. The kernel is kD⋅kH⋅kWkD \cdot kH \cdot kWkD⋅kH⋅kW floats indexed as a⋅kH⋅kW+b⋅kW+ca \cdot kH \cdot kW + b \cdot kW + ca⋅kH⋅kW+b⋅kW+c. The output is outD⋅outH⋅outW\texttt{outD} \cdot \texttt{outH} \cdot \texttt{outW}outD⋅outH⋅outW floats indexed as d⋅outH⋅outW+i⋅outW+jd \cdot \texttt{outH} \cdot \texttt{outW} + i \cdot \texttt{outW} + jd⋅outH⋅outW+i⋅outW+j. The kernel writes one element of output\texttt{output}output per thread.

Loading visualization...

Examples

Input: D = 2, H = 2, W = 2, kD = 2, kH = 2, kW = 2, input = [[[1, 2], [3, 4]], [[5, 6], [7, 8]]], kernel = [[[1, 0], [0, 1]], [[0, 1], [1, 0]]]

Output: [[[18]]]

Single scalar output. Matching the four nonzero kernel positions gives 1⋅1+4⋅1+6⋅1+7⋅1=181{\cdot}1 + 4{\cdot}1 + 6{\cdot}1 + 7{\cdot}1 = 181⋅1+4⋅1+6⋅1+7⋅1=18.

Input: D = 2, H = 3, W = 3, kD = 1, kH = 2, kW = 2, input = [[[1, 2, 3], [4, 5, 6], [7, 8, 9]], [[10, 11, 12], [13, 14, 15], [16, 17, 18]]], kernel = [[[1, 1], [1, 1]]]

Output: [[[12, 16], [24, 28]], [[48, 52], [60, 64]]]

Output shape is 2×2×22 \times 2 \times 22×2×2. Each entry sums a 2×22 \times 22×2 window from one depth slice.

Hint 1

Use a 3D thread block so each thread maps to one (d,i,j)(d, i, j)(d,i,j) in the output volume. Recover indices from blockIdx and threadIdx along all three axes.

Hint 2

Inside the kernel, run a triple-nested loop over (a,b,c)(a, b, c)(a,b,c) in (kD,kH,kW)(kD, kH, kW)(kD,kH,kW) to accumulate input[d+a,i+b,j+c]⋅kernel[a,b,c]\texttt{input}[d+a, i+b, j+c] \cdot \texttt{kernel}[a, b, c]input[d+a,i+b,j+c]⋅kernel[a,b,c] into a local float before writing once to global memory.

Hint 3

Remember row-major flatten: indexing input\texttt{input}input uses H⋅WH \cdot WH⋅W stride along depth and WWW stride along height; output\texttt{output}output uses outH⋅outW\texttt{outH} \cdot \texttt{outW}outH⋅outW and outW\texttt{outW}outW.

Requirements

  • Assign one thread per output element using a 3D grid and a 3D block
  • Accumulate the dot product with a triple-nested loop over (kD,kH,kW)(kD, kH, kW)(kD,kH,kW)
  • Respect row-major flatten with outer dimension DDD for input and outer dimension outD\texttt{outD}outD for output
  • The solve\texttt{solve}solve signature must not change

Constraints

  • 1≤D,H,W≤641 \le D, H, W \le 641≤D,H,W≤64
  • 1≤kD≤D1 \le kD \le D1≤kD≤D, 1≤kH≤H1 \le kH \le H1≤kH≤H, 1≤kW≤W1 \le kW \le W1≤kW≤W
  • Input, kernel, and output buffers are device pointers
  • Launch with dim3 threads(8, 8, 8)\texttt{dim3 threads(8, 8, 8)}dim3 threads(8, 8, 8)
  • Values fit in 32-bit float; operation is cross-correlation, no kernel flip
Try Similar Problems
2D Convolution as a CUDA KernelMediumMax Pool 2D as a CUDA KernelMedium1D Convolution as a CUDA KernelEasyTiled Matrix Multiplication as a CUDA KernelHardScaled Dot-Product Attention in CUDAHardArgmax as a CUDA KernelMediumRMS Normalization as a CUDA KernelMediumMean and Variance as a CUDA KernelMediumLayer Normalization as a CUDA KernelMediumSoftmax as a CUDA KernelMediumMatrix Multiplication as a CUDA KernelMediumL1 Normalization as a CUDA KernelMedium

Accepts: array

Accepts: array

Accepts: number

Accepts: number

Accepts: number

Accepts: number

Accepts: number

Accepts: number

You must run your code first.

Kernel performance

Submit a correct solution to see your kernel performance.

Problems
Loading...
1 / 1