TensorTonic
Problems
CUDA Dot Product: Multi-Block Reduction
Linear AlgebraGPU Programming
Medium

Implement a CUDA kernel that computes the scalar dot product of two equal-length 1-D float vectors AAA and BBB:

dot(A,B)=∑i=0N−1A[i]⋅B[i]\text{dot}(A, B) = \sum_{i=0}^{N-1} A[i] \cdot B[i]dot(A,B)=i=0∑N−1​A[i]⋅B[i]

The result is a single float32\texttt{float32}float32 scalar written to the first slot of the device buffer result\texttt{result}result (a one-element buffer). Because every element contributes to the same output, the kernel must accumulate across all NNN elements using a cross-thread reduction. Do not assume a single block covers NNN: the grid may span many blocks, and partial sums from different blocks must combine into result[0]\texttt{result}[0]result[0].

Loading visualization...

Examples

Input: A = [1, 2, 3, 4], B = [5, 6, 7, 8]

Output: 70.0

Computed as 1⋅5+2⋅6+3⋅7+4⋅8=701{\cdot}5 + 2{\cdot}6 + 3{\cdot}7 + 4{\cdot}8 = 701⋅5+2⋅6+3⋅7+4⋅8=70.

Input: A = [0.5, -1, 2], B = [4, 3, -1]

Output: -3.0

Computed as 0.5⋅4+(−1)⋅3+2⋅(−1)=2−3−2=−30.5{\cdot}4 + (-1){\cdot}3 + 2{\cdot}(-1) = 2 - 3 - 2 = -30.5⋅4+(−1)⋅3+2⋅(−1)=2−3−2=−3.

Hint 1

A clean pattern is a per-block tree reduction in __shared__ memory: each thread folds a strided slice of A[i]*B[i] into a register, writes the partial into shared memory, then participates in log⁡2(blockDim)\log_2(\text{blockDim})log2​(blockDim) pairwise merges under __syncthreads.

Hint 2

To merge across blocks, have thread 0 of each block call atomicAdd(result, block_sum). The launcher in the starter pre-zeros result, so accumulating into it directly is safe.

Hint 3

A pure-atomics alternative is to skip shared memory and have every thread atomicAdd(result, A[i]*B[i]), but contention is much higher; prefer one atomic per block.

Requirements

  • Perform a cross-thread reduction; a single thread cannot read all of AAA and BBB
  • Write the final scalar into result[0]\texttt{result}[0]result[0]
  • Do not assume one block covers NNN; combine partial sums from every block
  • The solve\texttt{solve}solve signature must not change

Constraints

  • 1≤N≤1071 \le N \le 10^71≤N≤107
  • A\texttt{A}A, B\texttt{B}B, and result\texttt{result}result are device pointers
  • All values are 32-bit floats
  • result\texttt{result}result refers to exactly one float\texttt{float}float slot
Try Similar Problems
L1 Normalization as a CUDA KernelMediumL2 Normalization as a CUDA KernelMediumRMS Normalization as a CUDA KernelMediumLayer Normalization as a CUDA KernelMediumMatrix Multiplication as a CUDA KernelMediumCosine Similarity as a CUDA KernelMediumMatrix-Vector Multiplication as a CUDA KernelMediumArgmax as a CUDA KernelMediumMean and Variance as a CUDA KernelMediumArgmin as a CUDA KernelMediumOuter Product as a CUDA KernelEasyTiled Matrix Multiplication as a CUDA KernelHard

Accepts: array

Accepts: array

You must run your code first.

Kernel performance

Submit a correct solution to see your kernel performance.

CUDA Dot Product: Multi-Block Reduction

CUDA Dot Product: Multi-Block Reduction

Linear AlgebraGPU Programming
Medium

Implement a CUDA kernel that computes the scalar dot product of two equal-length 1-D float vectors AAA and BBB:

dot(A,B)=∑i=0N−1A[i]⋅B[i]\text{dot}(A, B) = \sum_{i=0}^{N-1} A[i] \cdot B[i]dot(A,B)=i=0∑N−1​A[i]⋅B[i]

The result is a single float32\texttt{float32}float32 scalar written to the first slot of the device buffer result\texttt{result}result (a one-element buffer). Because every element contributes to the same output, the kernel must accumulate across all NNN elements using a cross-thread reduction. Do not assume a single block covers NNN: the grid may span many blocks, and partial sums from different blocks must combine into result[0]\texttt{result}[0]result[0].

Loading visualization...

Examples

Input: A = [1, 2, 3, 4], B = [5, 6, 7, 8]

Output: 70.0

Computed as 1⋅5+2⋅6+3⋅7+4⋅8=701{\cdot}5 + 2{\cdot}6 + 3{\cdot}7 + 4{\cdot}8 = 701⋅5+2⋅6+3⋅7+4⋅8=70.

Input: A = [0.5, -1, 2], B = [4, 3, -1]

Output: -3.0

Computed as 0.5⋅4+(−1)⋅3+2⋅(−1)=2−3−2=−30.5{\cdot}4 + (-1){\cdot}3 + 2{\cdot}(-1) = 2 - 3 - 2 = -30.5⋅4+(−1)⋅3+2⋅(−1)=2−3−2=−3.

Hint 1

A clean pattern is a per-block tree reduction in __shared__ memory: each thread folds a strided slice of A[i]*B[i] into a register, writes the partial into shared memory, then participates in log⁡2(blockDim)\log_2(\text{blockDim})log2​(blockDim) pairwise merges under __syncthreads.

Hint 2

To merge across blocks, have thread 0 of each block call atomicAdd(result, block_sum). The launcher in the starter pre-zeros result, so accumulating into it directly is safe.

Hint 3

A pure-atomics alternative is to skip shared memory and have every thread atomicAdd(result, A[i]*B[i]), but contention is much higher; prefer one atomic per block.

Requirements

  • Perform a cross-thread reduction; a single thread cannot read all of AAA and BBB
  • Write the final scalar into result[0]\texttt{result}[0]result[0]
  • Do not assume one block covers NNN; combine partial sums from every block
  • The solve\texttt{solve}solve signature must not change

Constraints

  • 1≤N≤1071 \le N \le 10^71≤N≤107
  • A\texttt{A}A, B\texttt{B}B, and result\texttt{result}result are device pointers
  • All values are 32-bit floats
  • result\texttt{result}result refers to exactly one float\texttt{float}float slot
Try Similar Problems
L1 Normalization as a CUDA KernelMediumL2 Normalization as a CUDA KernelMediumRMS Normalization as a CUDA KernelMediumLayer Normalization as a CUDA KernelMediumMatrix Multiplication as a CUDA KernelMediumCosine Similarity as a CUDA KernelMediumMatrix-Vector Multiplication as a CUDA KernelMediumArgmax as a CUDA KernelMediumMean and Variance as a CUDA KernelMediumArgmin as a CUDA KernelMediumOuter Product as a CUDA KernelEasyTiled Matrix Multiplication as a CUDA KernelHard

Accepts: array

Accepts: array

You must run your code first.

Kernel performance

Submit a correct solution to see your kernel performance.

Problems
Loading...
1 / 1