TensorTonic
Problems
L2 Normalization as a CUDA Kernel
Linear AlgebraGPU Programming
Medium

Implement L2 normalization of a 1-D vector of 32-bit floats. For each element of the input array, compute:

output[i]=input[i]∑j=0N−1input[j]2\text{output}[i] = \frac{\text{input}[i]}{\sqrt{\sum_{j=0}^{N-1} \text{input}[j]^2}}output[i]=∑j=0N−1​input[j]2​input[i]​

The kernel must first reduce the squared values into a single scalar, then take the square root before dividing every element by that scalar. A single embarrassingly-parallel kernel cannot compute this because the denominator depends on every element.

The output array has the same length as the input.

Loading visualization...

Examples

Input: input = [3.0, 4.0]

Output: [0.6, 0.8]

The L2 norm is 9+16=5\sqrt{9 + 16} = 59+16​=5, so each element is divided by 555.

Input: input = [-1.0, 2.0, -2.0]

Output: [-0.3333, 0.6667, -0.6667]

The L2 norm is 1+4+4=3\sqrt{1 + 4 + 4} = 31+4+4​=3.

Hint 1

You need one whole-array reduction: the sum of squares. A single thread cannot see all elements, so use a shared-memory tree reduction across one block with a grid-stride loop.

Hint 2

Two kernels is the cleanest split: one writes the scalar ∑jinput[j]2\sum_j \text{input}[j]^2∑j​input[j]2 to a single-float scratch buffer, the second reads that scratch, computes ⋅\sqrt{\cdot}⋅​ once per thread, and divides input[i]\texttt{input}[i]input[i] by it into output[i]\texttt{output}[i]output[i].

Hint 3

Use cudaMalloc\texttt{cudaMalloc}cudaMalloc inside solve\texttt{solve}solve for the scratch float, call cudaDeviceSynchronize\texttt{cudaDeviceSynchronize}cudaDeviceSynchronize before cudaFree\texttt{cudaFree}cudaFree, and remember rsqrtf\texttt{rsqrtf}rsqrtf as a fast alternative to 1/sqrtf1 / \texttt{sqrtf}1/sqrtf.

Requirements

  • Reduce the squared values to a single scalar before normalizing
  • Divide every element by ∑jinput[j]2\sqrt{\sum_j \text{input}[j]^2}∑j​input[j]2​
  • Allocate and free any scratch device memory inside solve\texttt{solve}solve
  • The solve\texttt{solve}solve signature must not change
  • Results must populate the output\texttt{output}output array

Constraints

  • 1≤N≤1071 \le N \le 10^71≤N≤107
  • Input and output buffers are device pointers
  • Values fit in 32-bit float
  • The input vector has nonzero norm
Try Similar Problems
L1 Normalization as a CUDA KernelMediumCUDA Dot Product: Multi-Block ReductionMediumRMS Normalization as a CUDA KernelMediumLayer Normalization as a CUDA KernelMediumMatrix Multiplication as a CUDA KernelMediumCosine Similarity as a CUDA KernelMediumMatrix-Vector Multiplication as a CUDA KernelMediumArgmax as a CUDA KernelMediumMean and Variance as a CUDA KernelMediumArgmin as a CUDA KernelMediumOuter Product as a CUDA KernelEasyTiled Matrix Multiplication as a CUDA KernelHard

Accepts: array

You must run your code first.

Kernel performance

Submit a correct solution to see your kernel performance.

L2 Normalization as a CUDA Kernel

L2 Normalization as a CUDA Kernel

Linear AlgebraGPU Programming
Medium

Implement L2 normalization of a 1-D vector of 32-bit floats. For each element of the input array, compute:

output[i]=input[i]∑j=0N−1input[j]2\text{output}[i] = \frac{\text{input}[i]}{\sqrt{\sum_{j=0}^{N-1} \text{input}[j]^2}}output[i]=∑j=0N−1​input[j]2​input[i]​

The kernel must first reduce the squared values into a single scalar, then take the square root before dividing every element by that scalar. A single embarrassingly-parallel kernel cannot compute this because the denominator depends on every element.

The output array has the same length as the input.

Loading visualization...

Examples

Input: input = [3.0, 4.0]

Output: [0.6, 0.8]

The L2 norm is 9+16=5\sqrt{9 + 16} = 59+16​=5, so each element is divided by 555.

Input: input = [-1.0, 2.0, -2.0]

Output: [-0.3333, 0.6667, -0.6667]

The L2 norm is 1+4+4=3\sqrt{1 + 4 + 4} = 31+4+4​=3.

Hint 1

You need one whole-array reduction: the sum of squares. A single thread cannot see all elements, so use a shared-memory tree reduction across one block with a grid-stride loop.

Hint 2

Two kernels is the cleanest split: one writes the scalar ∑jinput[j]2\sum_j \text{input}[j]^2∑j​input[j]2 to a single-float scratch buffer, the second reads that scratch, computes ⋅\sqrt{\cdot}⋅​ once per thread, and divides input[i]\texttt{input}[i]input[i] by it into output[i]\texttt{output}[i]output[i].

Hint 3

Use cudaMalloc\texttt{cudaMalloc}cudaMalloc inside solve\texttt{solve}solve for the scratch float, call cudaDeviceSynchronize\texttt{cudaDeviceSynchronize}cudaDeviceSynchronize before cudaFree\texttt{cudaFree}cudaFree, and remember rsqrtf\texttt{rsqrtf}rsqrtf as a fast alternative to 1/sqrtf1 / \texttt{sqrtf}1/sqrtf.

Requirements

  • Reduce the squared values to a single scalar before normalizing
  • Divide every element by ∑jinput[j]2\sqrt{\sum_j \text{input}[j]^2}∑j​input[j]2​
  • Allocate and free any scratch device memory inside solve\texttt{solve}solve
  • The solve\texttt{solve}solve signature must not change
  • Results must populate the output\texttt{output}output array

Constraints

  • 1≤N≤1071 \le N \le 10^71≤N≤107
  • Input and output buffers are device pointers
  • Values fit in 32-bit float
  • The input vector has nonzero norm
Try Similar Problems
L1 Normalization as a CUDA KernelMediumCUDA Dot Product: Multi-Block ReductionMediumRMS Normalization as a CUDA KernelMediumLayer Normalization as a CUDA KernelMediumMatrix Multiplication as a CUDA KernelMediumCosine Similarity as a CUDA KernelMediumMatrix-Vector Multiplication as a CUDA KernelMediumArgmax as a CUDA KernelMediumMean and Variance as a CUDA KernelMediumArgmin as a CUDA KernelMediumOuter Product as a CUDA KernelEasyTiled Matrix Multiplication as a CUDA KernelHard

Accepts: array

You must run your code first.

Kernel performance

Submit a correct solution to see your kernel performance.

Problems
Loading...
1 / 1