TensorTonic
Problems
Single-Pass Mean and Variance in Triton
Probability and StatisticsGPU Programming
Medium

Compute the population mean and variance of a 1D float tensor in a single pass over the data. For a tensor xxx of length NNN, write the scalar mean and variance into mean_out[0]\texttt{mean\_out}[0]mean_out[0] and var_out[0]\texttt{var\_out}[0]var_out[0]:

μ=1N∑i=0N−1x[i],σ2=1N∑i=0N−1x[i]2−μ2\mu = \frac{1}{N} \sum_{i=0}^{N-1} x[i], \qquad \sigma^2 = \frac{1}{N} \sum_{i=0}^{N-1} x[i]^2 - \mu^2μ=N1​i=0∑N−1​x[i],σ2=N1​i=0∑N−1​x[i]2−μ2

This problem teaches two-statistic fusion: each program accumulates ∑x\sum x∑x and ∑x2\sum x^2∑x2 for its tile in the same reduction pass, then atomic-adds both partials into scratch buffers. The host finalizes by dividing and applying the E[x2]−E[x]2E[x^2] - E[x]^2E[x2]−E[x]2 identity.

Loading visualization...

Examples

Input: x = [1.0, 2.0, 3.0, 4.0]

Output: mean = [2.5], var = [1.25]

Input: x = [0.0, 0.0, 0.0, 0.0]

Output: mean = [0.0], var = [0.0]

Hint 1

Use the identity σ2=E[x2]−μ2\sigma^2 = E[x^2] - \mu^2σ2=E[x2]−μ2 so one pass over xxx produces both statistics. Reduce tl.sum(x)\texttt{tl.sum(x)}tl.sum(x) and tl.sum(x * x)\texttt{tl.sum(x * x)}tl.sum(x * x) in the same tile.

Hint 2

Allocate two scalar scratch tensors with torch.zeros(1, device=’cuda’)\texttt{torch.zeros(1, device='cuda')}torch.zeros(1, device=’cuda’) inside solve\texttt{solve}solve. Pass their pointers to the kernel and atomic-add into each.

Hint 3

Finalize on the host after the launch: divide both sums by NNN, subtract μ2\mu^2μ2, and copy into the output tensors. Do not call .item()\texttt{.item()}.item().

Requirements

  • Compute both reductions in one pass over xxx using the identity σ2=E[x2]−μ2\sigma^2 = E[x^2] - \mu^2σ2=E[x2]−μ2
  • Accumulate ∑x\sum x∑x and ∑x2\sum x^2∑x2 into two scratch buffers via atomic adds, one atomic per program per statistic
  • Pre-zero the scratch buffers on the host before launch
  • Mask the tail so masked lanes contribute zero to both statistics
  • The solve\texttt{solve}solve signature must not change

Constraints

  • 1≤N≤1061 \le N \le 10^61≤N≤106.
  • Input tensor is torch.float32 on cuda and contiguous; outputs each have shape (1,).
  • Variance uses the population (correction=0) divisor NNN.
  • Tolerance: combined atol=1e-2, rtol=1e-2 for both outputs to absorb float32 sum-of-squares drift at large NNN.
Try Similar Problems
Row-Wise LogSumExp in TritonMediumFused Row-Wise Softmax in TritonMediumMean and Variance as a CUDA KernelMediumVector Sum Reduction in TritonEasyVector Max Reduction in TritonEasyL2 Vector Norm in TritonEasyScaled Dot-Product Attention in TritonMediumBlock-Pointer Matmul in TritonMediumAutotuned Matrix Multiplication in TritonMediumTiled Transpose in TritonMediumRotary Position Embedding in TritonMediumLayerNorm Forward in TritonMedium

Accepts: array

You must run your code first.

Kernel performance

Submit a correct solution to see your kernel performance.

Single-Pass Mean and Variance in Triton

Single-Pass Mean and Variance in Triton

Probability and StatisticsGPU Programming
Medium

Compute the population mean and variance of a 1D float tensor in a single pass over the data. For a tensor xxx of length NNN, write the scalar mean and variance into mean_out[0]\texttt{mean\_out}[0]mean_out[0] and var_out[0]\texttt{var\_out}[0]var_out[0]:

μ=1N∑i=0N−1x[i],σ2=1N∑i=0N−1x[i]2−μ2\mu = \frac{1}{N} \sum_{i=0}^{N-1} x[i], \qquad \sigma^2 = \frac{1}{N} \sum_{i=0}^{N-1} x[i]^2 - \mu^2μ=N1​i=0∑N−1​x[i],σ2=N1​i=0∑N−1​x[i]2−μ2

This problem teaches two-statistic fusion: each program accumulates ∑x\sum x∑x and ∑x2\sum x^2∑x2 for its tile in the same reduction pass, then atomic-adds both partials into scratch buffers. The host finalizes by dividing and applying the E[x2]−E[x]2E[x^2] - E[x]^2E[x2]−E[x]2 identity.

Loading visualization...

Examples

Input: x = [1.0, 2.0, 3.0, 4.0]

Output: mean = [2.5], var = [1.25]

Input: x = [0.0, 0.0, 0.0, 0.0]

Output: mean = [0.0], var = [0.0]

Hint 1

Use the identity σ2=E[x2]−μ2\sigma^2 = E[x^2] - \mu^2σ2=E[x2]−μ2 so one pass over xxx produces both statistics. Reduce tl.sum(x)\texttt{tl.sum(x)}tl.sum(x) and tl.sum(x * x)\texttt{tl.sum(x * x)}tl.sum(x * x) in the same tile.

Hint 2

Allocate two scalar scratch tensors with torch.zeros(1, device=’cuda’)\texttt{torch.zeros(1, device='cuda')}torch.zeros(1, device=’cuda’) inside solve\texttt{solve}solve. Pass their pointers to the kernel and atomic-add into each.

Hint 3

Finalize on the host after the launch: divide both sums by NNN, subtract μ2\mu^2μ2, and copy into the output tensors. Do not call .item()\texttt{.item()}.item().

Requirements

  • Compute both reductions in one pass over xxx using the identity σ2=E[x2]−μ2\sigma^2 = E[x^2] - \mu^2σ2=E[x2]−μ2
  • Accumulate ∑x\sum x∑x and ∑x2\sum x^2∑x2 into two scratch buffers via atomic adds, one atomic per program per statistic
  • Pre-zero the scratch buffers on the host before launch
  • Mask the tail so masked lanes contribute zero to both statistics
  • The solve\texttt{solve}solve signature must not change

Constraints

  • 1≤N≤1061 \le N \le 10^61≤N≤106.
  • Input tensor is torch.float32 on cuda and contiguous; outputs each have shape (1,).
  • Variance uses the population (correction=0) divisor NNN.
  • Tolerance: combined atol=1e-2, rtol=1e-2 for both outputs to absorb float32 sum-of-squares drift at large NNN.
Try Similar Problems
Row-Wise LogSumExp in TritonMediumFused Row-Wise Softmax in TritonMediumMean and Variance as a CUDA KernelMediumVector Sum Reduction in TritonEasyVector Max Reduction in TritonEasyL2 Vector Norm in TritonEasyScaled Dot-Product Attention in TritonMediumBlock-Pointer Matmul in TritonMediumAutotuned Matrix Multiplication in TritonMediumTiled Transpose in TritonMediumRotary Position Embedding in TritonMediumLayerNorm Forward in TritonMedium

Accepts: array

You must run your code first.

Kernel performance

Submit a correct solution to see your kernel performance.

Problems
Loading...
1 / 1