1 / 1

GELU in Triton

Activation FunctionsGPU Programming

Easy

Implement the exact GELU (Gaussian Error Linear Unit) activation in Triton. For each input element:

\texttt{out}[i] = \tfrac{1}{2} \, x[i] \, \Big(1 + \mathrm{erf}\!\left(\tfrac{x[i]}{\sqrt{2}}\right)\Big)

GELU is the default activation in Transformer feedforward blocks. The exact form needs the Gauss error function $\mathrm{erf}$ , which Triton exposes through a device intrinsic. This problem introduces calling transcendental intrinsics from inside a Triton kernel.

Loading visualization...

Examples

Input: x = [0.0, 1.0, -1.0, 2.0]

Output: [0.0, 0.8413, -0.1587, 1.9545]

Input: x = [-2.0, 0.5, 3.0]

Output: [-0.0455, 0.3457, 2.9960]

Hint 1

Triton exposes the error function as $\texttt{tl.math.erf}$ (on older releases via $\texttt{tl.libdevice.erf}$ ). Apply it elementwise on the loaded tile.

Hint 2

Pre-multiply by $1/\sqrt{2} \approx 0.7071067811865475$ instead of dividing on each lane. The compiler keeps the constant in a register.

Hint 3

The reference is $\texttt{torch.nn.functional.gelu}$ with default (exact) approximation. Match it within $\text{atol}=10^{-3}$ for every test.

Requirements

Use one Triton program per contiguous tile of the input
Implement the exact form using a device intrinsic for $\mathrm{erf}$
Pre-multiply by $1/\sqrt{2}$ rather than dividing on every lane
Mask the tail on every load and store
The $\texttt{solve}$ signature must not change

Constraints

$1 \le N \le 10^6$ .
All tensors are torch.float32 on cuda.
Tolerance: atol=1e-3 against torch.nn.functional.gelu (default exact mode).

Try Similar Problems

ReLU in TritonEasy SiLU in TritonEasy Vector Addition in TritonEasy Fused Multiply-Add in TritonEasy Online Softmax Kernel in TritonMedium Fused Row-Wise Softmax in TritonMedium Fused Matmul + Bias + ReLU in TritonMedium Implement Swish (SiLU) as a CUDA KernelEasy ReLU as a CUDA KernelEasy Implement Leaky ReLU as a CUDA KernelEasy Implement Sigmoid as a CUDA KernelEasy Implement Exact GELU as a CUDA KernelEasy

GELU in Triton

Activation FunctionsGPU Programming

Easy

Implement the exact GELU (Gaussian Error Linear Unit) activation in Triton. For each input element:

\texttt{out}[i] = \tfrac{1}{2} \, x[i] \, \Big(1 + \mathrm{erf}\!\left(\tfrac{x[i]}{\sqrt{2}}\right)\Big)

Loading visualization...

Examples

Input: x = [0.0, 1.0, -1.0, 2.0]

Output: [0.0, 0.8413, -0.1587, 1.9545]

Input: x = [-2.0, 0.5, 3.0]

Output: [-0.0455, 0.3457, 2.9960]

Hint 1

Triton exposes the error function as $\texttt{tl.math.erf}$ (on older releases via $\texttt{tl.libdevice.erf}$ ). Apply it elementwise on the loaded tile.

Hint 2

Pre-multiply by $1/\sqrt{2} \approx 0.7071067811865475$ instead of dividing on each lane. The compiler keeps the constant in a register.

Hint 3

The reference is $\texttt{torch.nn.functional.gelu}$ with default (exact) approximation. Match it within $\text{atol}=10^{-3}$ for every test.

Requirements

Use one Triton program per contiguous tile of the input
Implement the exact form using a device intrinsic for $\mathrm{erf}$
Pre-multiply by $1/\sqrt{2}$ rather than dividing on every lane
Mask the tail on every load and store
The $\texttt{solve}$ signature must not change

Constraints

$1 \le N \le 10^6$ .
All tensors are torch.float32 on cuda.
Tolerance: atol=1e-3 against torch.nn.functional.gelu (default exact mode).

Try Similar Problems

x =

Accepts: array

Examples

Requirements

Constraints

Kernel performance

GELU in Triton

Examples

Requirements

Constraints

Kernel performance