The Exponential Linear Unit (ELU) is an activation function that pushes mean activations closer to zero, which speeds up learning. Unlike ReLU, ELU produces a smooth curve for negative inputs, reducing the impact of the vanishing gradient problem.
Given a list of values and a parameter alpha, apply the ELU activation to each element.
The parameter alpha controls the negative saturation value. As x approaches negative infinity, ELU(x) approaches -alpha.
Input:
x = [1.0, -1.0, 0.0, 2.0, -0.5], alpha = 1.0
Output:
[1.0, -0.6321, 0.0, 2.0, -0.3935]
Positive values pass through unchanged. Negative values are mapped through alpha * (exp(x) - 1), approaching -alpha for very negative x.
Input:
x = [-1.0, -2.0, -3.0], alpha = 2.0
Output:
[-1.2642, -1.7293, -1.9004]
With alpha = 2.0 the negative saturation value doubles compared to alpha = 1.0.
Use math.exp(v) to compute the exponential. The boundary condition is at x = 0: for x > 0 return x, for x <= 0 return alpha * (exp(x) - 1).
When alpha = 0, all negative values map to 0 (similar to ReLU). When alpha = 1, the function is continuous and smooth at x = 0.
Sign in to take notes on this problem
Accepts: array
Accepts: number
The Exponential Linear Unit (ELU) is an activation function that pushes mean activations closer to zero, which speeds up learning. Unlike ReLU, ELU produces a smooth curve for negative inputs, reducing the impact of the vanishing gradient problem.
Given a list of values and a parameter alpha, apply the ELU activation to each element.
The parameter alpha controls the negative saturation value. As x approaches negative infinity, ELU(x) approaches -alpha.
Input:
x = [1.0, -1.0, 0.0, 2.0, -0.5], alpha = 1.0
Output:
[1.0, -0.6321, 0.0, 2.0, -0.3935]
Positive values pass through unchanged. Negative values are mapped through alpha * (exp(x) - 1), approaching -alpha for very negative x.
Input:
x = [-1.0, -2.0, -3.0], alpha = 2.0
Output:
[-1.2642, -1.7293, -1.9004]
With alpha = 2.0 the negative saturation value doubles compared to alpha = 1.0.
Use math.exp(v) to compute the exponential. The boundary condition is at x = 0: for x > 0 return x, for x <= 0 return alpha * (exp(x) - 1).
When alpha = 0, all negative values map to 0 (similar to ReLU). When alpha = 1, the function is continuous and smooth at x = 0.
Sign in to take notes on this problem
Accepts: array
Accepts: number