The Scaled Exponential Linear Unit (SELU) is a self-normalizing activation function. When used with proper weight initialization (LeCun normal), SELU automatically maintains zero mean and unit variance activations across layers, eliminating the need for batch normalization.
Given a list of values, apply the SELU activation to each element using the fixed constants lambda and alpha.
The constants are derived analytically to preserve self-normalizing properties:
λ≈1.0507α≈1.6733Input:
x = [1.0, -1.0, 0.0]
Output:
[1.0507, -1.1113, 0.0]
Positive values are scaled by lambda. Negative values are scaled by lambda * alpha * (exp(x) - 1). Zero maps to zero.
Input:
x = [0.5, 1.5, 2.5]
Output:
[0.5254, 1.5761, 2.6268]
All positive values are simply multiplied by lambda (approximately 1.0507).
Define the two constants at the top of your function. For positive x, return lambda * x. For non-positive x, return lambda * alpha * (exp(x) - 1).
Note that SELU(0) = lambda * alpha * (exp(0) - 1) = 0, so the function is continuous at x = 0.
Sign in to take notes on this problem
Accepts: array
The Scaled Exponential Linear Unit (SELU) is a self-normalizing activation function. When used with proper weight initialization (LeCun normal), SELU automatically maintains zero mean and unit variance activations across layers, eliminating the need for batch normalization.
Given a list of values, apply the SELU activation to each element using the fixed constants lambda and alpha.
The constants are derived analytically to preserve self-normalizing properties:
λ≈1.0507α≈1.6733Input:
x = [1.0, -1.0, 0.0]
Output:
[1.0507, -1.1113, 0.0]
Positive values are scaled by lambda. Negative values are scaled by lambda * alpha * (exp(x) - 1). Zero maps to zero.
Input:
x = [0.5, 1.5, 2.5]
Output:
[0.5254, 1.5761, 2.6268]
All positive values are simply multiplied by lambda (approximately 1.0507).
Define the two constants at the top of your function. For positive x, return lambda * x. For non-positive x, return lambda * alpha * (exp(x) - 1).
Note that SELU(0) = lambda * alpha * (exp(0) - 1) = 0, so the function is continuous at x = 0.
Sign in to take notes on this problem
Accepts: array