For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo module

activations

The module contains implementations of activation functions.

Functions

elu: Compute the Elu Op using the equation $z if z >= 0 else alpha*(e^z -1)$ .
gelu: Compute the exact GELU activation using the equation $0.5 * x * (1 + erf(x / sqrt(2)))$ .
gelu_quick: Compute the quick (sigmoid) approximation of the GELU activation: $x * sigmoid(1.702 * x)$ .
gelu_tanh: Compute the tanh approximation of the GELU activation: $0.5 * x * (1 + tanh(0.7978845608028654 * (x + 0.044715 * x^3)))$ .
leaky_relu: Compute the Leaky ReLU using the equation $max(x, 0) + negative_slope * min(x, 0)$ .
relu: Compute the Relu Op using the equation $max(x, 0)$ .
relu_n1: Compute the Relu N1 Op using the equation $max(min(x,1),-1)$ .
sigmoid: Compute the sigmoid activation using the equation $1 / (1 + e^{-x})$ .
sign: Compute the sign (0, 1) of the input value.
silu: Compute the SiLU (Swish) activation using the equation $x * sigmoid(x)$ .

Functions​

Functions