Mojo module

activations

The module contains implementations of activation functions.

Functions

• ​elu: Compute the Elu Op using the equation $z if z >= 0 else alpha*(e^z -1)$.
• ​gelu: Compute the GELU Op using the equation $0.5 * x * (1 + erf(x / sqrt(2)))$.
• ​gelu_approximate: Compute the approximate GELU Op using the equation $0.5 * x * (1 + tanh(sqrt(2 / pi) * (x + 0.044715 * x^3)))$.
• ​relu: Compute the Relu Op using the equation $max(0, x)$.
• ​relu_n1: Compute the Relu N1 Op using the equation $max(min(x,1),-1)$.
• ​sign: Compute the sign (0, 1) of the input value.

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit