For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo module

gemv

Provides general matrix-vector (GEMV) and general vector-matrix (GEVM) kernels for CPU and GPU.

`comptime` values

comptime logger = Logger(stdout, prefix=String(""), source_location=False)

GEMVAlgorithm: Enumerates the GEMV kernel algorithm variants used by the GPU dispatcher.

gemm_mma_cpasync: Launch the batched GEMM tensor-core kernel.
gemm_mma_cpasync_kernel:
gemv: Computes a CPU matrix-vector product C = A * b using vectorized element-wise reduction.
gemv_gpu: Selects and dispatches the appropriate GPU GEMV kernel based on shape and hardware.
gemv_gpu_dispatch: Launches the GPU GEMV kernel indicated by kernel_func with appropriate grid and block dims.
gemv_kernel:
gemv_kernel_vector:
gemv_split_k: GEMV with tiling in K dimension. Assuming the B (weight) matrix is transposed i.e. row major N x K, this kernel implements a vector (1 x K) times a matrix (N x K). The impl can actually handle M > 1 but it's only optimal for tiny M. We use it for M = 1 only.
gevm_kernel:
is_minimax_router_gemm: Returns whether a GEMM has the MiniMax-M3 fp32 router signature.
log_shape: Logs the shape of a named tensor dimension pair to the info logger.
naive_gemv: Computes a reference matrix-vector product C = A * b using a scalar nested loop.
reverse_idx: Returns an index pair (x, y) or (y, x) depending on the transpose parameter.
router_gate_mixed_gemv: Launches the mixed bf16-activation × fp32-weight router-gate GEMV.
router_gate_use_mixed_gemv: Returns whether runtime m should take the fused mixed router GEMV.