For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo module
gemv
Provides general matrix-vector (GEMV) and general vector-matrix (GEVM) kernels for CPU and GPU.
comptime values
logger
comptime logger = Logger(stdout, prefix=String(""), source_location=False)
Structs
-
GEMVAlgorithm: Enumerates the GEMV kernel algorithm variants used by the GPU dispatcher.
Functions
-
gemm_mma_cpasync: Launch the batched GEMM tensor-core kernel. -
gemm_mma_cpasync_kernel: SM100 (B200) warp-specialized GEMM kernel using cp.async and m16n8k16 MMA instructions. -
gemv: Computes a CPU matrix-vector product C = A * b using vectorized element-wise reduction. -
gemv_gpu: Selects and dispatches the appropriate GPU GEMV kernel based on shape and hardware. -
gemv_gpu_dispatch: Launches the GPU GEMV kernel indicated by kernel_func with appropriate grid and block dims. -
gemv_kernel: GPU kernel for matrix-vector multiplication using scalar warp-level reduction. -
gemv_kernel_vector: GPU kernel for matrix-vector multiplication using vectorized warp-level loads. -
gemv_split_k: GEMV with tiling in K dimension. Assuming the B (weight) matrix is transposed i.e. row major N x K, this kernel implements a vector (1 x K) times a matrix (N x K). The impl can actually handle M > 1 but it's only optimal for tiny M. We use it for M = 1 only. -
gevm_kernel: GPU kernel for row-vector times matrix multiplication (GEVM). -
is_minimax_router_gemm: Returns whether a GEMM has the MiniMax-M3 fp32 router signature. -
log_shape: Logs the shape of a named tensor dimension pair to the info logger. -
naive_gemv: Computes a reference matrix-vector product C = A * b using a scalar nested loop. -
reverse_idx: Returns an index pair (x, y) or (y, x) depending on the transpose parameter. -
router_gate_mixed_gemv: Launches the mixed bf16-activation × fp32-weight router-gate GEMV. -
router_gate_use_mixed_gemv: Returns whether runtimemshould take the fused mixed router GEMV.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!