For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

ncclCommInitAll

def ncclCommInitAll(comms: Pointer[Optional[Pointer[NoneType, MutUntrackedOrigin, _safe=False]], _safe=False], ndev: Int, devlist: Pointer[Int32, _safe=False]) -> ncclResult_t

Initializes NCCL/RCCL communicators for a set of GPUs.

Thin FFI wrapper around ncclCommInitAll. Allocates one communicator per device in devlist and stores the handles in comms. Must be called from a single thread; concurrent calls for the same device set cause undefined behavior in the NCCL library.

Args:

comms (Pointer[Optional[Pointer[NoneType, MutUntrackedOrigin, _safe=False]], _safe=False]): Output array of communicator handles; must have room for ndev entries.
ndev (Int): Number of GPUs to include in the communicator group.
devlist (Pointer[Int32, _safe=False]): Array of CUDA/ROCm device IDs to include.

Returns:

ncclResult_t: ncclResult_t.ncclSuccess on success; a non-zero status on failure.

Raises:

If the CCL function symbol cannot be resolved from the vendor library.