Mojo package

comm

The gpu.comm package provides communication primitives for GPUs.

This package includes functions for sending and receiving data between GPUs, as well as for synchronizing threads across GPUs.

Modules

• ​allgather: Multi-GPU allgather implementation that gathers values from multiple GPUs into an output buffer.
• ​allreduce: Multi-GPU allreduce implementation for efficient tensor reduction across GPUs.

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit