DistributedContext#

class nvmath.distributed.DistributedContext( device_id: int, communicator: mpi4py.MPI.Comm, nvshmem_available: bool, nccl_comm: int | None, )[source]#

Context of initialized nvmath.distributed runtime.

device_id#

CUDA device ID associated with the distributed runtime on this process.

Type:: int

communicator#

MPI communicator of participating processes.

Type:: mpi4py.MPI.Comm

nvshmem_available#

True if NVSHMEM backend was selected at initialization.

Type:: bool

nccl_comm#

pointer to NCCL communicator if NCCL backend was selected at initialization, None otherwise.

Type:: int | None