Communicators#

`pylibwholegraph.torch.WholeMemoryCommunicator`(...)	WholeMemory Communicator.
`pylibwholegraph.torch.comm.set_world_info`(...)	Set the global world's information.
`pylibwholegraph.torch.create_group_communicator`([...])	Create WholeMemory Communicator. For example: 24 ranks with group_size = 4 and comm_stride = 2 will create following groups: [0, 2, 4, 6], [1, 3, 5, 7], [8, 10, 12, 14], [9, 11, 13, 15], [16, 18, 20, 22], [17, 19, 21, 23] :param group_size: Size of each group, -1 means to use all ranks in just one single group. :param comm_stride: Stride of each rank in each group :return: WholeMemoryCommunicator.
`pylibwholegraph.torch.split_communicator`(...)	Split Communicator.
`pylibwholegraph.torch.destroy_communicator`(wm_comm)	Destroy WholeMemoryCommunicator :param wm_comm: WholeMemoryCommunicator to destroy :return: None
`pylibwholegraph.torch.get_global_communicator`([...])	Get the global communicator of this job :return: WholeMemoryCommunicator that has all GPUs in it.
`pylibwholegraph.torch.get_local_node_communicator`()	Get the local node communicator of this job :return: WholeMemoryCommunicator that has GPUs in the same node.
`pylibwholegraph.torch.get_local_device_communicator`()	Get the local device communicator of this job :return: WholeMemoryCommunicator that has only the GPU belonging to current process.
`pylibwholegraph.torch.get_local_mnnvl_communicator`()
`pylibwholegraph.torch.comm.comm_set_distributed_backend`(...)