Introduction

NVIDIA® Scalable Hierarchical Aggregation and Reduction Protocol (SHARP)™ technology improves upon the performance of MPI and Machine Learning collective operations, by offloading collective operations from the CPU and GPU to the network and eliminating the need to send data multiple times between endpoints.

This innovative approach decreases the amount of data traversing the network as aggregation nodes are reached, and dramatically reduces collective operations time. Implementing collective offloads communication algorithms supporting streaming for Machine Learning in the network also has additional benefits, such as freeing up valuable CPU resources for computation rather than using them to process communication.

© Copyright 2023, NVIDIA. Last updated on May 23, 2023.