NVSHMEM Unsupported Operations

This section describes some key NVSHMEM unsupported operations considerations when developing applications using NVSHMEM runtime.

Toolchaining

NVSHMEM does not natively support compiling applications by using the nvc++ compiler.

Operations

The following list of operations are not natively supported, and the workarounds are included with each item:

  • Host-side memory allocation: NVSHMEM does not support host memory as a target for its device-initiated communication operations. Applications that want to use data in the host memory as a target for device-initiated communication operations need to use OpenSHMEM for data in host memory or use NVSHMEM to copy data to/from the symmetric heap. Here is an overview of the process:

    1. Copy the data from host memory to the symmetric heap by using cudaMemcpy and its friends.

    2. Complete the device op to/from data in the symmetric heap.

    3. Copy the data back from symmetric heap to host memory by using cudaMemcpy and its friends.

  • Atomic floating point min/max operation.

    This is possible to emulate using nvshmem_TYPENAME_OPERATION_reduce. Refer to the supported reduce ops table for more information.