2. Known Limitations#

Shared Runtime and Configuration Selection:

cuFile and the cuObject client continue to share one process-wide runtime. Direct application use of both interfaces in the same process is unsupported in this release.

The first successful initialization selects the configuration used by the shared runtime. Initialization can occur explicitly or implicitly through an API call. Subsequent initialization does not load the other configuration file or switch the active configuration. Configuration selection can occur again after the shared runtime has been fully torn down.

A later cuObject initialization after cuFile initialization, or an explicit cuFileDriverOpen() after cuObject initialization, generates a WARN-level log message and retains the existing configuration. Implicit cuFile initialization paths may reuse the runtime without this warning. Warning visibility depends on the configured logging level.

2.1. Memory and Size Constraints#

  • Maximum Memory Registration: (4GiB - 64K) per cuMemObjGetDescriptor() call

  • buffer_offset is ignored and defaults to 0 in the GET/PUT API calls.

  • Concurrent GET/PUT operations on a single RDMA buffer are not supported.

  • GET/PUT for GPU unregistered buffers is limited to per_buffer_cache_size_kb config value specified in the selected cuObject client JSON configuration. GET/PUT for host memory is not supported for unregistered buffers

  • cuObject does not use the cuFile parallel-I/O thread pool. Setting execution.parallel_io to true in the cuObject configuration does not enable this path.

  • The memory key can be reused after a deregister followed by a re-registration. For server error handling scenarios, either the server should guarantee that all outstanding IOs are drained or the client should call cuFileDriverClose() to reject any stale RDMA IO requests from corrupting newer registrations.

  • For unregistered CUDA managed memory, current behavior returns -1 from cuMemObjGetMaxRequestCallbackSize()

  • RDMA multipathing requires at least two active RDMA devices during driver initialization to support failover/failback.