Dynamo NIXL Connect#

Dynamo connect provides utilities for using the NIXL base RDMA subsystem via a set of Python classes. The primary goal of this library to simplify the integration of NIXL based RDMA into inference applications. The dynamo.nixl_connect library can be imported by any Dynamo container hosted application.

import dynamo.nixl_connect

All operations using the NIXL Connect library begin with the Connector class and the type of operation required. There are four types of supported operations:

  1. Register local readable memory:

    Register local memory buffer(s) with the RDMA subsystem to enable a remote worker to read from.

  2. Register local writable memory:

    Register local memory buffer(s) with the RDMA subsystem to enable a remote worker to write to.

  3. Read from registered, remote memory:

    Read remote memory buffer(s), registered by a remote worker to be readable, into local memory buffer(s).

  4. Write to registered, remote memory:

    Write local memory buffer(s) to remote memory buffer(s) registered by a remote worker to writable.

By connecting correctly paired operations, high-throughput GPU Direct RDMA data transfers can be completed. Given the list above, the correct pairing of operations would be 1 & 3 or 2 & 4. Where one side is a “(read|write)-able operation” and the other is its correctly paired “(read|write) operation”. Specifically, a read operation must be paired with a readable operation, and a write operation must be paired with a writable operation.

        sequenceDiagram
    participant LocalWorker
    participant RemoteWorker
    participant NIXL

    LocalWorker ->> NIXL: Register memory (Descriptor)
    RemoteWorker ->> NIXL: Register memory (Descriptor)
    LocalWorker ->> LocalWorker: Create Readable/WritableOperation
    LocalWorker ->> RemoteWorker: Send RDMA metadata (via HTTP/TCP+NATS)
    RemoteWorker ->> NIXL: Begin Read/WriteOperation with metadata
    NIXL -->> RemoteWorker: Data transfer (RDMA)
    RemoteWorker -->> LocalWorker: Notify completion (unblock awaiter)

Python Classes#

References#

On this page