dynamo.nixl_connect | NVIDIA Dynamo Documentation

dynamo.nixl_connect publishes 17 classes and 0 functions. Source: lib/bindings/python/src/dynamo/nixl_connect/__init__.py

AbstractOperation (class)

Abstract base class for awaitable NIXL based RDMA operations.

1 from dynamo.nixl_connect import AbstractOperation

1 AbstractOperation(connection: Connection, operation_kind: OperationKind, local_descriptors: Descriptor | list[Descriptor], remote_descriptors: Optional[Descriptor | list[Descriptor]], notification_key: Optional[str]) -> None

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L105

Public methods

init

1 __init__(connection: Connection, operation_kind: OperationKind, local_descriptors: Descriptor | list[Descriptor], remote_descriptors: Optional[Descriptor | list[Descriptor]], notification_key: Optional[str]) -> None

No summary available.

source

wait_for_completion

1 wait_for_completion() -> None

Blocks the caller asynchronously until the operation has completed.

source

ActiveOperation (class)

Abstract class for active operations that initiates a NIXL based RDMA transfer based RdmaMetadata provided by the remote worker’s corresponding PassiveOperation.

1 from dynamo.nixl_connect import ActiveOperation

1 ActiveOperation(remote: Remote, operation_kind: OperationKind, local_descriptors: Descriptor | list[Descriptor], remote_descriptors: Descriptor | list[Descriptor], notification_key: str) -> None

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L283

Public methods

init

1 __init__(remote: Remote, operation_kind: OperationKind, local_descriptors: Descriptor | list[Descriptor], remote_descriptors: Descriptor | list[Descriptor], notification_key: str) -> None

No summary available.

source

cancel

1 cancel() -> None

Cancels the operation. No affect if the operation has already completed or errored, or has been cancelled.

source

Connection (class)

No summary available.

1 from dynamo.nixl_connect import Connection

1 Connection(connector: Connector, number: int)

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L586

Public methods

init

1 __init__(connector: Connector, number: int)

Creates a new Connection instance.

Parameters

connector

Connector

The connector associated with this connection.

number

int

The connection number. Used to create a unique name for the connection.

Raises

TypeError — When connector is provided and not of type dynamo.nixl_connect.Connector.
TypeError — When number is provided and not of type int.
ValueError — When number is provided and not greater than 0.

source

acquire_remote_ref

1 acquire_remote_ref(name: str) -> None

No summary available.

source

release_remote_ref

1 release_remote_ref(name: str) -> bool

Returns True when the last reference is released.

source

initialize

1 initialize() -> None

No summary available.

source

Connector (class)

Core class for managing the connection between workers in a distributed environment. Use this class to create readable and writable operations, or read and write data to remote workers.

1 from dynamo.nixl_connect import Connector

1 Connector(worker_id: Optional[str] = None) -> None

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L692

Public methods

init

1 __init__(worker_id: Optional[str] = None) -> None

Creates a new Connector instance.

Parameters

worker_id

Optional[str]

Unique identifier of the worker, defaults to a new UUID when None.

Raises

TypeError — When worker_id is provided and not of type uuid.UUID.

source

begin_read

1 begin_read(remote_metadata: RdmaMetadata, local_descriptors: Descriptor | list[Descriptor]) -> ReadOperation

Creates a read operation for fulfilling a remote readable operation.

Parameters

remote_metadata

RdmaMetadata

RDMA metadata from a remote worker that has created a readable operation.

local_descriptors

Descriptor | list[Descriptor]

Local descriptor(s) to receive data from the remote worker described by remote_metadata.

Returns

ReadOperation — Awaitable read operation that can be used to transfer data from a remote worker.

Raises

TypeError — When remote_metadata is not of type RdmaMetadata.
TypeError — When local_descriptors is not of type dynamo.nixl_connect.Descriptor or list[dynamo.nixl_connect.Descriptor].

source

begin_write

1 begin_write(local_descriptors: Descriptor | list[Descriptor], remote_metadata: RdmaMetadata) -> WriteOperation

Creates a write operation for transferring data to a remote worker.

Parameters

local_descriptors

Descriptor | list[Descriptor]

Local descriptors of one or more data objects to be transferred to the remote worker.

remote_metadata

RdmaMetadata

Serialized request from a remote worker that has created a readable operation.

source

create_readable

1 create_readable(local_descriptors: Descriptor | list[Descriptor]) -> ReadableOperation

Creates a readable operation for transferring data from a remote worker.

Returns

ReadableOperation — A readable operation that can be used to transfer data from a remote worker.

source

create_writable

1 create_writable(local_descriptors: Descriptor | list[Descriptor]) -> WritableOperation

Creates a writable operation for transferring data to a remote worker.

Returns

WritableOperation — A writable operation that can be used to transfer data to a remote worker.

source

initialize

1 initialize() -> None

Deprecated method.

source

Descriptor (class)

Memory descriptor that ensures memory is registered w/ NIXL, used for transferring data between workers.

1 from dynamo.nixl_connect import Descriptor

1 Descriptor(data: torch.Tensor | tuple[array_module.ndarray, Device | str] | bytes | tuple[int, int, Device | str, Any]) -> None

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L919

Public methods

init

1 __init__(data: torch.Tensor | tuple[array_module.ndarray, Device | str] | bytes | tuple[int, int, Device | str, Any]) -> None

Memory descriptor for transferring data between workers.

Parameters

data

The data to be transferred.

When torch.Tensor is provided, the attributes of the tensor will be used to create the descriptor.

When tuple[ndarray, Device] is provided, the tuple must contain:

ndarray: The CuPy or NumPy array to be transferred.
Device: Either a dynamo.nixl_connect.Device or a string representing the device type (e.g., “cuda” or “cpu”).

When bytes is provided, the pointer and size derived from the bytes object and memory type will be assumed to be CPU.

When tuple[int, int, Device|str, Any] is provided, the tuple must contain the following elements:

int: Pointer to the data in memory.
int: Size of the data in bytes.
Device: Either a dynamo.nixl_connect.Device or a string representing the device type (e.g., “cuda” or “cpu”).
Any: Optional reference to the data (e.g., the original tensor or bytes object). This is useful for keeping a reference to the data in memory, but it is not required.

Raises

ValueError — When data is None.
TypeError — When data is not a valid type (i.e., not torch.Tensor, bytes, or a valid tuple).
TypeError — When data is a tuple but the elements are not of the expected types (i.e., [ndarray, Device|str] OR [int, int, Device|str, Any]).

source

from_serialized

1 from_serialized(serialized: SerializedDescriptor) -> Descriptor

Deserializes a SerializedDescriptor into a Descriptor object.

Parameters

serialized

SerializedDescriptor

The serialized descriptor to deserialize.

Returns

Descriptor — The deserialized descriptor.

source

deregister_with_connector

1 deregister_with_connector(connection: Connection) -> None

Deregisters the memory of the descriptor with NIXL.

source

register_with_connector

1 register_with_connector(connection: Connection) -> None

Registers the memory of the descriptor with NIXL.

source

Device (class)

Represents a device in the system.

1 from dynamo.nixl_connect import Device

1 Device(metadata: str | tuple[DeviceKind, int]) -> None

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L1283

Public methods

init

1 __init__(metadata: str | tuple[DeviceKind, int]) -> None

No summary available.

source

DeviceKind (class)

Type of memory a descriptor has been allocated to.

1 from dynamo.nixl_connect import DeviceKind

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L1353

OperationKind (class)

Kind of an operation.

1 from dynamo.nixl_connect import OperationKind

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L1396

OperationStatus (class)

Status of an operation.

1 from dynamo.nixl_connect import OperationStatus

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L1414

PassiveOperation (class)

Abstract class for common functionality of passive operations.

1 from dynamo.nixl_connect import PassiveOperation

1 PassiveOperation(connection: Connection, operation_kind: OperationKind, local_descriptors: Descriptor | list[Descriptor]) -> None

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L1452

Public methods

init

1 __init__(connection: Connection, operation_kind: OperationKind, local_descriptors: Descriptor | list[Descriptor]) -> None

No summary available.

source

metadata

1 metadata(hex_encode: bool = False) -> RdmaMetadata

Gets the request descriptor for the operation.

source

wait_for_completion

1 wait_for_completion() -> None

Blocks the caller asynchronously until the operation has completed.

source

RdmaMetadata (class)

Pydantic serialization type for describing the passive side of a transfer.

1 from dynamo.nixl_connect import RdmaMetadata

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L1758

Public methods

to_descriptors

1 to_descriptors() -> Descriptor | list[Descriptor]

Deserializes the request descriptor into a dynamo.nixl_connect.Descriptor or list of dynamo.nixl_connect.Descriptor objects.

source

validate_operation_kind

1 validate_operation_kind(cls, v: int) -> int

No summary available.

source

ReadOperation (class)

Operation that initiates an RDMA read operation to transfer data from a remote worker’s ReadableOperation, as described by remote_metadata, to local buffers.

1 from dynamo.nixl_connect import ReadOperation

1 ReadOperation(connection: Connection, remote_metadata: RdmaMetadata, local_descriptors: Descriptor | list[Descriptor]) -> None

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L1614

Public methods

init

1 __init__(connection: Connection, remote_metadata: RdmaMetadata, local_descriptors: Descriptor | list[Descriptor]) -> None

Creates a new instance of ReadOperation, registers local_descriptors with NIXL, and begins an RDMA read operation which will transfer data described by remote_metadata to local_descriptors.

Parameters

connection

Connection

Connection instance to use for the operation.

remote_metadata

RdmaMetadata

Serialized request from the remote worker.

local_descriptors

Descriptor | list[Descriptor]

Local descriptor(s) to to receive the data from the remote worker.

source

cancel

1 cancel() -> None

Cancels the operation. No affect if the operation has already completed or errored, or been cancelled.

source

results

1 results() -> list[Descriptor]

Gets the results of the operation. Returns a single descriptor if only one was requested, or a list of descriptors if multiple were requested.

source

wait_for_completion

1 wait_for_completion() -> None

Blocks the caller asynchronously until the operation has completed.

source

ReadableOperation (class)

Operation that can be awaited until a remote worker has completed a ReadOperation.

1 from dynamo.nixl_connect import ReadableOperation

1 ReadableOperation(connection: Connection, local_descriptors: Descriptor | list[Descriptor]) -> None

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L1720

Public methods

init

1 __init__(connection: Connection, local_descriptors: Descriptor | list[Descriptor]) -> None

No summary available.

source

wait_for_completion

1 wait_for_completion() -> None

Blocks the caller asynchronously until the operation has completed.

source

Remote (class)

Identifies a remote NIXL enabled worker relative to a local NIXL enabled worker.

1 from dynamo.nixl_connect import Remote

1 Remote(connection: Connection, nixl_metadata: bytes | str) -> None

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L1795

Public methods

init

1 __init__(connection: Connection, nixl_metadata: bytes | str) -> None

No summary available.

source

SerializedDescriptor (class)

Pydantic serialization type for memory descriptors.

1 from dynamo.nixl_connect import SerializedDescriptor

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L1883

Public methods

to_descriptor

1 to_descriptor() -> Descriptor

Deserialize the serialized descriptor into a Descriptor object.

source

validate_device

1 validate_device(cls, v: str) -> str

No summary available.

source

validate_ptr

1 validate_ptr(cls, v: int) -> int

No summary available.

source

validate_size

1 validate_size(cls, v: int) -> int

No summary available.

source

WritableOperation (class)

Operation which can be awaited until written to by a WriteOperation from a remote worker.

1 from dynamo.nixl_connect import WritableOperation

1 WritableOperation(connection: Connection, local_descriptors: Descriptor | list[Descriptor]) -> None

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L1933

Public methods

init

1 __init__(connection: Connection, local_descriptors: Descriptor | list[Descriptor]) -> None

Creates a new instance of WritableOperation, registers the operation and descriptors w/ NIXL, and enables an RDMA write operation to occur.

Parameters

connection

Connection

Connection instance to use for the operation.

local_descriptors

Descriptor | list[Descriptor]

Descriptors to receive data from a remote worker.

Raises

TypeError

When connection is not a dynamo.nixl_connect.Connection.

TypeError

When local_descriptors is not a dynamo.nixl_connect.Descriptor or list[dynamo.nixl_connect.Descriptor].

source

wait_for_completion

1 wait_for_completion() -> None

Blocks the caller asynchronously until the operation has completed.

source

WriteOperation (class)

Awaitable write operation which initiates an RDMA write operation to a remote worker which provided a RdmaMetadata object from a WritableOperation.

1 from dynamo.nixl_connect import WriteOperation

1 WriteOperation(connection: Connection, local_descriptors: Descriptor | list[Descriptor], remote_metadata: RdmaMetadata) -> None

lib/bindings/python/src/dynamo/nixl_connect/__init__.py#L1988

Public methods

init

1 __init__(connection: Connection, local_descriptors: Descriptor | list[Descriptor], remote_metadata: RdmaMetadata) -> None

Creates a new instance of WriteOperation, registers local_descriptors with NIXL, and begins an RDMA write operation which will transfer from local_descriptors to remote target(s) described by remote_metadata

Parameters

connection

Connection

Connection instance to use for the operation.

local_descriptors

Descriptor | list[Descriptor]

Local descriptor(s) to send from, to the remote worker.

remote_metadata

RdmaMetadata

Serialized request from the remote worker that describes the target(s) to send to.

Raises

TypeError

When connector is not a dynamo.nixl_connect.Connector.

TypeError

When remote_metadata is not a dynamo.nixl_connect.RdmaMetadata.

ValueError

When remote_metadata is not of kind WRITE.

ValueError

When remote_metadata.nixl_metadata is not a non-empty str.

TypeError

When local_descriptors is not a dynamo.nixl_connect.Descriptor or list[dynamo.nixl_connect.Descriptor].

source

cancel

1 cancel() -> None

Cancels the operation. No affect if the operation has already completed or errored, or has been cancelled.

source

wait_for_completion

1 wait_for_completion() -> None

Blocks the caller asynchronously until the operation has completed.

source