holoscan.payload

Beta

Registered Holoscan payload identities.

Submodules

holoscan.payload.schema

Package Contents

Classes

Name	Description
`PayloadType`	A registered payload identity token.
`Tensor`	An n-dimensional array, interoperable with NumPy, CuPy, and PyTorch via DLPack and the array

Functions

Name	Description
`resolve`	-

Data

API

class holoscan.payload.PayloadType()

A registered payload identity token.

epoch

int

name

str

transport_profile_tag

int

holoscan.payload.PayloadType.__call__(
    args: object = (),
    kwargs: object = {}
) -> typing.NoReturn

holoscan.payload.PayloadType.__eq__(
    arg: object
) -> bool

holoscan.payload.PayloadType.__hash__() -> int

holoscan.payload.PayloadType.__repr__() -> str

class holoscan.Tensor()

An n-dimensional array, interoperable with NumPy, CuPy, and PyTorch via DLPack and the array interface protocols. Wraps a DLPack managed tensor; instances are cheap handles that share the underlying memory and its access authority. Zero-copy imports are read-only; writable consumers must use an explicit allocation-and-copy path.

__array_interface__

dict[str, object]

The NumPy __array_interface__ dict. Present only for host (CPU / pinned-host) tensors; accessing it on a device tensor raises AttributeError. The dictionary carries no stream, so a readiness-aware tensor first waits on the host for its producer’s work, such as a pinned-host tensor written on a CUDA stream. A tensor written on the host returns at once.

__cuda_array_interface__

dict[str, object]

The __cuda_array_interface__ dict. Present only for device (CUDA / managed) tensors; accessing it on a host tensor raises AttributeError. Readiness-aware tensors are synchronized to the legacy default stream before export because the returned dictionary reports stream=1.

__dlpack_c_exchange_api__

object = ...

byte_offset

int

Offset, in bytes, from the start of the underlying allocation to the tensor data (int).

data

int

Integer address of the tensor’s data pointer.

device

_tensor_DLDevice

The DLDevice the tensor data resides on.

dtype

_tensor_DLDataType

The DLDataType of the tensor elements.

is_read_only

bool

Whether the tensor’s effective backing authority is permanently read-only.

itemsize

int

Size of a single element, in bytes (int).

nbytes

int

Total size of the elements, in bytes (int).

ndim

int

Number of dimensions (int).

shape

tuple[int, ...]

Tuple of the tensor’s dimensions.

size

int

Total number of elements (int).

strides

tuple[int, ...]

Tuple of the tensor’s strides, in bytes.

holoscan.Tensor.__dlpack__(
    stream: object | None = None,
    max_version: tuple[int, int] | None = None,
    dl_device: tuple[int, int] | None = None,
    copy: bool | None = None
) -> object

Export the tensor as a DLPack capsule (the array-API __dlpack__ protocol).

Parameters:

stream

int or NoneDefaults to None

Consumer stream for synchronization. None/1 = legacy default stream, 2 = per-thread default, -1 = no synchronization, a stream pointer as a positive integer, or an object with callable __cuda_stream__() returning (0, stream_pointer). Ignored for host tensors.

max_version

tuple[int, int] or NoneDefaults to None

Maximum supported DLPack (major, minor) version; a major >= 1 selects the versioned capsule.

dl_device

tuple[int, int] or NoneDefaults to None

Requested (device_type, device_id). A different device requests an explicit copy.

copy

bool or NoneDefaults to None

Request an owning contiguous copy before export. copy=False rejects a different dl_device request.

Returns: PyCapsule

Raises:

BufferError: If the tensor is read-only and max_version does not select a versioned capsule. Legacy DLPack cannot carry read-only authority, so Holoscan rejects rather than widening access.

For imported CUDA storage, stream=2 and stream=-1 use conservative device retirement. stream=-1 still skips producer-readiness ordering; the caller must establish it separately. Prefer explicit streams for precise retirement.

holoscan.Tensor.__dlpack_device__() -> tuple[int, int]

Return the tensor’s (device_type, device_id) tuple (the array-API __dlpack_device__ protocol). Both values are ints. CUDA device tensors preserve the owning CUDA ordinal; CPU, pinned-host, and managed tensors report device_id == 0.

Returns: tuple[int, int]

holoscan.Tensor.as_tensor(
    obj: object,
    stream: object | None = None
) -> holoscan.Tensor

staticmethod

Create a Tensor from a tensor-like object (zero-copy).

References

.. [1] https://dmlc.github.io/dlpack/latest/python_spec.html .. [2] https://data-apis.org/array-api/latest/API_specification/generated/array_api.array.__dlpack__.html .. [3] https://numba.readthedocs.io/en/stable/cuda/cuda_array_interface.html .. [4] https://numpy.org/doc/stable/reference/arrays.interface.html

Parameters:

obj

array - like

A NumPy array, CuPy array, PyTorch tensor, or any object supporting the DLPack protocol or an array interface.

stream

int or objectDefaults to None

CUDA import stream, as an integer or an object exposing __cuda_stream__(). Defaults to the activation stream inside compute() and the legacy default stream outside a callback. DLPack producers negotiate directly on this stream. Array-interface and C exchange producers retain their advertised readiness and order an explicitly supplied stream after it. Integer 0 is invalid; use 1 or cupy.cuda.Stream.null for the default stream.

Returns: holoscan.Tensor

Examples:

With Tensor input/output ports declared, prefer the SDK stream in ``compute(ctx)``::
    import cupy as cp
    with ctx.stream():
        view = cp.from_dlpack(self.inp.receive_data())
        self.out.emit(view * 2)
If you need a separate stream, keep it on the operator across callbacks and synchronize before
dropping a view, including a view retained from an earlier callback::
    with ctx.stream():
        if not hasattr(self, "_cupy_stream"):
            self._cupy_stream = cp.cuda.Stream(non_blocking=True)
        with self._cupy_stream:
            view = cp.from_dlpack(self.inp.receive_data())
            try:
                result = view * 2
            finally:
                self._cupy_stream.synchronize()
        del view
        self.out.emit(result)

Use with ctx.stream(): for CuPy work in compute(). Prefer as_=cupy on inputs or cupy.from_dlpack(tensor) for conversion. Outside callbacks, keep external consumer streams alive until their exported views are released.

Zero-copy imported storage is permanently read-only. The returned Tensor keeps its producer alive through queued consumer work, including release in a later callback and exception cleanup. Retained views may outlive the session. For work on the SDK callback stream, release a retained view in the operator that last used it, or synchronize that operator’s stream before release elsewhere, including another partition or cyclic garbage collection. Inside callbacks, keep a stream you create alive until the callback returns, and synchronize it before releasing a view you used on it. Keep that stream on the operator so it survives callback-exit completion recording. CUDA array-interface, no-sync C exchange, and no-sync or thread-relative DLPack exports require conservative device completion at final release.

Readiness is captured at import. Finish producing the array before importing it; after subsequent writes on other streams, import again or emit the array directly. Importing a read-only Tensor returns the same object; importing a writable Tensor creates a new read-only wrapper.

The interchange protocol is selected by precedence: the DLPack C exchange API [1]_ first, then the Python DLPack protocol [1], [2], then __cuda_array_interface__ [3], then __array_interface__ [4]. DLPack is preferred because it transfers ownership (so the imported Tensor keeps its data alive independently of the source), has explicit stream-synchronization semantics, and is the array-API-endorsed interchange path. The C exchange API avoids the per-call Python capsule path when a producer type exposes __dlpack_c_exchange_api__.

holoscan.Tensor.from_dlpack(
    obj: object,
    stream: object | None = None
) -> holoscan.Tensor

staticmethod

Create a Tensor from an object supporting the DLPack protocol [1], [2] (zero-copy).

References

.. [1] https://dmlc.github.io/dlpack/latest/python_spec.html .. [2] https://data-apis.org/array-api/latest/API_specification/generated/array_api.array.__dlpack__.html

Parameters:

obj

array - like

An object exposing __dlpack__ and __dlpack_device__.

stream

int or objectDefaults to None

CUDA import stream, as an integer or an object exposing __cuda_stream__(). Defaults to the activation stream inside compute() and the legacy default stream outside a callback. stream=-1 is rejected because it supplies no readiness guarantee. Integer 0 is invalid; use 1 or cupy.cuda.Stream.null for the default stream.

Returns: holoscan.Tensor

Examples:

With Tensor input/output ports declared, prefer the SDK stream in ``compute(ctx)``::
    import cupy as cp
    with ctx.stream():
        view = cp.from_dlpack(self.inp.receive_data())
        self.out.emit(view * 2)
If you need a separate stream, keep it on the operator across callbacks and synchronize before
dropping a view, including a view retained from an earlier callback::
    with ctx.stream():
        if not hasattr(self, "_cupy_stream"):
            self._cupy_stream = cp.cuda.Stream(non_blocking=True)
        with self._cupy_stream:
            view = cp.from_dlpack(self.inp.receive_data())
            try:
                result = view * 2
            finally:
                self._cupy_stream.synchronize()
        del view
        self.out.emit(result)

Zero-copy imported storage is permanently read-only.

CUDA imports negotiate __dlpack__(stream=...) on the selected stream and record readiness with a CUDA event. The Tensor retains the producer’s storage until its views and queued consumer work have finished. SDK callbacks snapshot consumer completion before their streams are destroyed. Outside callbacks, keep external consumer streams alive until their exported views are released. For work on the SDK callback stream, release retained views in the operator that last used them, or synchronize that operator’s stream before release elsewhere. Inside callbacks, keep a stream you create alive until the callback returns, and synchronize it before releasing a view you used on it. Keep that stream on the operator so it survives callback-exit completion recording.

holoscan.Tensor.is_contiguous() -> bool

Whether the tensor has a contiguous, row-major (C-contiguous) memory layout.

Returns: bool

holoscan.payload.resolve(
    name: str
) -> holoscan.payload.PayloadType

holoscan.payload.TensorType: PayloadType = ...

holoscan.payload.boolean: PayloadType = ...

holoscan.payload.float32: PayloadType = ...

holoscan.payload.float64: PayloadType = ...

holoscan.payload.int16: PayloadType = ...

holoscan.payload.int32: PayloadType = ...

holoscan.payload.int64: PayloadType = ...

holoscan.payload.int8: PayloadType = ...

holoscan.payload.pyobject: PayloadType = ...

holoscan.payload.uint16: PayloadType = ...

holoscan.payload.uint32: PayloadType = ...

holoscan.payload.uint64: PayloadType = ...

holoscan.payload.uint8: PayloadType = ...