Struct MultiResponseMessageInterfaceProxy

struct MultiResponseMessageInterfaceProxy

Interface proxy, used to insulate python bindings.

Public Static Functions

static std::shared_ptr<MultiResponseMessage> init(std::shared_ptr<MessageMeta> meta, cudf::size_type mess_offset, cudf::size_type mess_count, std::shared_ptr<ResponseMemory> memory, cudf::size_type offset, cudf::size_type count)

Create and initialize a MultiResponseMessage, and return a shared pointer to the result.

Parameters
  • meta – Holds a data table, in practice a cudf DataFrame, with the ability to return both Python and C++ representations of the table

  • mess_offset – Offset into the metadata batch

  • mess_count – Messages count

  • memory – Shared pointer of a tensor memory

  • offset – Message offset in inference memory instance

  • count – Message count in inference memory instance

Returns

std::shared_ptr<MultiResponseMessage>

static std::shared_ptr<ResponseMemory> memory(MultiResponseMessage &self)

GReturns a shared pointer of a response memory probs object.

Returns

std::shared_ptr<ResponseMemory>

static std::size_t offset(MultiResponseMessage &self)

Message offset in response memory probs object.

Parameters

self

Returns

std::size_t

static std::size_t count(MultiResponseMessage &self)

Messages count in response memory probs object.

Parameters

self

Returns

std::size_t

static pybind11::object get_output(MultiResponseMessage &self, const std::string &name)

Returns the output tensor for a given name.

Parameters
  • self

  • name – : Tensor name

Returns

pybind11::object

© Copyright 2023, NVIDIA. Last updated on Feb 3, 2023.