NVIDIA Morpheus (24.06)
(Latest Version)

Class TensorMemory

Derived Types

class TensorMemory

Container for holding a collection of named TensorObjects in a std::map keyed by name. Base class for InferenceMemory & ResponseMemory

Subclassed by morpheus::InferenceMemory, morpheus::ResponseMemory

Public Functions

TensorMemory(TensorIndex count)

Construct a new Tensor Memory object.

Parameters

count

TensorMemory(TensorIndex count, TensorMap &&tensors)

Construct a new Tensor Memory object.

Parameters
  • count

  • tensors

virtual ~TensorMemory() = default

bool has_tensor(const std::string &name) const

Verify whether the specified tensor name is present in the tensor memory.

Parameters

name

Returns

true

Returns

false

TensorObject &get_tensor(const std::string &name)

Get the tensor object identified by name

Parameters

name

Throws

std::runtime_error – If no tensor matching name exists

Returns

TensorObject&

const TensorObject &get_tensor(const std::string &name) const

Get the tensor object identified by name

Parameters

name

Throws

std::runtime_error – If no tensor matching name exists

Returns

const TensorObject&

void set_tensor(const std::string &name, TensorObject &&tensor)

Set the tensor object identified by name

Parameters
  • name

  • tensor

Throws

std::length_error – If the number of rows in tensor does not match count.

const TensorMap &get_tensors() const

Get a reference to the internal tensors map.

Returns

const TensorMap&

void set_tensors(TensorMap &&tensors)

Set the tensors object.

Parameters

tensors

Throws

std::length_error – If the number of rows in the tensors do not match count.

TensorMap copy_tensor_ranges(const std::vector<RangeType> &ranges, TensorIndex num_selected_rows) const

Copy tensor ranges.

Parameters
  • ranges

  • num_selected_rows

Returns

TensorMap

Public Members

TensorIndex count = {0}

Protected Functions

void check_tensor_length(const TensorObject &tensor)

Checks if the number of rows in tensor matches count

Parameters

tensor

Throws

std::length_error – If the number of rows in tensor do not match count.

void check_tensors_length(const TensorMap &tensors)

Checks each tesnor in tensors verifying that the number of rows matches count.

Parameters

tensors

Throws

std::length_error – If the number of rows in the tensors do not match count.

void verify_tensor_exists(const std::string &name) const

Verify that a tensor identified by name exists, throws a runtime_error othwerwise.

Parameters

name

Throws

std::runtime_error – If no tensor matching name exists

Previous Class Tensor
Next Class TritonInferenceClient
© Copyright 2024, NVIDIA. Last updated on Jul 8, 2024.