morpheus.stages.inference.triton_inference_stage.ResourcePool

class ResourcePool(create_fn, max_size=1000)[source]

Bases: Generic[morpheus.stages.inference.triton_inference_stage._T]

This class provides a bounded pool of resources. Users of the pool can borrow a resource where they will get exclusive access to that resource until it is returned. New objects will be created if the pool is empty when a user requets to borrow a resource. If the max size has been hit, the user thread will be blocked until another thread returns a resource.

Parameters
create_fntyping.Callable[[], typing.Any]

Function used to create new resource objects when needed.

max_sizeint, default = 10000

Maximum number of messages in a queue.

Attributes
added_count

The number of items that have been generated by the pool.

Methods

borrow_obj([timeout]) Returns an item from the pool.
return_obj(obj) Returns a borrowed item back to the pool to be used by new calls to borrow().
property added_count

The number of items that have been generated by the pool. Starts at 0 and increases for ever borrow request when the current pool is empty.

Returns
int

Current number of added items.

borrow_obj(timeout=None)[source]

Returns an item from the pool. If the pool is empty, a new item will be created and returned.

Returns
obj

Item from the queue.

return_obj(obj)[source]

Returns a borrowed item back to the pool to be used by new calls to borrow().

Parameters
obj

An item to be added to the queue.

Previous morpheus.stages.inference.triton_inference_stage.InputWrapper
Next morpheus.stages.inference.triton_inference_stage.ShmInputWrapper
© Copyright 2023, NVIDIA. Last updated on Feb 2, 2024.