morpheus.stages.inference.triton_inference_stage.ResourcePool

class ResourcePool(create_fn, max_size=1000)[source]

Bases: object

This class provides a bounded pool of resources. Users of the pool can borrow a resource where they will get exclusive access to that resource until it is returned. New objects will be created if the pool is empty when a user requets to borrow a resource. If the max size has been hit, the user thread will be blocked until another thread returns a resource.

Parameters
create_fntyping.Callable[[], typing.Any]

Function used to create new resource objects when needed.

max_sizeint, default = 10000

Maximum number of messages in a queue.

Methods

borrow()

Returns an item from the pool.

return_obj(obj)

Returns a borrowed item back to the pool to be used by new calls to borrow().

borrow()[source]

Returns an item from the pool. If the pool is empty, a new item will be created and returned.

Returns
objtyping.Any

Item from the queue.

return_obj(obj)[source]

Returns a borrowed item back to the pool to be used by new calls to borrow().

Parameters
objtyping.Any

An item to be added to the queue.

self._queue.put(obj)
© Copyright 2023, NVIDIA. Last updated on Feb 3, 2023.