morpheus.stages.inference.triton_inference_stage.ResourcePool

class ResourcePool(create_fn, max_size=1000)

Bases: Generic[_T]

This class provides a bounded pool of resources. Users of the pool can borrow a resource where they will get exclusive access to that resource until it is returned. New objects will be created if the pool is empty when a user requets to borrow a resource. If the max size has been hit, the user thread will be blocked until another thread returns a resource.

Parameters:

create_fntyping.Callable[[], typing.Any]

Function used to create new resource objects when needed.

max_sizeint, default = 10000

Maximum number of messages in a queue.

Attributes:

added_count

The number of items that have been generated by the pool.

Methods

borrow_obj([timeout])	Returns an item from the pool.
return_obj(obj)	Returns a borrowed item back to the pool to be used by new calls to borrow().

property added_count

The number of items that have been generated by the pool. Starts at 0 and increases for ever borrow request when the current pool is empty.

Returns:

int

Current number of added items.

borrow_obj(timeout=None)

Returns an item from the pool. If the pool is empty, a new item will be created and returned.

Returns:

obj

Item from the queue.

return_obj(obj)

Returns a borrowed item back to the pool to be used by new calls to borrow().

Parameters:

obj

An item to be added to the queue.