morpheus_llm.llm.services.llm_service.LLMClient#

class LLMClient[source]#

Bases: ABC

Abstract interface for clients which are able to interact with LLM models. Concrete implementations of this class will have an associated implementation of LLMService which is able to construct instances of this class.

Methods

`generate`(**input_dict)	Issue a request to generate a response based on a given prompt.
`generate_async`(**input_dict)	Issue an asynchronous request to generate a response based on a given prompt.
`generate_batch`()	Issue a request to generate a list of responses based on a list of prompts.
`generate_batch_async`()	Issue an asynchronous request to generate a list of responses based on a list of prompts.
`get_input_names`()	Returns the names of the inputs to the model.

abstractmethod generate(**input_dict)[source]#

Issue a request to generate a response based on a given prompt.

Parameters:

input_dictdict: Input containing prompt data.

Returns:

str: Generated response for prompt.

abstractmethod async generate_async(**input_dict)[source]#

Issue an asynchronous request to generate a response based on a given prompt.

Parameters:

input_dictdict: Input containing prompt data.

Returns:

str: Generated async response for prompt.

abstractmethod generate_batch( inputs: dict[str, list], return_exceptions: Literal[True], ) → list[str | BaseException][source]#

abstractmethod generate_batch( inputs: dict[str, list], return_exceptions: Literal[False], ) → list[str]

abstractmethod generate_batch( inputs: dict[str, list], return_exceptions: bool = False, ) → list[str] | list[str | BaseException]

Issue a request to generate a list of responses based on a list of prompts.

Parameters:

inputsdict: Inputs containing prompt data.
return_exceptionsbool: Whether to return exceptions in the output list or raise them immediately.

Returns:

list[str] | list[str | BaseException]: List of responses or list of responses and exceptions.

abstractmethod async generate_batch_async( inputs: dict[str, list], return_exceptions: Literal[True], ) → list[str | BaseException][source]#

abstractmethod async generate_batch_async( inputs: dict[str, list], return_exceptions: Literal[False], ) → list[str]

abstractmethod async generate_batch_async( inputs: dict[str, list], return_exceptions: bool = False, ) → list[str] | list[str | BaseException]

Issue an asynchronous request to generate a list of responses based on a list of prompts.

Parameters:

inputsdict: Inputs containing prompt data.
return_exceptionsbool: Whether to return exceptions in the output list or raise them immediately.

Returns:

list[str] | list[str | BaseException]: List of responses or list of responses and exceptions.

abstractmethod get_input_names()[source]#

Returns the names of the inputs to the model.

Returns:

list[str]: List of input names.