- class LLMClient[source]
Bases:
abc.ABC
Abstract interface for clients which are able to interact with LLM models. Concrete implementations of this class will have an associated implementation of
LLMService
which is able to construct instances of this class.Methods
generate
(**input_dict)Issue a request to generate a response based on a given prompt. generate_async
(**input_dict)Issue an asynchronous request to generate a response based on a given prompt. generate_batch
()Issue a request to generate a list of responses based on a list of prompts. generate_batch_async
()Issue an asynchronous request to generate a list of responses based on a list of prompts. get_input_names
()Returns the names of the inputs to the model. - abstract generate(**input_dict)[source]
Issue a request to generate a response based on a given prompt.
- Parameters
- input_dict
Input containing prompt data.
- abstract async generate_async(**input_dict)[source]
Issue an asynchronous request to generate a response based on a given prompt.
- Parameters
- input_dict
Input containing prompt data.
- abstract generate_batch(inputs: dict[str, list], return_exceptions: Literal[True] = True) → list[str | BaseException][source]
- abstract generate_batch(inputs: dict[str, list], return_exceptions: Literal[False] = False) → list[str]
Issue a request to generate a list of responses based on a list of prompts.
- Parameters
- inputs
- return_exceptions
Inputs containing prompt data.
Whether to return exceptions in the output list or raise them immediately.
- abstract async generate_batch_async(inputs: dict[str, list], return_exceptions: Literal[True] = True) → list[str | BaseException][source]
- abstract async generate_batch_async(inputs: dict[str, list], return_exceptions: Literal[False] = False) → list[str]
Issue an asynchronous request to generate a list of responses based on a list of prompts.
- Parameters
- inputs
- return_exceptions
Inputs containing prompt data.
Whether to return exceptions in the output list or raise them immediately.
- abstract get_input_names()[source]
Returns the names of the inputs to the model.
morpheus.llm.services.llm_service.LLMClient
Previous
morpheus.llm.services.llm_service
Next
morpheus.llm.services.llm_service.LLMService