- class LLMService[source]
Bases:
abc.ABCAbstract interface for services which are able to construct clients for interacting with LLM models.
Methods
create()Returns a service for interacting with LLM models. get_client(*, model_name, **model_kwargs)Returns a client for interacting with a specific model. - static create(service_type: Literal['nemo'], *service_args, **service_kwargs) → NeMoLLMService[source]
- static create(service_type: Literal['openai'], *service_args, **service_kwargs) → OpenAIChatService
- static create(service_type: str, *service_args, **service_kwargs) → morpheus.llm.services.llm_service.LLMService
Returns a service for interacting with LLM models.
- Parameters
- service_type
- service_args
- service_kwargs
The type of the service to create
Additional arguments to pass to the service.
Additional keyword arguments to pass to the service.
- abstract get_client(*, model_name, **model_kwargs)[source]
Returns a client for interacting with a specific model.
- Parameters
- model_name
- model_kwargs
The name of the model to create a client for.
Additional keyword arguments to pass to the model.
- Returns
- LLMClient
Client for interacting with LLM models.
morpheus.llm.services.llm_service.LLMService
(Latest Version)
Previous
morpheus.llm.services.llm_service.LLMClient
Next
morpheus.llm.services.nemo_llm_service