morpheus.llm.services.llm_service.LLMService

class LLMService[source]

Bases: abc.ABC

Abstract interface for services which are able to construct clients for interacting with LLM models.

Methods

get_client(model_name, **model_kwargs) Returns a client for interacting with a specific model.
abstract get_client(model_name, **model_kwargs)[source]

Returns a client for interacting with a specific model.

Parameters
model_namestr

The name of the model to create a client for.

model_kwargsdict[str, typing.Any]

Additional keyword arguments to pass to the model.

Previous morpheus.llm.services.llm_service.LLMClient
Next morpheus.llm.services.nemo_llm_service
© Copyright 2023, NVIDIA. Last updated on Feb 2, 2024.