NVIDIA Morpheus (24.06)
(Latest Version)

morpheus.llm.services.llm_service.LLMService

class LLMService[source]

Bases: abc.ABC

Abstract interface for services which are able to construct clients for interacting with LLM models.

Methods

create() Returns a service for interacting with LLM models.
get_client(*, model_name, **model_kwargs) Returns a client for interacting with a specific model.
static create(service_type: Literal['nemo'], *service_args, **service_kwargs)NeMoLLMService[source]
static create(service_type: Literal['openai'], *service_args, **service_kwargs)OpenAIChatService
static create(service_type: str, *service_args, **service_kwargs)morpheus.llm.services.llm_service.LLMService

Returns a service for interacting with LLM models.

Parameters
service_type

The type of the service to create

service_args

Additional arguments to pass to the service.

service_kwargs

Additional keyword arguments to pass to the service.

abstract get_client(*, model_name, **model_kwargs)[source]

Returns a client for interacting with a specific model.

Parameters
model_name

The name of the model to create a client for.

model_kwargs

Additional keyword arguments to pass to the model.

Returns
LLMClient

Client for interacting with LLM models.

Previous morpheus.llm.services.llm_service.LLMClient
Next morpheus.llm.services.nemo_llm_service
© Copyright 2024, NVIDIA. Last updated on Jul 8, 2024.