morpheus.llm.services.nemo_llm_service.NeMoLLMService

class NeMoLLMService(*, api_key=None, org_id=None)[source]

Bases: morpheus.llm.services.llm_service.LLMService

A service for interacting with NeMo LLM models, this class should be used to create a client for a specific model.

Parameters
api_keystr, optional

The API key for the LLM service, by default None. If None the API key will be read from the NGC_API_KEY environment variable. If neither are present an error will be raised.

org_idstr, optional

The organization ID for the LLM service, by default None. If None the organization ID will be read from the NGC_ORG_ID environment variable. This value is only required if the account associated with the api_key is a member of multiple NGC organizations.

Methods

get_client(model_name, **model_kwargs) Returns a client for interacting with a specific model.
get_client(model_name, **model_kwargs)[source]

Returns a client for interacting with a specific model. This method is the preferred way to create a client.

Parameters
model_namestr

The name of the model to create a client for.

model_kwargsdict[str, typing.Any]

Additional keyword arguments to pass to the model when generating text.

Previous morpheus.llm.services.nemo_llm_service.NeMoLLMClient
Next morpheus.llm.services.openai_chat_service
© Copyright 2023, NVIDIA. Last updated on Feb 2, 2024.