NVIDIA Morpheus (24.06)
(Latest Version)

morpheus.llm.services.nemo_llm_service.NeMoLLMService

class NeMoLLMService(*, api_key=None, org_id=None, base_url=None, retry_count=5)[source]

Bases: morpheus.llm.services.llm_service.LLMService

A service for interacting with NeMo LLM models, this class should be used to create a client for a specific model.

Parameters
api_key

The API key for the LLM service, by default None. If None the API key will be read from the NGC_API_KEY environment variable. If neither are present an error will be raised., by default None

org_id

The organization ID for the LLM service, by default None. If None the organization ID will be read from the NGC_ORG_ID environment variable. This value is only required if the account associated with the api_key is a member of multiple NGC organizations, by default None

base_url

The api host url, by default None. If None the url will be read from the NGC_BASE_URL environment variable. If neither are present the NeMo default will be used, by default None

retry_count

The number of times to retry a request before raising an exception, by default 5

Methods

create(service_type, *service_args, ...) Returns a service for interacting with LLM models.
get_client(*, model_name, **model_kwargs) Returns a client for interacting with a specific model.
APIKey
BaseURL
OrgId
class APIKey(value=None, use_env=True)[source]

Bases: morpheus.utils.env_config_value.EnvConfigValue

Attributes
source

use_env

value

class BaseURL(value=None, use_env=True)[source]

Bases: morpheus.utils.env_config_value.EnvConfigValue

Attributes
source

use_env

value

class OrgId(value=None, use_env=True)[source]

Bases: morpheus.utils.env_config_value.EnvConfigValue

Attributes
source

use_env

value

static create(service_type, *service_args, **service_kwargs)[source]

Returns a service for interacting with LLM models.

Parameters
service_type

The type of the service to create

service_args

Additional arguments to pass to the service.

service_kwargs

Additional keyword arguments to pass to the service.

get_client(*, model_name, **model_kwargs)[source]

Returns a client for interacting with a specific model. This method is the preferred way to create a client.

Parameters
model_name

The name of the model to create a client for.

model_kwargs

Additional keyword arguments to pass to the model when generating text.

Previous morpheus.llm.services.nemo_llm_service.NeMoLLMClient
Next morpheus.llm.services.nvfoundation_llm_service
© Copyright 2024, NVIDIA. Last updated on Jul 8, 2024.