morpheus_llm.llm.services.nvfoundation_llm_service.NVFoundationLLMService

class NVFoundationLLMService(*, api_key=None, base_url=None, **model_kwargs)

Bases: LLMService

A service for interacting with NeMo LLM models, this class should be used to create a client for a specific model.

Parameters:

api_keystr, optional

The API key for the LLM service, by default None. If None the API key will be read from the NVIDIA_API_KEY environment variable. If neither are present an error will be raised, by default None

base_urlstr, optional

The api host url, by default None. If None the url will be read from the NVIDIA_API_BASE environment variable. If neither are present the NVIDIA default will be used, by default None

Attributes:

api_key

base_url

Methods

create(service_type, *service_args, ...)	Returns a service for interacting with LLM models.
get_client(*, model_name, **model_kwargs)	Returns a client for interacting with a specific model.

APIKey
BaseURL

class APIKey(value=None, use_env=True)

Bases: EnvConfigValue

Attributes:

source

use_env

value

class BaseURL(value=None, use_env=True)

Bases: EnvConfigValue

Attributes:

source

use_env

value

static create(

service_type,

*service_args,

**service_kwargs,

)

Returns a service for interacting with LLM models.

Parameters:

service_typestr

The type of the service to create

service_argslist

Additional arguments to pass to the service.

service_kwargsdict[str, typing.Any]

Additional keyword arguments to pass to the service.

get_client(*, model_name, **model_kwargs)

Returns a client for interacting with a specific model. This method is the preferred way to create a client.

Parameters:

model_namestr

The name of the model to create a client for.

model_kwargsdict[str, typing.Any]

Additional keyword arguments to pass to the model when generating text.