nat.llm.litellm_llm

Classes

LiteLlmModelConfig

A LiteLlm provider to be used with an LLM client.

Functions

litellm_model(...)

Litellm model provider.

Module Contents

class LiteLlmModelConfig

Bases: nat.data_models.llm.LLMBaseConfig, nat.data_models.optimizable.OptimizableMixin, nat.data_models.retry_mixin.RetryMixin, nat.data_models.thinking_mixin.ThinkingMixin, nat.data_models.ssl_verification_mixin.SSLVerificationMixin

A LiteLlm provider to be used with an LLM client.

model_config

api_key: nat.data_models.common.OptionalSecretStr = None

base_url: str | None = None

model_name: str

seed: int | None = None

temperature: float | None

top_p: float | None

async litellm_model(

config: LiteLlmModelConfig,

_builder: nat.builder.builder.Builder,

) → collections.abc.AsyncIterator[nat.builder.llm.LLMProviderInfo]

Litellm model provider.

Args:

config (LiteLlmModelConfig): The LiteLlm model configuration. _builder (Builder): The NAT builder instance.

Returns:

AsyncIterator[LLMProviderInfo]: An async iterator that yields an LLMProviderInfo object.