nat.llm.utils.http_client

Classes

HttpClients

Return type for the http_clients context manager.

Functions

_create_http_client(→ httpx.AsyncClient | httpx.Client)	Create an httpx client with timeout and verify setting based on LLM configuration parameters.
http_client(→ httpx.Client)	Context manager for a synchronous httpx client, to ensure that the client is properly closed after use.
async_http_client(→ httpx.AsyncClient)	Async context manager for an asynchronous httpx client, to ensure that the client is properly closed after use.
http_clients(→ HttpClients)	Get a dictionary of HTTP clients, one sync one async.
_handle_litellm_verify_ssl(→ None)	Disable SSL verification for litellm if verify_ssl is set to False in the LLM configuration.

Module Contents

_create_http_client(

llm_config: nat.data_models.llm.LLMBaseConfig,

use_async: bool = True,

\*\*kwargs,

) → httpx.AsyncClient | httpx.Client

Create an httpx client with timeout and verify setting based on LLM configuration parameters.

Args:

llm_config: LLM configuration object use_async: Whether to create an AsyncClient (True) or a regular Client (False). Defaults to True.

Returns:

An httpx.AsyncClient or httpx.Client

http_client(

llm_config: nat.data_models.llm.LLMBaseConfig,

\*\*kwargs,

) → httpx.Client

Context manager for a synchronous httpx client, to ensure that the client is properly closed after use.

async async_http_client(

llm_config: nat.data_models.llm.LLMBaseConfig,

\*\*kwargs,

) → httpx.AsyncClient

Async context manager for an asynchronous httpx client, to ensure that the client is properly closed after use.

class HttpClients

Bases: TypedDict

Return type for the http_clients context manager.

Initialize self. See help(type(self)) for accurate signature.

http_client: httpx.Client

async_http_client: httpx.AsyncClient

async http_clients(

llm_config: nat.data_models.llm.LLMBaseConfig,

\*\*kwargs,

) → HttpClients

Get a dictionary of HTTP clients, one sync one async.

This is a wrapper around async_http_client and http_client, useful for LLMs that support both sync and async clients.

_handle_litellm_verify_ssl(

llm_config: nat.data_models.llm.LLMBaseConfig,

) → None

Disable SSL verification for litellm if verify_ssl is set to False in the LLM configuration.

Currently litellm does not support disabling this on a per-LLM basis for any backend other than Bedrock and AIM Guardrail, calling this function will set the global litellm.ssl_verify and impact all subsequent litellm calls.