nemo_curator.models.client

Submodules

Package Contents

Classes

Name	Description
`AsyncLLMClient`	Interface representing a client connecting to an LLM inference server
`AsyncOpenAIClient`	A wrapper around OpenAI’s Python async client for querying models
`LLMClient`	Interface representing a client connecting to an LLM inference server
`OpenAIClient`	A wrapper around OpenAI’s Python client for querying models

API

class nemo_curator.models.client.AsyncLLMClient(
    max_concurrent_requests: int = 5,
    max_retries: int = 3,
    base_delay: float = 1.0
)

Abstract

Interface representing a client connecting to an LLM inference server and making requests asynchronously

nemo_curator.models.client.AsyncLLMClient._execute_with_retries(
    request: collections.abc.Callable[[], collections.abc.Awaitable[nemo_curator.models.client.llm_client._T]]
) -> nemo_curator.models.client.llm_client._T

async

Run an async request with the client’s concurrency and retry policy.

nemo_curator.models.client.AsyncLLMClient._query_model_impl(
    messages: collections.abc.Iterable,
    model: str,
    conversation_formatter: nemo_curator.models.client.llm_client.ConversationFormatter | None = None,
    generation_config: nemo_curator.models.client.llm_client.GenerationConfig | dict | None = None
) -> list[str]

asyncabstract

Internal implementation of query_model without retry/concurrency logic. Subclasses should implement this method instead of query_model.

nemo_curator.models.client.AsyncLLMClient.query_model(
    messages: collections.abc.Iterable,
    model: str,
    conversation_formatter: nemo_curator.models.client.llm_client.ConversationFormatter | None = None,
    generation_config: nemo_curator.models.client.llm_client.GenerationConfig | dict | None = None
) -> list[str]

async

Query the model with automatic retry and concurrency control.

nemo_curator.models.client.AsyncLLMClient.setup() -> None

abstract

Setup the client.

class nemo_curator.models.client.AsyncOpenAIClient(
    max_concurrent_requests: int = 5,
    max_retries: int = 3,
    base_delay: float = 1.0,
    kwargs = {}
)

Bases: AsyncLLMClient

A wrapper around OpenAI’s Python async client for querying models

timeout

= kwargs.pop('timeout', 120)

nemo_curator.models.client.AsyncOpenAIClient._query_model_impl(
    messages: collections.abc.Iterable,
    model: str,
    conversation_formatter: nemo_curator.models.client.llm_client.ConversationFormatter | None = None,
    generation_config: nemo_curator.models.client.llm_client.GenerationConfig | dict | None = None
) -> list[str]

async

Internal implementation of query_model without retry/concurrency logic.

nemo_curator.models.client.AsyncOpenAIClient._query_model_response_impl(
    messages: collections.abc.Iterable,
    model: str,
    conversation_formatter: nemo_curator.models.client.llm_client.ConversationFormatter | None = None,
    generation_config: nemo_curator.models.client.llm_client.GenerationConfig | dict | None = None
) -> openai.types.chat.ChatCompletion

async

nemo_curator.models.client.AsyncOpenAIClient.query_model_response(
    messages: collections.abc.Iterable,
    model: str,
    conversation_formatter: nemo_curator.models.client.llm_client.ConversationFormatter | None = None,
    generation_config: nemo_curator.models.client.llm_client.GenerationConfig | dict | None = None
) -> openai.types.chat.ChatCompletion

async

Query a model and return its raw response with retry and concurrency control.

nemo_curator.models.client.AsyncOpenAIClient.setup() -> None

Setup the client.

class nemo_curator.models.client.LLMClient()

Abstract

Interface representing a client connecting to an LLM inference server and making requests synchronously

nemo_curator.models.client.LLMClient.query_model(
    messages: collections.abc.Iterable,
    model: str,
    conversation_formatter: nemo_curator.models.client.llm_client.ConversationFormatter | None = None,
    generation_config: nemo_curator.models.client.llm_client.GenerationConfig | dict | None = None
) -> list[str]

abstract

nemo_curator.models.client.LLMClient.setup() -> None

abstract

Setup the client.

class nemo_curator.models.client.OpenAIClient(
    kwargs = {}
)

Bases: LLMClient

A wrapper around OpenAI’s Python client for querying models

timeout

= kwargs.pop('timeout', 120)

nemo_curator.models.client.OpenAIClient.query_model(
    messages: collections.abc.Iterable,
    model: str,
    conversation_formatter: nemo_curator.models.client.llm_client.ConversationFormatter | None = None,
    generation_config: nemo_curator.models.client.llm_client.GenerationConfig | dict | None = None
) -> list[str]

nemo_curator.models.client.OpenAIClient.setup() -> None

Setup the client.