> This page is for version 26.07 · v1.3.0.
> For other versions, use one of these documentation indexes:
> - Latest · v1.4.0 (26.09) (default): https://docs.nvidia.com/nemo/curator/latest/llms.txt
> - Main · preview: https://docs.nvidia.com/nemo/curator/main/llms.txt
> - 26.09 · v1.4.0: https://docs.nvidia.com/nemo/curator/v26.09/llms.txt
> - 26.07 · v1.3.0: https://docs.nvidia.com/nemo/curator/v26.07/llms.txt
> - 26.04 · v1.2.0: https://docs.nvidia.com/nemo/curator/v26.04/llms.txt
> - 26.02 · v1.1.0: https://docs.nvidia.com/nemo/curator/v26.02/llms.txt
> - 25.09 · v1.0.0: https://docs.nvidia.com/nemo/curator/v25.09/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# nemo_curator.models.client.llm_client

## Module Contents

### Classes

| Name                                                                                    | Description                                                           |
| --------------------------------------------------------------------------------------- | --------------------------------------------------------------------- |
| [`AsyncLLMClient`](#nemo_curator-models-client-llm_client-AsyncLLMClient)               | Interface representing a client connecting to an LLM inference server |
| [`ConversationFormatter`](#nemo_curator-models-client-llm_client-ConversationFormatter) | Represents a way of formatting a conversation with an LLM             |
| [`GenerationConfig`](#nemo_curator-models-client-llm_client-GenerationConfig)           | Configuration class for LLM generation parameters.                    |
| [`LLMClient`](#nemo_curator-models-client-llm_client-LLMClient)                         | Interface representing a client connecting to an LLM inference server |

### Data

[`_T`](#nemo_curator-models-client-llm_client-_T)

### API

```python
class nemo_curator.models.client.AsyncLLMClient(
    max_concurrent_requests: int = 5,
    max_retries: int = 3,
    base_delay: float = 1.0
)
```

Abstract

Interface representing a client connecting to an LLM inference server
and making requests asynchronously

```python
nemo_curator.models.client.AsyncLLMClient._execute_with_retries(
    request: collections.abc.Callable[[], collections.abc.Awaitable[nemo_curator.models.client.llm_client._T]]
) -> nemo_curator.models.client.llm_client._T
```

async

Run an async request with the client's concurrency and retry policy.

```python
nemo_curator.models.client.AsyncLLMClient._query_model_impl(
    messages: collections.abc.Iterable,
    model: str,
    conversation_formatter: nemo_curator.models.client.llm_client.ConversationFormatter | None = None,
    generation_config: nemo_curator.models.client.llm_client.GenerationConfig | dict | None = None
) -> list[str]
```

async

abstract

Internal implementation of query\_model without retry/concurrency logic.
Subclasses should implement this method instead of query\_model.

```python
nemo_curator.models.client.AsyncLLMClient.query_model(
    messages: collections.abc.Iterable,
    model: str,
    conversation_formatter: nemo_curator.models.client.llm_client.ConversationFormatter | None = None,
    generation_config: nemo_curator.models.client.llm_client.GenerationConfig | dict | None = None
) -> list[str]
```

async

Query the model with automatic retry and concurrency control.

```python
nemo_curator.models.client.AsyncLLMClient.setup() -> None
```

abstract

Setup the client.

```python
class nemo_curator.models.client.llm_client.ConversationFormatter()
```

Abstract

Represents a way of formatting a conversation with an LLM
such that it can response appropriately

```python
nemo_curator.models.client.llm_client.ConversationFormatter.format_conversation(
    conv: list[dict]
) -> str
```

abstract

```python
class nemo_curator.models.client.llm_client.GenerationConfig(
    max_tokens: int | None = 2048,
    n: int | None = 1,
    seed: int | None = 0,
    stop: str | None | list[str] = None,
    stream: bool = False,
    temperature: float | None = 0.0,
    top_k: int | None = None,
    top_p: float | None = 0.95,
    extra_kwargs: dict | None = None
)
```

Dataclass

Configuration class for LLM generation parameters.

**`extra_kwargs`** `dict | None = None`

---

**`max_tokens`** `int | None = 2048`

---

**`n`** `int | None = 1`

---

**`seed`** `int | None = 0`

---

**`stop`** `str | None | list[str] = None`

---

**`stream`** `bool = False`

---

**`temperature`** `float | None = 0.0`

---

**`top_k`** `int | None = None`

---

**`top_p`** `float | None = 0.95`

---

```python
class nemo_curator.models.client.LLMClient()
```

Abstract

Interface representing a client connecting to an LLM inference server
and making requests synchronously

```python
nemo_curator.models.client.LLMClient.query_model(
    messages: collections.abc.Iterable,
    model: str,
    conversation_formatter: nemo_curator.models.client.llm_client.ConversationFormatter | None = None,
    generation_config: nemo_curator.models.client.llm_client.GenerationConfig | dict | None = None
) -> list[str]
```

abstract

```python
nemo_curator.models.client.LLMClient.setup() -> None
```

abstract

Setup the client.

```python
nemo_curator.models.client.llm_client._T = TypeVar('_T')
```