nemo_curator.models.omni.base

NVIDIA Inference API client for reasoning VLMs.

Reasoning models on NVIDIA Inference (e.g. Nemotron-Nano-Omni-Reasoning) split their output into delta.reasoning_content (chain-of-thought) and delta.content (the final answer), and their non-stream response shape is not deserialized cleanly by the OpenAI SDK. This client therefore streams and reassembles only delta.content.

Module Contents

Classes

Name	Description
`NVInferenceClient`	Async OpenAI-compatible client that streams reasoning-model output.

Data

_PRIORITY_HEADER

API

class nemo_curator.models.omni.base.NVInferenceClient(
    base_url: str = 'https://integrate.api.nvid...,
    api_key_env_var: str = 'NVINFERENCE_API_KEY',
    priority_mode: bool = False,
    max_concurrent_requests: int = 10,
    timeout: int = 120
)

Bases: AsyncOpenAIClient

Async OpenAI-compatible client that streams reasoning-model output.

Resolves the API key from api_key_env_var at setup() time (so the key is read on the worker, not serialized from the driver), then reassembles delta.content from a streaming completion.

nemo_curator.models.omni.base.NVInferenceClient._query_model_impl(
    messages: collections.abc.Iterable,
    model: str,
    conversation_formatter: nemo_curator.models.client.llm_client.ConversationFormatter | None = None,
    generation_config: nemo_curator.models.client.llm_client.GenerationConfig | dict | None = None
) -> list[str]

async

nemo_curator.models.omni.base.NVInferenceClient.setup() -> None

nemo_curator.models.omni.base._PRIORITY_HEADER = {'X-Vertex-AI-LLM-Shared-Request-Type': 'priority'}