> This page is for version 0.11.
> For other versions, use one of these documentation indexes:
> - Latest (default): https://docs.nvidia.com/nemo-helix/latest/llms.txt
> - 0.11: https://docs.nvidia.com/nemo-helix/v0.11/llms.txt
> - 0.10: https://docs.nvidia.com/nemo-helix/v0.10/llms.txt
> - 0.9: https://docs.nvidia.com/nemo-helix/v0.9/llms.txt
> - 0.8: https://docs.nvidia.com/nemo-helix/v0.8/llms.txt
> - 0.7: https://docs.nvidia.com/nemo-helix/v0.7/llms.txt
> - 0.6: https://docs.nvidia.com/nemo-helix/v0.6/llms.txt
> - 0.5: https://docs.nvidia.com/nemo-helix/v0.5/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo-helix/llms.txt.

# Model Configuration

Online evaluations use `Model` objects for model endpoints. A model can be the evaluation target that produces outputs, or it can be part of a judge-style metric such as LLM-as-a-Judge, RAG, or agentic metrics.

The Evals plugin SDK uses inline model objects from `nhx_evals_sdk`. Pass the model either as `target=...` or as a field on the metric class that needs a judge or embeddings model.

## Initialize the SDK

```python
import os

from nemo_evals.sdk import Evaluator
from nemo_helix_plugin.client.client import NemoClient

client = NemoClient.from_config()
evaluator = Evaluator.from_client(client)
```

> **Note**
>
> These examples use the `default` workspace. If your current CLI context uses a different workspace, run `nemo config set --workspace default` first. To check, run `nemo config view`.

## Inline Model

Define the endpoint URL and model name directly:

Set `NVIDIA_BASE_URL` to override the full OpenAI-compatible API base URL (including `/v1`),
for example `https://inference-api.nvidia.com/v1`. The default remains `https://integrate.api.nvidia.com/v1`.

```python
import os

from nhx_evals_sdk import Model, SecretRef

model = Model(
    url=os.environ.get("NVIDIA_BASE_URL", "https://integrate.api.nvidia.com/v1").rstrip("/"),
    name="nvidia/nemotron-3.5-lightning-30b-a3b",
    api_key_secret=SecretRef(root="nvidia-api-key"),
)
```

| Field            | Required | Description                                                                                                                                |
| ---------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| `url`            | Yes      | Base URL of the inference endpoint.                                                                                                        |
| `name`           | Yes      | Model name to send in inference requests.                                                                                                  |
| `format`         | No       | **Deprecated and ignored.** Structured output support is detected from the endpoint during preflight rather than inferred from this label. |
| `api_key_secret` | No       | Model API key reference. See [Model API Authentication](#model-api-authentication).                                                        |

## Model API Authentication

`api_key_secret` is an optional property on the `Model` object. Omit it when the endpoint does not require API-key authentication.

For `nhx_evals_sdk.Evaluator` runs, the `SecretRef` root names an environment variable available to the local Python process. For example, `api_key_secret=SecretRef(root="NVIDIA_API_KEY")` reads `os.environ["NVIDIA_API_KEY"]`.

For remote `evaluator.submit(...)` jobs, the `SecretRef` root must name a NeMo Helix secret in the target workspace. Create the secret before submitting the job:

```python
from pydantic import SecretStr
from nemo_helix_plugin.secrets.types import HelixSecretCreateRequest

client.secrets.create_secret(
    body=HelixSecretCreateRequest(
        name="nvidia-api-key",
        value=SecretStr(os.environ["NVIDIA_API_KEY"]),
    ),
)
```

## Model as the Evaluation Target

Use `target=model` when the evaluator should call the model to generate the sample output before scoring.

```python
import os

from nhx_evals_sdk import (
    RunConfigOnlineModel,
    ExactMatchMetric,
    InferenceParams,
    Model,
    SecretRef,
)

model = Model(
    url=os.environ.get("NVIDIA_BASE_URL", "https://integrate.api.nvidia.com/v1").rstrip("/"),
    name="nvidia/nemotron-3.5-lightning-30b-a3b",
    api_key_secret=SecretRef(root="nvidia-api-key"),
)

metric = ExactMatchMetric(reference="{{item.expected_answer}}")
dataset = [
    {"question": "What is the capital of France?", "expected_answer": "Paris"},
]

job = evaluator.submit(
    metric=metric,
    dataset=dataset,
    config=RunConfigOnlineModel(
        parallelism=4,
        inference=InferenceParams(temperature=0.1, max_tokens=64),
    ),
    target=model,
    prompt_template={
        "messages": [
            {
                "role": "user",
                "content": "Answer this question concisely: {{item.question}}",
            },
        ],
    },
)
job.wait_until_done()
result = job.get_result()
```

## Model on a Judge Metric

Use a model field on the metric when the metric itself calls an LLM to score existing outputs.

```python
import os

from nhx_evals_sdk import LLMJudgeMetric, Model, RangeScore, SecretRef

judge_model = Model(
    url=os.environ.get("NVIDIA_BASE_URL", "https://integrate.api.nvidia.com/v1").rstrip("/"),
    name="nvidia/nemotron-3.5-lightning-30b-a3b",
    api_key_secret=SecretRef(root="nvidia-api-key"),
)
metric = LLMJudgeMetric(
    model=judge_model,
    scores=[
        RangeScore(
            name="correctness",
            description="Correctness from 1 to 5.",
            minimum=1,
            maximum=5,
        ),
    ],
    prompt_template={
        "messages": [
            {
                "role": "system",
                "content": "Return JSON with a correctness score from 1 to 5.",
            },
            {
                "role": "user",
                "content": "Question: {{item.question}}\nAnswer: {{item.output}}\nExpected: {{item.expected_answer}}",
            },
        ],
    },
)

job = evaluator.submit(
    metric=metric,
    dataset=[
        {
            "question": "What is the capital of France?",
            "output": "Paris",
            "expected_answer": "Paris",
        },
    ],
)
job.wait_until_done()
result = job.get_result()
```

## Runtime Parameters

Use `RunConfigOnlineModel` for model-target evaluations:

```python
from nhx_evals_sdk import (
    RunConfigOnlineModel,
    InferenceParams,
    ReasoningParams,
)

params = RunConfigOnlineModel(
    parallelism=4,
    request_timeout=60,
    max_retries=2,
    ignore_request_failure=False,
    inference=InferenceParams(temperature=0.2, max_tokens=256),
    reasoning=ReasoningParams(end_token="</think>"),
)
```

Use plain `RunConfig` for offline evaluations where the dataset already contains the output to score.

## Model References

You can supply the evaluation target two ways. Which one is valid depends on whether you submit the evaluation as a durable platform job or run it locally with `nhx_evals_sdk.Evaluator`.

### Inline `Model`

An inline `Model` carries the resolved endpoint details. Always pass an inline `Model` as the `target` (or as a judge/embeddings field on the metric). If your deployment stores platform model entities, resolve the entity into endpoint details before constructing the `Model`:

```python
from nhx_evals_sdk import Model, RunConfigOnlineModel, SecretRef

model_entity = client.models.get_model(name="my-model", workspace="default").data()
model = Model(
    url=client.models.get_model_entity_route_openai_url(model_entity),
    name="my-model",
    api_key_secret=SecretRef(root="nvidia-api-key"),
)

job = evaluator.submit(
    metric=metric,
    dataset=dataset,
    config=RunConfigOnlineModel(parallelism=4),
    target=model,
    prompt_template={
        "messages": [
            {
                "role": "user",
                "content": "Answer this question concisely: {{item.question}}",
            },
        ],
    },
)
job.wait_until_done()
result = job.get_result()
```

### `ModelRef` (supported by `evaluator.submit(...)`)

Durable remote `evaluator.submit(...)` jobs additionally accept a `ModelRef` target. A `ModelRef` names a model the Inference Gateway serves, a platform model entity or a virtual model (`workspace/name`), and is resolved by the evaluator backend when the job runs, so you do not have to resolve the endpoint yourself. Use this for platform-managed model routing. A `ModelRef` target generates outputs online, so it requires an online run config (`RunConfigOnlineModel`):

```python
from nhx_evals_sdk import ModelRef, RunConfigOnlineModel

job = evaluator.submit(
    metric=metric,
    dataset=dataset,
    config=RunConfigOnlineModel(),
    target=ModelRef(root="default/my-model"),
    prompt_template={
        "messages": [
            {
                "role": "user",
                "content": "Answer this question concisely: {{item.question}}",
            },
        ],
    },
)
```

`evaluator.submit(...)` accepts either a `Model` or a `ModelRef`. `nhx_evals_sdk.Evaluator` resolves no platform entities, so pass it an inline `Model`. See the [Define and Run Custom Python Metrics](/nemo-helix/v0.11/documentation/evaluate-models/tutorials/define-and-run-custom-python-metrics) tutorial for an end-to-end `ModelRef` + `FilesetRef` submit example.

> **Note**
>
> For evaluating agentic systems, use an `Agent` request target instead of a `Model`. See [Agent Configuration](/nemo-helix/v0.11/documentation/evaluate-models/metrics/agent-configuration).

## Images in the Prompt

A vision-language model reads images from `image_url` parts in a chat message. A request through the Inference Gateway can give the image in three forms:

* `data:image/png;base64,...`: the image bytes, inline in the request.
* `http(s)://...`: the model server downloads the image itself.
* `fileset://<workspace>/<fileset>#<path>`: the gateway downloads the file from the fileset with the caller's permissions, and inlines it as a `data:` URL. PNG, JPEG, GIF, or WebP, up to 25 MiB.

Dataset rows usually hold image paths relative to the fileset they were uploaded to, which is what training reads. Write the `fileset://` reference in the prompt template, and leave the rows as they are. This example scores a receipt-reading model on the CORD-v2 validation split. Each row's user turn is one image part followed by one text part, and its assistant turn is the expected answer:

```python
from nemo_evals.sdk import FilesetRef
from nhx_evals_sdk import InferenceParams, ModelRef, RunConfigOnlineModel, SequenceSimilarityMetric

# An example row: {"messages": [
#   {"role": "user", "content": [
#     {"type": "image_url", "image_url": {"url": "validation/images/00000.png"}},
#     {"type": "text", "text": "Describe this image."}]},
#   {"role": "assistant", "content": "<s_total><s_total_price>45,500</s_total_price>..."}]}
job = evaluator.submit(
    metric=SequenceSimilarityMetric(
        reference="{{ item.messages[-1].content }}",
        candidate="{{ sample.output_text | trim }}",
    ),
    dataset=FilesetRef(root="default/cord-v2#validation/validation.jsonl"),
    config=RunConfigOnlineModel(
        inference=InferenceParams(temperature=0, max_tokens=1024),
    ),
    target=ModelRef(root="default/my-vision-model"),
    prompt_template={
        "messages": [
            {
                "role": "user",
                "content": [
                    {
                        "type": "image_url",
                        "image_url": {
                            "url": "fileset://default/cord-v2#{{ item.messages[0].content[0].image_url.url }}"
                        },
                    },
                    {"type": "text", "text": "{{ item.messages[0].content[1].text }}"},
                ],
            },
        ],
    },
)
```

For the first row, the request carries `fileset://default/cord-v2#validation/images/00000.png`. The job downloads only the JSONL, and the gateway fetches each image as its request arrives, so saved results hold the short reference rather than the image bytes.

* The template's indices follow the row shown in the comment: `messages[0]` is the user turn, `content[0]` is its image part, and `content[1]` is its text part. Rows with a different shape need different indices.
* The request must go through the Inference Gateway, as a `ModelRef` target's requests do. A model server called directly reads only `data:` and `http(s)` URLs.
* The template names the fileset, so a copy of the dataset under another name needs the template changed.
* When rows already hold `data:`, `http(s)`, or `fileset://` URLs, `{"messages": "{{ item.messages[:-1] }}"}` sends their image parts unchanged.