Build.nvidia.com Model Target

The following target references the NVIDIA Llama 3.1 Nemotron Nano 8B V1 model from build.nvidia.com.

Refer to garak.generators.nim.NVOpenAIChat for the parameters to specify in the options.nim field. The options override the default values from the DEFAULT_PARAMS in the API reference.

Important

Export the NIM_API_KEY environment variable with your API key when you start the microservice container.

Python SDK

import os
from nemo_microservices import NeMoMicroservices

client = NeMoMicroservices(base_url=os.getenv("AUDITOR_BASE_URL"))

target = client.beta.audit.targets.create(
    namespace="default",
    name="demo-build-nvidia-com-target",
    type="nim.NVOpenAIChat",
    model="nvidia/llama-3.1-nemotron-nano-8b-v1",
    options={
        "nim": {
            "skip_seq_start": "<think>",
            "skip_seq_end": "</think>",
            "max_tokens": 3200,
            "uri": "https://integrate.api.nvidia.com/v1/"
        }
    }
)

print(target.model_dump_json(indent=2))

cURL

curl -X POST "${AUDITOR_BASE_URL}/v1beta1/audit/targets" \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -d '{
    "namespace": "default",
    "name": "demo-build-nvidia-com-target",
    "type": "nim.NVOpenAIChat",
    "model": "deepseek-ai/deepseek-r1-distill-llama-8b",
    "options": {
      "nim": {
          "skip_seq_start": "<think>",
          "skip_seq_end": "</think>",
          "max_tokens": 3200,
          "uri": "https://integrate.api.nvidia.com/v1/"
      }
    }
  }' | jq

Example Output

Python SDK

{
  "model": "nvidia/llama-3.1-nemotron-nano-8b-v1",
  "type": "nim.NVOpenAIChat",
  "id": "audit_target-TRmG4hXw22tF945sNixSSN",
  "created_at": "2025-09-24T14:38:49.925517",
  "custom_fields": {},
  "description": null,
  "entity_id": "audit_target-TRmG4hXw22tF945sNixSSN",
  "name": "demo-build-nvidia-com-target",
  "namespace": "default",
  "options": {
    "nim": {
      "skip_seq_start": "<think>",
      "skip_seq_end": "</think>",
      "max_tokens": 3200,
      "uri": "https://integrate.api.nvidia.com/v1/"
    }
  },
  "ownership": null,
  "project": null,
  "schema_version": "1.0",
  "type_prefix": null,
  "updated_at": "2025-09-24T14:38:49.925520"
}

cURL

{
  "schema_version": "1.0",
  "id": "audit_target-7J4xBmoKgZhZouWBNTdHU1",
  "description": null,
  "type_prefix": null,
  "namespace": "default",
  "project": null,
  "created_at": "2025-08-18T12:37:46.858506",
  "updated_at": "2025-08-18T12:37:46.858510",
  "custom_fields": {},
  "ownership": null,
  "name": "demo-build-nvidia-com-target",
  "entity_id": "audit_target-7J4xBmoKgZhZouWBNTdHU1",
  "type": "nim.NVOpenAIChat",
  "model": "deepseek-ai/deepseek-r1-distill-llama-8b",
  "options": {
    "nim": {
      "skip_seq_start": "<think>",
      "skip_seq_end": "</think>",
      "max_tokens": 3200,
      "uri": "https://integrate.api.nvidia.com/v1/"
    }
  }
}