Build.nvidia.com Model Target

The following target references the Deepseek R1 Distill Llama 8B model from build.nvidia.com.

Refer to garak.generators.nim.NVOpenAIChat for the parameters to specify in the options.nim field. The options override the default values from the DEFAULT_PARAMS in the API reference.

Important

Export the NIM_API_KEY environment variable with your API key when you start the microservice container.

Python SDK

    import os
    from nemo_microservices import NeMoMicroservices

    client = NeMoMicroservices(base_url=os.getenv("AUDITOR_BASE_URL"))

    target = client.beta.audit.targets.create(
        namespace="default",
        name="demo-build-nvidia-com-target",
        type="nim.NVOpenAIChat",
        model="nvidia/llama-3.1-nemotron-nano-8b-v1",
        options={
            "nim": {
                "skip_seq_start": "<think>",
                "skip_seq_end": "</think>",
                "max_tokens": 3200,
                "uri": "https://integrate.api.nvidia.com/v1/"
            }
        }
    )

    print(target)

cURL

curl -X POST "${AUDITOR_BASE_URL}/v1beta1/audit/targets" \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -d '{
    "namespace": "default",
    "name": "demo-build-nvidia-com-target",
    "type": "nim.NVOpenAIChat",
    "model": "deepseek-ai/deepseek-r1-distill-llama-8b",
    "options": {
      "nim": {
          "skip_seq_start": "<think>",
          "skip_seq_end": "</think>",
          "max_tokens": 3200,
          "uri": "https://integrate.api.nvidia.com/v1/"
      }
    }
  }' | jq

Example Output

Python SDK

AuditTarget(model='nvidia/llama-3.1-nemotron-nano-8b-v1',
type='nim.NVOpenAIChat', id='audit_target-GbB1D5KLF4QUDm8y8w9yoK',
created_at=datetime.datetime(2025, 8, 18, 12, 32, 57, 686007),
custom_fields={}, description=None, entity_id='audit_target-
GbB1D5KLF4QUDm8y8w9yoK', name='demo-build-nvidia-com-target',
namespace='default', options={'nim': {'skip_seq_start': '<think>',
'skip_seq_end': '</think>', 'max_tokens': 3200, 'uri':
'https://integrate.api.nvidia.com/v1/'}}, ownership=None, project=None,
schema_version='1.0', type_prefix=None, updated_at=datetime.datetime(2025,
8, 18, 12, 32, 57, 686011))

cURL

{
  "schema_version": "1.0",
  "id": "audit_target-7J4xBmoKgZhZouWBNTdHU1",
  "description": null,
  "type_prefix": null,
  "namespace": "default",
  "project": null,
  "created_at": "2025-08-18T12:37:46.858506",
  "updated_at": "2025-08-18T12:37:46.858510",
  "custom_fields": {},
  "ownership": null,
  "name": "demo-build-nvidia-com-target",
  "entity_id": "audit_target-7J4xBmoKgZhZouWBNTdHU1",
  "type": "nim.NVOpenAIChat",
  "model": "deepseek-ai/deepseek-r1-distill-llama-8b",
  "options": {
    "nim": {
      "skip_seq_start": "<think>",
      "skip_seq_end": "</think>",
      "max_tokens": 3200,
      "uri": "https://integrate.api.nvidia.com/v1/"
    }
  }
}