Basic Audit Target

When you run an audit job in NVIDIA NeMo Auditor, you create a separate audit target and audit configuration for the job.

The target specifies the model name, model type, and free-form key-value pairs for model-specific inference options. For more information, refer to Schema for Audit Targets.

The following target references the DeepSeek R1 Distill Llama 8B model from a locally-accessible NIM microservice. The service name for the LLM container is llm.

Set AUDITOR_BASE_URL to specify the service:

$ export AUDITOR_BASE_URL=http://localhost:8080

Python SDK

import os
from nemo_microservices import NeMoMicroservices

client = NeMoMicroservices(base_url=os.getenv("AUDITOR_BASE_URL"))

target = client.beta.audit.targets.create(
    namespace="default",
    name="demo-basic-target",
    type="nim.NVOpenAIChat",
    model="deepseek-ai/deepseek-r1-distill-llama-8b",
    options={
        "nim": {
            "skip_seq_start": "<think>",
            "skip_seq_end": "</think>",
            "max_tokens": 3200,
            "uri": "https://integrate.api.nvidia.com/v1/"
        }
    }
)

print(target.model_dump_json(indent=2))

cURL

curl -X POST "${AUDITOR_BASE_URL}/v1beta1/audit/targets" \
  -H "Accept: application/json" \
  -H "Content-Type: application/json" \
  -d '{
    "namespace": "default",
    "name": "demo-basic-target",
    "type": "nim.NVOpenAIChat",
    "model": "deepseek-ai/deepseek-r1-distill-llama-8b",
    "options": {
      "nim": {
          "skip_seq_start": "<think>",
          "skip_seq_end": "</think>",
          "max_tokens": 3200,
          "uri": "https://integrate.api.nvidia.com/v1/"
      }
    }
  }' | jq