Schema for Audit Targets#

Field

Description

Default Value

name

Specifies the name of the audit target. Names must be unique within a namespace. The maximum length is 250 characters.

None (Required)

namespace

Specifies the namespace for the audit target. The maximum length is 250 characters.

None (Required)

options

Specifies a dictionary of model-specific inference options. Specify the type as the top-level key and then the key-value pairs.

The following sample JSON is valid when type is nim:

"options": {
  "nim": {
    "context_len": null,
    "extra_params": {},
    "frequency_penalty": 0.0,
    "max_tokens": 150,
    "presence_penalty": 0.0,
    "retry_json": true,
    "seed": null,
    "skip_seq_end": null,
    "skip_seq_start": null,
    "stop": ["#", ";"],
    "suppressed_params": [
       "frequency_penalty",
       "n",
       "presence_penalty",
       "timeout"
     ],
     "temperature": 0.1,
     "top_k": 0,
     "top_p": 0.7,
     "uri": "https://integrate.api.nvidia.com/v1/",
     "vary_seed_each_call": true,
     "vary_temp_each_call": true
  }
}

To access models from NeMo NIM Proxy, set the service name as the uri:

"generator_config": {
  "nim": {
    "uri": "http://nemo-nim-proxy:8000/v1/"
  }
}

Refer to the DEFAULT_PARAMS for the generator class:

model

Specifies the model to audit, such as o3-mini or meta/llama-3.1-8b-instruct.

None (Required)

type

Specifies the type of model to audit. These correspond to a garak generator.

The microservice supports a subset of the generators that garak supports. Specify one of the following generators:

  • openai

  • nim

  • rest

None (Required)