Schema for Audit Targets#

Field

Description

Default Value

name

Specifies the name of the audit target. Names must be unique within a workspace. The maximum length is 250 characters.

None (Required)

workspace

Specifies the workspace for the audit target. The maximum length is 250 characters.

None (Required)

options

Specifies a dictionary of model-specific inference options. Specify the type as the top-level key and then the key-value pairs.

The following sample JSON is valid when type is nim:

"options": {
  "nim": {
    "context_len": null,
    "extra_params": {},
    "frequency_penalty": 0.0,
    "max_tokens": 150,
    "presence_penalty": 0.0,
    "retry_json": true,
    "seed": null,
    "skip_seq_end": null,
    "skip_seq_start": null,
    "stop": ["#", ";"],
    "suppressed_params": [
       "frequency_penalty",
       "n",
       "presence_penalty",
       "timeout"
     ],
     "temperature": 0.1,
     "top_k": 0,
     "top_p": 0.7,
     "nmp_uri_spec": {"inference_gateway": {"workspace": "default", "provider": "build"}},
     "vary_seed_each_call": true,
     "vary_temp_each_call": true
  }
}

To access models from NeMo Inference Gateway, use nmp_uri_spec under options.

"options": {
  "nim": {
    "nmp_uri_spec": {"inference_gateway": {"workspace": "default", "provider": "build"}},
  }
}

Refer to the DEFAULT_PARAMS for the generator class:

model

Specifies the model to audit, such as o3-mini or meta/llama-3.1-8b-instruct.

None (Required)

type

Specifies the type of model to audit. These correspond to a garak generator.

The microservice supports a subset of the generators that garak supports. Specify one of the following generators:

  • openai

  • nim

  • rest

None (Required)