Manage Audit Configurations

An AuditConfig selects which garak probes and detectors run during an audit, how many generations to make per probe, and where the reports land. Configurations are persisted in NeMo Helix entity store and referenced by name from garak.run(...).

Initialize the SDK

from nemo_garak.sdk import GarakPluginResource
from nemo_helix_plugin.client.client import NemoClient
client = NemoClient.from_config()
garak = GarakPluginResource(client)

These examples use the default workspace. If your current CLI context uses a different workspace, run nemo config set --workspace default first. To check, run nemo config view.

What an AuditConfig Holds

A configuration has four sub-blocks plus a free-form description. Each sub-block has sensible defaults, so a minimal configuration only needs a workspace and a name.

Sub-block	What it controls
`system`	garak runtime knobs — verbosity, parallelism, lite mode, experimental flags.
`run`	Per-probe generation count, random seed, evaluation threshold, optional `probe_tags` filter.
`plugins`	The probes and detectors to run (`probe_spec`, `detector_spec`) and any plugin-specific options.
`reporting`	Output file prefix and directory, optional taxonomy, and which modules are surfaced in summaries.

The full field reference is in Configuration Schema.

The Seeded Default

The plugin seeds a default configuration in the system workspace at install time. Retrieve it to see a reasonable starting point:

default_config = garak.configs.get(workspace="system", name="default")
print(default_config.model_dump_json(indent=2))

You can pass "system/default" directly to garak.run(config=..., ...) to run an audit with the seeded configuration without creating your own.

Create a Configuration

Build a configuration by populating any of the four sub-blocks. Omitted sub-blocks fall back to their pydantic defaults.

from nemo_garak.entities import (
    AuditPluginsData,
    AuditReportData,
    AuditRunData,
    AuditSystemData,
)
config = garak.configs.create(
    workspace="default",
    name="quick-scan",
    description="Bounded scan using one probe and one generation.",
    system=AuditSystemData(lite=True, parallel_attempts=4),
    run=AuditRunData(generations=1),
    plugins=AuditPluginsData(probe_spec="goodside.Tag", detector_spec="auto"),
    reporting=AuditReportData(report_prefix="quick-scan"),
)

The probe_spec field takes a comma-separated list of probe categories or fully-qualified probe classes. See Selecting Probes for the syntax and worked examples.

List Configurations

listing = garak.configs.list(workspace="default")
for entry in listing["data"]:
    print(entry["name"], entry["description"])

list() returns the standard envelope (data, pagination, sort); pass page, page_size, and sort keyword arguments for pagination.

Retrieve a Configuration

config = garak.configs.get(workspace="default", name="quick-scan")

Update a Configuration

update() replaces every sub-block — any sub-block you omit resets to its default. Pass each block you want to keep.

updated = garak.configs.update(
    workspace="default",
    name="quick-scan",
    description="Updated probe selection.",
    system=AuditSystemData(parallel_attempts=8),
    run=AuditRunData(generations=5),
    plugins=AuditPluginsData(probe_spec="latentinjection,encoding.InjectROT13"),
    reporting=AuditReportData(report_prefix="quick-scan"),
)

Delete a Configuration

garak.configs.delete(workspace="default", name="quick-scan")