Run an Anonymizer Job

This tutorial walks through the anonymizer.run job: defining a run spec, submitting it to NeMo Helix Jobs worker, and loading the parquet artifacts it produces.

For detection, rewrite, and replacement strategy details, see the open-source library documentation.

Prerequisites

Complete the tutorials prerequisites, which cover:

• A running NeMo Helix cluster with the nemo anonymizer CLI available (see Setup).
• An inference provider configured (default examples use nvidia-build).
• A fileset named anonymizer-inputs containing anonymizer-input.csv in your current workspace. Upload a CSV with id and biography columns before running this notebook; see Upload an Input Fileset.

What run Does

anonymizer.run executes the full Anonymizer pipeline on every record of an input file and writes the output as job artifacts.

The CLI exposes one run command:

Job artifacts (under the artifacts/ directory):

Step 1: Build an AnonymizerRequest

Set NVIDIA_MODEL and NHX_ANON_GENERAL_MODEL to override the thinking and general-purpose model IDs below. Use exact IDs from the chosen provider’s catalog, with credentials authorized for both models; the role aliases stay unchanged.

AnonymizerRequest contains the execution fields shared by preview and run (config, data, model_configs, and selected_models). A run processes the full input file, so it does not include num_records:

import os
from anonymizer.config.anonymizer_config import AnonymizerConfig
from anonymizer.config.replace_strategies import Redact
from data_designer.config import ModelConfig
from nemo_anonymizer_plugin.app.input import AnonymizerInputSpec
from nemo_anonymizer_plugin.app.task_config import AnonymizerRequest
from nemo_anonymizer_plugin.sdk.resources import AnonymizerResource
from nemo_helix_plugin.client.client import NemoClient
client = NemoClient.from_config()
anonymizer = AnonymizerResource(client)
WORKSPACE = client.resolve_workspace()
MODEL_PROVIDER = os.environ.get("NHX_ANON_PROVIDER", "nvidia-build")
GENERAL_MODEL = os.environ.get("NHX_ANON_GENERAL_MODEL", "openai/gpt-oss-20b")
config = AnonymizerConfig(
    replace=Redact(format_template="[REDACTED_{label}]"),
)
model_configs = [
    # The Anonymizer plugin manages a default PII entity detector, so it is not
    # configured here (you may override it with a ModelConfig, but that is
    # optional). These entries cover the remaining LLM roles.
    # Keep the library's role alias; the retired 120B endpoint is not used.
    ModelConfig(alias="gpt-oss-120b", provider=MODEL_PROVIDER, model=GENERAL_MODEL),
    ModelConfig(alias="nemotron-30b-thinking", provider=MODEL_PROVIDER, model=os.environ.get("NVIDIA_MODEL", "nvidia/nemotron-3.5-lightning-30b-a3b")),
]
request = AnonymizerRequest(
    config=config,
    data=AnonymizerInputSpec(
        source=f"fileset://{WORKSPACE}/anonymizer-inputs#anonymizer-input.csv",
        text_column="biography",
        id_column="id",
    ),
    model_configs=model_configs,
)

Step 2: Write the Spec to YAML

The CLI run command reads a YAML spec file. Serialize the AnonymizerRequest directly:

import yaml
from pathlib import Path
spec_path = Path("/tmp/anonymizer-run.yaml")
spec_path.write_text(yaml.safe_dump(request.model_dump(mode="json", exclude_none=True)))

Step 3: Run the Job

Submit the spec to NeMo Helix Jobs worker:

nemo anonymizer run \
  --spec-file /tmp/anonymizer-run.yaml

The command prints the assigned job name. You need that name to poll status and download artifacts in Step 4.

The SDK equivalent is anonymizer.run(request). It posts the request to the plugin’s /jobs/run endpoint and returns an AnonymizerJobResource:

job = anonymizer.run(request)

The run path rejects local file paths in data.source — use a fileset reference (<fileset>#<path>) or http(s) URL. It also requires explicit model_configs referencing Inference Gateway providers.

Step 4: Get Results

For the SDK job created in Step 3, wait for completion and download its artifacts before loading them. The job is ready for artifact download when its status is completed:

from pathlib import Path
from tempfile import mkdtemp
output_dir = Path(mkdtemp(prefix="anonymizer-artifacts-"))
job.wait_until_done()
job.download_artifacts(path=output_dir)

get_job_status() reads the current status, check_if_complete() tests whether artifacts are ready, wait_until_done() blocks until a terminal state, and download_artifacts() downloads and extracts the result.

CLI equivalent

If you submitted the job with the CLI instead, track the job using the name it printed:

# Replace with the job name printed by `nemo anonymizer run`.
nemo jobs get-status <job-name>
nemo jobs get-logs <job-name>

To download from the CLI, fetch the artifacts result and extract it:

nemo jobs results download artifacts \
  --job <job-name> \
  --output-file /tmp/anonymizer-artifacts.tar.gz
mkdir -p /tmp/anonymizer-artifacts
tar -xzf /tmp/anonymizer-artifacts.tar.gz -C /tmp/anonymizer-artifacts
ls /tmp/anonymizer-artifacts/artifacts

Load the Downloaded Results

Both download methods produce an artifacts directory under the extraction root. The SDK flow above sets output_dir to a fresh directory. If you used only the CLI, set output_dir to the directory you extracted into, as shown in the comment below, before running this block:

from pathlib import Path
from nemo_anonymizer_plugin.sdk.job_results import AnonymizerJobResults
# CLI-only, after the shell download and extraction above:
# output_dir = Path("/tmp/anonymizer-artifacts")
results = AnonymizerJobResults(output_dir / "artifacts")
dataset = results.load_dataset()
trace   = results.load_trace()
failed  = results.load_failed_records()

AnonymizerJobResults exposes load_dataset(), load_trace(), load_failed_records(), and display_record() over the same underlying files. See SDK Resources.

How the Job Compiles

For each request, the plugin:

1. Validates the Anonymizer library AnonymizerConfig.
2. Validates the input source (rejects local paths; checks fileset refs).
3. Validates that selected_models overrides also have model_configs.
4. Resolves model_configs providers through the Inference Gateway.
5. Renders a unified model_configs YAML body for the library.
6. Stores the resolved providers and YAML in the internal AnonymizerStepConfig consumed by the Jobs worker.

Provider endpoints are re-resolved at runtime so the job uses the in-cluster Inference Gateway address rather than the address captured at submission time.

Next Steps

• Iterate faster with preview before scaling to a full job.
• Refer to SDK Resources for AnonymizerJobResource and AnonymizerJobResults details.
• Replacement strategy parameters and rewrite mode are documented in the library docs.