> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo-helix/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo-helix/_mcp/server. # Customization Job Reference Use this page when you need field-level details for customization job specifications, the complete API schema, or integration options. For concepts, see [Customization Job overview](/documentation/customizer-reference/manage-customization-jobs). ## Key Fields A customization job is submitted with a backend-specific spec (`AutomodelJobInput` for the automodel backend, `UnslothJobInput` for the unsloth backend). Both specs share the same top-level envelope, but several field names differ between backends. The table below lists the common envelope; for the full per-backend field list, see [Training Configuration](/documentation/customizer-reference/manage-customization-jobs/training-configuration). | Field | Required | Description | | ------------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `name` | No | Name for this customization job. Auto-generated if not provided | | `workspace` | Yes | Workspace where the job runs. Determines which datasets and models are authorized for the job. Passed to `jobs.create()`, not part of the spec | | `spec.model` | Yes | Base Model Entity to fine-tune. Automodel takes a string (`workspace/name`); Unsloth takes an object (`{"name": "workspace/name", ...}`) | | `spec.dataset` | Yes | Training data filesets. Automodel uses `{"training": "...", "validation": "..."}`; Unsloth uses `{"path": "..."}` | | `spec.training` | Yes | Training method and hyperparameters (see [Training Configuration](/documentation/customizer-reference/manage-customization-jobs/training-configuration)) | | `spec.training.training_type` | No | Training method. Automodel: `sft` or `distillation`. Unsloth: `sft`. Defaults to `sft` | | `spec.training.finetuning_type` | No | Adapter regime. Automodel: `lora`, `lora_merged`, or `all_weights`. Unsloth: `lora` or `all_weights`. Defaults to `lora` | | `spec.training.lora` | No | LoRA configuration (`{"rank": 16, "alpha": 32, ...}`). Auto-filled with defaults for LoRA regimes | | `spec.output` | No | Output artifact configuration (`{"name": "..."}`). Auto-generated if not provided | | `spec.integrations` | No | Optional W\&B / MLflow tracking configuration (see below) | | `spec.deployment_config` | No | Auto-deploy the trained model. Pass a string to reference an existing config by name, or an object with inline NIM deployment parameters (e.g., `{"lora_enabled": true}`). Omit to skip deployment. A referenced config that names no model (neither `model_entity_id` nor `model_spec.model_name`) is an unbound template: its engine and executor settings are copied onto a config for the trained model at deploy time, so it works for full-weight training, whose output model does not exist at submit time. For a LoRA job, the deployment is created against the **base** model with `lora_enabled: true`, and the adapter is served from it. The RL backend accepts the same field — see [GRPO and Reward Environments](/documentation/customizer-reference/grpo-and-reward-environments) | ### Deployment config validation All three backends (automodel, unsloth, RL) resolve a referenced `deployment_config` at submit time and reject it before any GPU time is spent. A job fails to submit when: * The string reference does not resolve to a deployment config. * The job trains a **standalone LoRA adapter** and the referenced config has `lora_enabled: false` — the deployment could never load the adapter. * The job trains a standalone LoRA adapter and the referenced config names a model other than the job's base model. The adapter is served from its base model's deployment, so the config has to target that base. * The job registers a **new model entity** (`all_weights`, or a LoRA job with a merged save method) and the referenced config names a different model. The output entity does **not** have to exist yet — a config may point forward at the model the run will produce, which is what Studio creates up front. Retraining is fine for the same reason. * The config sets `tool_call_config.tool_call_plugin` and the caller lacks the `models.tool-call-plugin.set` permission. A merged save (`lora_merged`, or unsloth's `merged_16bit` / `merged_4bit`) folds the adapter into the base weights and produces a standalone model, so the LoRA-specific rules above do not apply to it. Two config shapes are therefore valid ahead of a model that does not exist yet: * A **forward reference** — the config names the model the run will produce. Creating it up front means a bad engine or image is rejected in milliseconds at submit instead of after a training run, and the deployment can start the moment training ends. One config per run. * An **unbound template** — a config with neither `model_entity_id` nor `model_spec.model_name`. It carries only engine and executor settings, is accepted by any job, and is bound to the trained model at deploy time. Use this to share settings across runs. --- ## Complete API Reference For generated REST API details, see the [Customizer API Reference](/documentation/reference/api-reference) and search for `AutomodelJobInput` or `UnslothJobInput`. --- ## Weights & Biases Integration Both backends accept the same `integrations` object on the job spec. Add a `wandb` block to request W\&B tracking; the training runtime activates it when the required credentials are available. ```python import os from nemo_helix import NeMoHelix from nemo_automodel_plugin.schema import AutomodelJobInput client = NeMoHelix(base_url=os.environ.get("NHX_BASE_URL", "http://localhost:8080"), workspace="default") spec = AutomodelJobInput( model="default/qwen3-1.7b", dataset={"training": "default/my-dataset"}, training={"training_type": "sft", "finetuning_type": "lora"}, schedule={"epochs": 3}, integrations={ "wandb": { "project": "my-finetuning-project", "entity": "my-team", "tags": ["fine-tuning", "qwen"], "api_key_secret": "my-wandb-key", } }, ) job = client.customization.automodel.jobs.create(spec=spec, workspace="default", name="my-job") ``` The `api_key_secret` field references a stored secret containing your `WANDB_API_KEY`. Use the secret name (e.g., `"my-wandb-key"`) to resolve it from the request workspace. To create the secret, see [Weights & Biases Keys](/documentation/get-started/core-concepts/manage-secrets). | Field | Description | | ---------------- | --------------------------------------------------------------------------------------------------- | | `project` | W\&B project name. Defaults to `output.name` if not set | | `entity` | W\&B entity (team or username) | | `tags` | List of tags for filtering runs | | `api_key_secret` | Reference to a secret containing `WANDB_API_KEY` | | `name` | W\&B run name. Defaults to the job ID if not provided | | `notes` | Notes or description for the run | | `base_url` | Base URL for self-hosted W\&B servers (e.g., `https://wandb.mycompany.com`). Omit to use W\&B cloud | To view your training metrics in W\&B after the job starts, see [ft-tut-metrics-wandb](/documentation/customizer-reference/tutorials/metrics). --- ## MLflow Integration Add an `mlflow` block to the same `integrations` object to request MLflow tracking: ```python spec = AutomodelJobInput( model="default/qwen3-1.7b", dataset={"training": "default/my-dataset"}, training={"training_type": "sft", "finetuning_type": "lora"}, schedule={"epochs": 3}, integrations={ "mlflow": { "experiment_name": "qwen-finetuning", "tracking_uri": "http://mlflow.example.com:5000", } }, ) job = client.customization.automodel.jobs.create(spec=spec, workspace="default", name="my-job") ``` | Field | Description | | ----------------- | --------------------------------------------------------------------------- | | `experiment_name` | MLflow experiment name. Defaults to `output.name` if not set | | `tracking_uri` | The MLflow tracking server URI. Can also be set via `MLFLOW_TRACKING_URI` | | `name` | MLflow run name. Defaults to the job ID if not provided | | `tags` | Key-value pairs for filtering runs (e.g., `{"team": "nlp", "task": "sft"}`) | | `description` | Description for the MLflow run | ## Next Steps * [Create a customization job](/documentation/customizer-reference/manage-customization-jobs/create-a-customization-job): Start a job with a model, dataset, training configuration, and optional integrations. * [Monitor training metrics](/documentation/customizer-reference/tutorials/metrics): View logs and metrics through MLflow or W\&B. * [Manage secrets](/documentation/get-started/core-concepts/manage-secrets): Store credentials such as W\&B API keys and provider tokens. * [Troubleshooting MLflow integrations](/documentation/reference/troubleshooting/customizer): Diagnose failed or misconfigured customization jobs.