Configure Models#

Model configuration is managed through two main sections in the configuration YAML:

  • customizationTargets.targets: Defines the available models, their metadata, and resource requirements.

  • customizationConfigTemplates.templates: Defines training templates for each model, including training methods, resource allocations, and Kubernetes pod customizations.

Before You Start#

Note

The previous customizerConfig.models structure is deprecated as of v25.06.

Targets#

Each model must be defined as a target under customizationTargets.targets. This section specifies the model’s name, namespace, URI, path, number of parameters, and whether it is enabled.

Configuration Parameters#

Model Target Configuration Parameters#

Parameter

Description

Example

name

The unique name and version of the model target.

<model_name>@<version>

namespace

The namespace or organization for the model.

meta, nvidia

enabled

Set to true to make the model available for training.

true

model_uri

The URI to download the model. Refer to the Model Catalog for a list of options.

ngc://nvidia/nemo/<model_name>:2.0

model_path

The local path or cache location for the model files.

llama-3_1-8b-instruct_0_0_1

num_parameters

The number of parameters in the model (integer).

8000000000

precision

The numerical precision for the model.

bf16-mixed

base_model

(Optional) The base model name if this is a derivative or fine-tuned model.

nvidia/<model_name>

Example:

customizationTargets:
  targets:
    meta/llama-3.2-1b-instruct@2.0:
      name: meta/llama-3.2-1b-instruct@2.0
      namespace: meta
      enabled: true
      model_uri: ngc://nvidia/nemo/llama-3_2-1b-instruct-nemo:2.0
      model_path: llama-3_2-1b-instruct_0_0_1
      num_parameters: 8000000000
      precision: bf16-mixed

  • Set enabled: true to make the model available for training.

  • The model_uri and model_path must match the source and cache location for the model.

Templates#

Training templates are defined under customizationConfigTemplates.templates. Each template references a model target and specifies one or more training options, resource allocations, and optional Kubernetes pod customizations.

Configuration Parameters#

Model Template Configuration Parameters#

Parameter

Description

Example

training_type

The training objective.

sft, distillation

finetuning_type

The fine-tuning method.

lora, all_weights

num_gpus, num_nodes, tensor_parallel_size, micro_batch_size

Resource allocations for the training job.

num_gpus: 8, num_nodes: 1, tensor_parallel_size: 4, micro_batch_size: 1

pod_spec

(Optional) Add Kubernetes tolerations, node selectors, or annotations for advanced scheduling.

tolerations: ..., node_selectors: ..., annotations: ...

prompt_template

(Optional) Template for prompt formatting.

{prompt} {completion}

max_seq_length

(Optional) Maximum sequence length for training.

4096

Example:

customizationConfigTemplates:
  templates:
    meta/llama-3.2-1b-instruct@2.0+A100:
      name: llama-3.2-1b-instruct@2.0+A100
      namespace: meta
      target: meta/llama-3.2-1b-instruct@2.0
      training_options:
        - training_type: sft
          finetuning_type: lora
          num_gpus: 1
          micro_batch_size: 1
      max_seq_length: 4096
      pod_spec:
        tolerations:
          - key: app
            operator: Equal
            value: customizer
            effect: NoSchedule
        nodeSelectors:
          kubernetes.io/hostname: minikube
        annotations:
          nmp/job-type: customization
      prompt_template: "{prompt} {completion}"

Overriding Existing Targets and Templates#

By default, Customizer overrides any existing model targets and templates in the database with those defined in your configuration. If you want to preserve existing records and only add new ones, set the following flags to false in your values file:

customizationTargets:
  overrideExistingTargets: false

customizationConfigTemplates:
  overrideExistingTemplates: false

  • overrideExistingTargets: Set to false to prevent overwriting existing model targets in the database. Only new targets will be added.

  • overrideExistingTemplates: Set to false to prevent overwriting existing training templates in the database. Only new templates will be added.

If these flags are set to true (the default), any targets or templates with matching names will be replaced by your configuration on application start.

Enable and Configure a Model#

To enable a model and configure its training options:

  1. Set enabled: true for the model in customizationTargets.targets.

  2. Add or update a template in customizationConfigTemplates.templates referencing the model target and specifying the desired training options.

Minimal Example:

customizationTargets:
  targets:
    meta/llama-3.2-1b-instruct@2.0:
      name: llama-3.2-1b-instruct@2.0
      namespace: meta
      enabled: true
      model_uri: ngc://nvidia/nemo/llama-3_2-1b-instruct-nemo:2.0
      model_path: llama-3_2-1b-instruct_0_0_1
      num_parameters: 8000000000
      precision: bf16-mixed

customizationConfigTemplates:
  templates:
    meta/llama-3.2-1b-instruct@2.0+A100:
      name: meta/llama-3.2-1b-instruct@2.0+A100
      namespace: meta
      target: meta/llama-3.2-1b-instruct@2.0
      training_options:
        - training_type: sft
          finetuning_type: lora
          num_gpus: 1
          micro_batch_size: 1