List Customization Configs#

Get a list of available customization configurations and their details to determine which models are available for fine-tuning.

Tip

These configs are typically added by your cluster administrator during the initial setup of NeMo Customizer.

Prerequisites#

Before you can get a list of customization configurations, make sure that you have:

  • Access to the NeMo Customizer service

  • Set the CUSTOMIZER_BASE_URL environment variable to your NeMo Customizer service endpoint

export CUSTOMIZER_BASE_URL="https://your-customizer-service-url"

To List Customization Configs#

Choose one of the following options to get a list of available customization configurations.

import os
from nemo_microservices import NeMoMicroservices

# Initialize the client
client = NeMoMicroservices(
    base_url=os.environ['CUSTOMIZER_BASE_URL']
)

# List customization configs with filters
configs = client.customization.configs.list(
    page=1,
    page_size=10,
    sort="-created_at",
    filter={
        "target_base_model": "meta/llama-3.1-8b-instruct",
        "training_type": "sft",
        "finetuning_type": "lora",
        "enabled": True
    }
)

print(f"Found {len(configs.data)} configs")
for config in configs.data:
    print(f"Config: {config.name} - {config.description}")

BASE_MODEL="meta/llama-3.1-8b-instruct"
TRAINING_TYPE="sft"
FINETUNING_TYPE="lora"

curl \
  "${CUSTOMIZER_BASE_URL}/customization/configs" \
  --data-urlencode "page=1" \
  --data-urlencode "page_size=10" \
  --data-urlencode "sort=-created_at" \
  --data-urlencode "filter[target_base_model]=${BASE_MODEL}" \
  --data-urlencode "filter[training_type]=${TRAINING_TYPE}" \
  --data-urlencode "filter[finetuning_type]=${FINETUNING_TYPE}" \
  --data-urlencode "filter[enabled]=true" | jq

You can use either use the full {namespace}/{name} fields of a returned config object, or just use name (which would use the default namespace), and set the config parameter when creating a customization job.

Example Response
{
    "object": "list",
    "data": [
        {
            "created_at": "2024-11-26T02:58:55.339737",
            "updated_at": "2024-11-26T02:58:55.339737",
            "id": "customization_config-MedVscVbr4pgLhLgKTLbv9",
            "name": "llama-3.1-8b-instruct@v1.0.0+A100",
            "namespace": "default",
            "description": "Configuration for training LLama 3.1 8B on A100 GPUs",
            "target": {
                "id": "customization_target-A5bK7mNpR8qE9sL2fG3hJ6",
                "name": "meta/llama-3.1-8b-instruct@2.0",
                "namespace": "default",
                "base_model": "meta/llama-3.1-8b-instruct",
                "enabled": true,
                "num_parameters": 8000000000,
                "precision": "bf16",
                "status": "ready"
            },
            "training_options": [
                {
                    "training_type": "sft",
                    "finetuning_type": "lora",
                    "num_gpus": 2,
                    "num_nodes": 1,
                    "tensor_parallel_size": 1,
                    "pipeline_parallel_size": 1,
                    "micro_batch_size": 1,
                    "use_sequence_parallel": false
                }
            ],
            "training_precision": "bf16",
            "max_seq_length": 2048,
            "pod_spec": {
                "node_selectors": {
                    "nvidia.com/gpu.product": "NVIDIA-A100-SXM4-80GB"
                },
                "annotations": {
                    "nmp/job-type": "customization"
                },
                "tolerations": [
                    {
                        "key": "app",
                        "operator": "Equal",
                        "value": "customizer",
                        "effect": "NoSchedule"
                    }
                ]
            },
            "prompt_template": "{input} {output}",
            "chat_prompt_template": null,
            "dataset_schemas": [],
            "project": null,
            "ownership": {}
        },
    ],
    "pagination": {
        "page": 1,
        "page_size": 10,
        "current_page_size": 2,
        "total_pages": 1,
        "total_results": 2
    },
    "sort": "-created_at"
}

Tip

The num_gpus multiplied by the num_nodes is the total number of GPUs required to run a fine tuning.