CustomizationConfig#

class nemo_microservices.types.CustomizationConfig(*args: Any, **kwargs: Any)

Bases: BaseModel

max_seq_length: int

The largest context used for training.

Datasets are truncated based on the maximum sequence length.

training_options: List[CustomizationTrainingOption]: Resource configuration for each training option for the model.

chat_prompt_template: str | None = None: Chat Prompt Template to apply to the model to make it compatible with chat datasets

created_at: datetime | None = None: Timestamp for when the entity was created.

dataset_schemas: List[object] | None = None: Descriptions of the expected formats of the datasets uploaded.

name: str | None = None

The name of the entity.

Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.

namespace: str | None = None

The namespace of the entity.

This can be missing for namespace entities or in deployments that don’t use namespaces.

ownership: Ownership | None = None

Information about ownership of an entity.

If the entity is a namespace, the access_policies will typically apply to all entities inside the namespace.

pod_spec: TrainingPodSpec | None = None: Additional parameters to ensure these training jobs get run on the appropriate hardware.

project: str | None = None: The URN of the project associated with this entity.

prompt_template: str | None = None

Prompt template used to extract keys from the dataset.

E.g. prompt_template=’{input} {output}’, and sample looks like ‘{“input”: “Q: 2x2 A:”, “output”: “4”}’ then the model sees ‘Q: 2x2 A: 4’

target: str | CustomizationTarget | None = None: The target to perform the customization on

training_precision: Literal['int8', 'bf16', 'fp16', 'fp32', 'fp8-mixed', 'bf16-mixed'] | None = None

Type of model precision.

## Values

“int8” - 8-bit integer precision
“bf16” - Brain floating point precision
“fp16” - 16-bit floating point precision
“fp32” - 32-bit floating point precision
“fp8-mixed” - Mixed 8-bit floating point precision available on Hopper and later architectures.
“bf16-mixed” - Mixed Brain floating point precision

updated_at: datetime | None = None: Timestamp for when the entity was last updated.