ConfigCreateParams#

class nemo_microservices.types.customization.ConfigCreateParams

Bases: TypedDict

max_seq_length: Required[int]

The largest context used for training.

Datasets are truncated based on the maximum sequence length.

training_options: Required[Iterable[CustomizationTrainingOptionParam]]

Resource configuration for each training option for the model.

chat_prompt_template: str

Chat Prompt Template to apply to the model to make it compatible with chat datasets, or to train it on a different template for your use case.

This parameter is only used for the “SFT” and “Distillation” Training Types on non embedding models.

custom_fields: Dict[str, str]

A set of custom fields that the user can define and use for various purposes.

dataset_schemas: Iterable[object]

JSON Schema used for validating datasets that can be used with the configured finetuning jobs.

description: str

The description of the entity.

name: str

The name of the entity.

Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.

namespace: str

The namespace of the entity.

This can be missing for namespace entities or in deployments that don’t use namespaces.

ownership: Ownership

Information about ownership of an entity.

If the entity is a namespace, the access_policies will typically apply to all entities inside the namespace.

pod_spec: TrainingPodSpecParam

Additional parameters to ensure these training jobs get run on the appropriate hardware.

project: str

The URN of the project associated with this entity.

prompt_template: str

Prompt template used to extract keys from the dataset. E.g.

prompt_template=’{input} {output}’, and sample looks like ‘{“input”: “Q: 2x2 A:”, “output”: “4”}’ then the model sees ‘Q: 2x2 A: 4’.

This parameter is only used for the “SFT” and “Distillation” Training Types on non embeddding models.

target: str | CustomizationTargetParam

The target to perform the customization on

training_precision: Literal['int8', 'bf16', 'fp16', 'fp32', 'fp8-mixed', 'bf16-mixed']

Type of model precision.

## Values

  • “int8” - 8-bit integer precision

  • “bf16” - Brain floating point precision

  • “fp16” - 16-bit floating point precision

  • “fp32” - 32-bit floating point precision

  • “fp8-mixed” - Mixed 8-bit floating point precision available on Hopper and later architectures.

  • “bf16-mixed” - Mixed Brain floating point precision