nemo_microservices.types.customization.config_update_params#

Module Contents#

Classes#

ConfigUpdateParams

API#

class nemo_microservices.types.customization.config_update_params.ConfigUpdateParams#

Bases: typing_extensions.TypedDict

add_training_options: Iterable[nemo_microservices.types.customization_training_option_param.CustomizationTrainingOptionParam]#

None

List of training options to add in the existing training options for the config.

chat_prompt_template: str#

None

Chat Prompt Template to apply to the model to make it compatible with chat datasets, or to train it on a different template for your use case.

This parameter is only used for the "SFT" and "Distillation" Training Types on non embedding models.
custom_fields: Dict[str, object]#

None

A set of custom fields that the user can define and use for various purposes.

dataset_schemas: Iterable[Dict[str, object]]#

None

JSON Schema used for validating datasets that can be used with the configured finetuning jobs.

description: str#

None

The description of the entity.

max_seq_length: int#

None

The largest context used for training.

Datasets are truncated based on the maximum sequence length.

namespace: typing_extensions.Required[str]#

None

ownership: nemo_microservices.types.shared_params.ownership.Ownership#

None

Information about ownership of an entity.

If the entity is a namespace, the access_policies will typically apply to all entities inside the namespace.

pod_spec: nemo_microservices.types.training_pod_spec_param.TrainingPodSpecParam#

None

Additional parameters to ensure these training jobs get run on the appropriate hardware.

project: str#

None

The URN of the project associated with this entity.

prompt_template: str#

None

Prompt template used to extract keys from the dataset. E.g.

prompt_template=’{input} {output}’, and sample looks like ‘{“input”: “Q: 2x2 A:”, “output”: “4”}’ then the model sees ‘Q: 2x2 A: 4’.

This parameter is only used for the "SFT" and "Distillation" Training Types on non embeddding models.
remove_training_options: Iterable[nemo_microservices.types.customization.customization_training_option_removal_param.CustomizationTrainingOptionRemovalParam]#

None

List of training options to remove from the existing training options for the config.

training_options: Iterable[nemo_microservices.types.customization_training_option_param.CustomizationTrainingOptionParam]#

None

Resource configuration for each training option for the model.

training_precision: nemo_microservices.types.shared.model_precision.ModelPrecision#

None

Type of model precision.

Values

  • "int8" - 8-bit integer precision

  • "bf16" - Brain floating point precision

  • "fp16" - 16-bit floating point precision

  • "fp32" - 32-bit floating point precision

  • "fp8-mixed" - Mixed 8-bit floating point precision available on Hopper and later architectures.

  • "bf16-mixed" - Mixed Brain floating point precision