nemo_microservices.types.customization_config_param#

Module Contents#

Classes#

CustomizationConfigParam

Data#

Target

API#

class nemo_microservices.types.customization_config_param.CustomizationConfigParam#

Bases: typing_extensions.TypedDict

chat_prompt_template: str#

None

Chat Prompt Template to apply to the model to make it compatible with chat datasets, or to train it on a different template for your use case.

This parameter is only used for the "SFT" and "Distillation" Training Types on non embedding models.
custom_fields: Dict[str, object]#

None

A set of custom fields that the user can define and use for various purposes.

dataset_schemas: Iterable[Dict[str, object]]#

None

JSON Schema used for validating datasets that can be used with the configured finetuning jobs.

description: str#

None

The description of the entity.

max_seq_length: typing_extensions.Required[int]#

None

The largest context used for training.

Datasets are truncated based on the maximum sequence length.

name: str#

None

The name of the entity.

Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.

namespace: str#

None

The namespace of the entity.

This can be missing for namespace entities or in deployments that don’t use namespaces.

ownership: nemo_microservices.types.shared_params.ownership.Ownership#

None

Information about ownership of an entity.

If the entity is a namespace, the access_policies will typically apply to all entities inside the namespace.

pod_spec: nemo_microservices.types.training_pod_spec_param.TrainingPodSpecParam#

None

Additional parameters to ensure these training jobs get run on the appropriate hardware.

project: str#

None

The URN of the project associated with this entity.

prompt_template: str#

None

Prompt template used to extract keys from the dataset. E.g.

prompt_template=’{input} {output}’, and sample looks like ‘{“input”: “Q: 2x2 A:”, “output”: “4”}’ then the model sees ‘Q: 2x2 A: 4’.

This parameter is only used for the "SFT" and "Distillation" Training Types on non embeddding models.
target: nemo_microservices.types.customization_config_param.Target#

None

The target to perform the customization on

training_options: typing_extensions.Required[Iterable[nemo_microservices.types.customization_training_option_param.CustomizationTrainingOptionParam]]#

None

Resource configuration for each training option for the model.

training_precision: nemo_microservices.types.shared.model_precision.ModelPrecision#

None

Type of model precision.

Values

  • "int8" - 8-bit integer precision

  • "bf16" - Brain floating point precision

  • "fp16" - 16-bit floating point precision

  • "fp32" - 32-bit floating point precision

  • "fp8-mixed" - Mixed 8-bit floating point precision available on Hopper and later architectures.

  • "bf16-mixed" - Mixed Brain floating point precision

nemo_microservices.types.customization_config_param.Target: typing_extensions.TypeAlias#

None