nat.data_models.finetuning

Attributes

logger
TrainerConfigT
TrajectoryBuilderConfigT
TrainerAdapterConfigT

Classes

RewardFunctionConfig	Configuration for the reward function
TrainerConfig	Base configuration for the Trainer
TrajectoryBuilderConfig	Configuration for the trajectory collector
TrainerAdapterConfig	Configuration for the trainer adapter
TrainingJobRef	A reference to a training job.
TrainingStatusEnum	str(object='') -> str
TrainingJobStatus	The status of a training job.
EpisodeItemRole	str(object='') -> str
EpisodeItem	A single step in an episode.
OpenAIMessage	A message in the OpenAI chat format.
DPOItem	A single step in an episode for DPO training.
Trajectory	A trajectory is a sequence of states, actions, and rewards.
TrajectoryCollection	A collection of trajectories.
CurriculumLearningConfig	Configuration for curriculum learning in fine-tuning.
FinetuneRunConfig	CLI Args for running finetuning and configuring
FinetuneConfig	Parameters used for a Trainer run

Module Contents

logger

class RewardFunctionConfig(/, **data: Any)

Bases: pydantic.BaseModel

Configuration for the reward function

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

name: str = None

class TrainerConfig(/, **data: Any)

Bases: nat.data_models.common.TypedBaseModel, nat.data_models.common.BaseModelRegistryTag

Base configuration for the Trainer

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

reward: RewardFunctionConfig | None = None

class TrajectoryBuilderConfig(/, **data: Any)

Bases: nat.data_models.common.TypedBaseModel, nat.data_models.common.BaseModelRegistryTag

Configuration for the trajectory collector

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

reward: RewardFunctionConfig | None = None

class TrainerAdapterConfig(/, **data: Any)

Bases: nat.data_models.common.TypedBaseModel, nat.data_models.common.BaseModelRegistryTag

Configuration for the trainer adapter

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

reward: RewardFunctionConfig | None = None

TrainerConfigT

TrajectoryBuilderConfigT

TrainerAdapterConfigT

class TrainingJobRef(/, **data: Any)

Bases: pydantic.BaseModel

A reference to a training job.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

run_id: str = None

backend: str = None

metadata: dict | None = None

class TrainingStatusEnum

Bases: str, enum.Enum

str(object=’’) -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.__str__() (if defined) or repr(object). encoding defaults to ‘utf-8’. errors defaults to ‘strict’.

Initialize self. See help(type(self)) for accurate signature.

PENDING = 'pending'

RUNNING = 'running'

COMPLETED = 'completed'

FAILED = 'failed'

CANCELED = 'canceled'

class TrainingJobStatus(/, **data: Any)

Bases: pydantic.BaseModel

The status of a training job.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

run_id: str = None

backend: str = None

status: TrainingStatusEnum = None

progress: float | None = None

message: str | None = None

metadata: dict | None = None

class EpisodeItemRole

Bases: str, enum.Enum

str(object=’’) -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.__str__() (if defined) or repr(object). encoding defaults to ‘utf-8’. errors defaults to ‘strict’.

Initialize self. See help(type(self)) for accurate signature.

USER = 'user'

ASSISTANT = 'assistant'

SYSTEM = 'system'

FUNCTION = 'function'

TOOL = 'tool'

ENVIRONMENT = 'environment'

OTHER = 'other'

class EpisodeItem(/, **data: Any)

Bases: pydantic.BaseModel

A single step in an episode.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

role: EpisodeItemRole = None

content: str = None

logprobs: Any | None = None

metadata: dict | None = None

check_logprobs() → EpisodeItem

class OpenAIMessage(/, **data: Any)

Bases: pydantic.BaseModel

A message in the OpenAI chat format.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

role: str = None

content: str = None

class DPOItem(/, **data: Any)

Bases: pydantic.BaseModel

A single step in an episode for DPO training.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

prompt: list[OpenAIMessage] | str = None

chosen_response: str = None

rejected_response: str = None

class Trajectory(/, **data: Any)

Bases: pydantic.BaseModel

A trajectory is a sequence of states, actions, and rewards.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

episode: list[EpisodeItem] | list[DPOItem] = None

reward: float = None

shaped_rewards: list[float] | None = None

metadata: dict | None = None

class TrajectoryCollection(/, **data: Any)

Bases: pydantic.BaseModel

A collection of trajectories.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

trajectories: list[list[Trajectory]] = None

run_id: str = None

class CurriculumLearningConfig(/, **data: Any)

Bases: pydantic.BaseModel

Configuration for curriculum learning in fine-tuning.

Curriculum learning progressively introduces harder training examples to improve model learning and convergence.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

enabled: bool = None

initial_percentile: float = None

increment_percentile: float = None

expansion_interval: int = None

min_reward_diff: float = None

sort_ascending: bool = None

random_subsample: float | None = None

validate_percentiles() → CurriculumLearningConfig

Validate that percentile values are in valid range.

class FinetuneRunConfig(/, **data: Any)

Bases: pydantic.BaseModel

CLI Args for running finetuning and configuring

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

config_file: pathlib.Path | pydantic.BaseModel = None

dataset: str | pathlib.Path | None = None

result_json_path: str = '$'

endpoint: str | None = None

endpoint_timeout: int = 300

override: tuple[tuple[str, str], Ellipsis] = ()

validation_dataset: str | pathlib.Path | None = None

validation_interval: int = None

validation_config_file: str | pathlib.Path | None = None

class FinetuneConfig(/, **data: Any)

Bases: pydantic.BaseModel

Parameters used for a Trainer run

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

enabled: bool = None

trainer: str | None = None

trajectory_builder: str | None = None

trainer_adapter: str | None = None

reward_function: RewardFunctionConfig | None = None

target_functions: list[str] = ['<workflow>']

target_model: str | None = None

curriculum_learning: CurriculumLearningConfig = None

num_epochs: int = None

output_dir: pathlib.Path = None

run_configuration: FinetuneRunConfig | None = None

validate_finetuning_enabled(values: dict[str, Any]) → dict[str, Any]