nemo_automodel.components.datasets.llm.xlam

Module Contents

Classes

Name	Description
`XlamConfig`	Construction-time configuration for the xLAM function-calling dataset (tokenizer is a build arg).

Functions

Name	Description
`_convert_tool_calls`	Convert xLAM answers field into OpenAI tool_calls messages.
`_convert_tools`	Convert xLAM tool definitions into OpenAI tool schema.
`_format_example`	-
`_json_load_if_str`	-
`make_xlam_dataset`	Load and preprocess the xLAM function-calling dataset to OpenAI messages

Data

_TYPE_MAP

logger

API

class nemo_automodel.components.datasets.llm.xlam.XlamConfig(
    seq_length: int | None = None,
    limit_dataset_samples: int | None = None,
    fp8: bool = False,
    split: str = 'train',
    dataset_name: str = 'Salesforce/xlam-function-c...,
    padding: bool | str = False,
    truncation: bool | str = False
)

Dataclass

Construction-time configuration for the xLAM function-calling dataset (tokenizer is a build arg).

accepts_tokenizer

bool = True

dataset_name

str = 'Salesforce/xlam-function-calling-60k'

Identifier for the HuggingFace dataset to load.

fp8

bool = False

Flag reserved for future mixed-precision use (currently unused).

limit_dataset_samples

int | None = None

If set, limit the number of examples loaded from the split.

padding

bool | str = False

Optional padding strategy.

seq_length

int | None = None

If set, pad/truncate each example to this length.

split

str = 'train'

Which split of the dataset to load (e.g. train, validation).

truncation

bool | str = False

Optional truncation strategy.

nemo_automodel.components.datasets.llm.xlam.XlamConfig.build(
    tokenizer: 'PreTrainedTokenizerBase | None'
) -> nemo_automodel.components.datasets.lazy_mapped_dataset.LazyMappedDataset

Build the xLAM :class:LazyMappedDataset from this :class:XlamConfig and a runtime tokenizer.

nemo_automodel.components.datasets.llm.xlam._convert_tool_calls(
    raw_calls: typing.List[typing.Dict],
    example_id: typing.Optional[int] = None
) -> typing.List[typing.Dict]

Convert xLAM answers field into OpenAI tool_calls messages.

nemo_automodel.components.datasets.llm.xlam._convert_tools(
    raw_tools: typing.List[typing.Dict]
) -> typing.List[typing.Dict]

Convert xLAM tool definitions into OpenAI tool schema.

nemo_automodel.components.datasets.llm.xlam._format_example(
    example,
    tokenizer,
    eos_token_id,
    pad_token_id,
    seq_length = None,
    padding = None,
    truncation = None
)

nemo_automodel.components.datasets.llm.xlam._json_load_if_str(
    value
)

nemo_automodel.components.datasets.llm.xlam.make_xlam_dataset(
    tokenizer,
    seq_length = None,
    limit_dataset_samples = None,
    fp8 = False,
    split = 'train',
    dataset_name = 'Salesforce/xlam-function-c...,
    padding = False,
    truncation = False
)

Load and preprocess the xLAM function-calling dataset to OpenAI messages compatible with the bulbasaur chat template (tool-calling aware).

nemo_automodel.components.datasets.llm.xlam._TYPE_MAP = {'str': 'string', 'string': 'string', 'int': 'integer', 'integer': 'integer', 'f...

nemo_automodel.components.datasets.llm.xlam.logger = logging.getLogger(__name__)