nemo_automodel.components.datasets.decision.records

SystemOne records: typed questions, option orders, and labelled decision examples.

A record is the Jev / SystemOne POST /v1/systemone request body plus supervision::

{ “id”: “soc-1a2b”, “state”: {…}, # any JSON value “questions”: {“verdict”: {“type”: “choice”, “instructions”: ”…”, “criteria”: {…}}, …}, “answers”: {“verdict”: “malicious”, …}, # gold option id per question (SFT) “rewards”: {“verdict”: {“malicious”: 1.0, …}}, # optional per-option rewards (RL) “meta”: {…}, # optional slicing metadata }

Question types follow the SystemOne API: noul (true/false), choice (named options) and score (ordered levels "0".."K-1"). Each model family enumerates options in its own order, so every tensor in this package is built from an explicit list of option ids rather than from dictionary order.

Module Contents

Classes

Name	Description
`QuestionTarget`	Supervision for one question, aligned with a model-specific option order.

Functions

Name	Description
`canonical_option_ids`	All option ids of a question (order-free set semantics; used for validation).
`load_records`	Read a JSONL split, requiring unique nonempty stringified record IDs.
`pad_and_shard_records`	Pad cyclically and return equal-length data-parallel record shards.
`question_targets`	Align a record’s answers and rewards with the option order a model uses.
`validate_record`	Raise `ValueError` if a record is not a well-formed SystemOne example.

Data

QUESTION_TYPES

QUESTION_TYPE_IDS

_Record

API

class nemo_automodel.components.datasets.decision.records.QuestionTarget(
    question_id: str,
    question_type: int,
    option_ids: tuple[str, ...],
    label: int,
    rewards: tuple[float, ...] | None
)

Dataclass

Supervision for one question, aligned with a model-specific option order.

label

int

option_ids

tuple[str, ...]

question_id

str

question_type

int

rewards

tuple[float, ...] | None

nemo_automodel.components.datasets.decision.records.canonical_option_ids(
    question: dict[str, typing.Any]
) -> list[str]

All option ids of a question (order-free set semantics; used for validation).

nemo_automodel.components.datasets.decision.records.load_records(
    path: str | pathlib.Path,
    limit: int | None = None
) -> list[dict[str, typing.Any]]

Read a JSONL split, requiring unique nonempty stringified record IDs.

Validate before distributed padding, which may intentionally repeat records. Numeric IDs are preserved but share an identity with their string equivalent.

Parameters:

path

str | Path

JSONL file containing one unpadded input split.

limit

int | NoneDefaults to None

Optional maximum number of records to read and validate.

Returns: list[dict[str, Any]]

Loaded records in file order, preserving their original ID values.

Raises:

ValueError: If a loaded record has a missing, empty, or duplicate ID.

nemo_automodel.components.datasets.decision.records.pad_and_shard_records(
    records: collections.abc.Sequence[nemo_automodel.components.datasets.decision.records._Record],
    rank: int,
    world_size: int,
    shuffle_seed: int | None = None
) -> list[nemo_automodel.components.datasets.decision.records._Record]

Pad cyclically and return equal-length data-parallel record shards.

Equal shard lengths keep collective-bearing forwards aligned across ranks. Padding duplicates existing records; callers that aggregate evaluation predictions must deduplicate by record ID. An empty input yields empty shards on every rank.

Parameters:

records

Sequence[_Record]

Ordered input records.

rank

int

Data-parallel rank in [0, world_size).

world_size

int

Positive data-parallel group size.

shuffle_seed

int | NoneDefaults to None

Deterministic shuffle seed, or None to preserve record order.

Returns: list[_Record]

This rank’s ceil(len(records) / world_size) records.

nemo_automodel.components.datasets.decision.records.question_targets(
    record: dict[str, typing.Any],
    option_orders: dict[str, typing.Iterable[str]]
) -> list[nemo_automodel.components.datasets.decision.records.QuestionTarget]

Align a record’s answers and rewards with the option order a model uses.

Parameters:

record

dict[str, Any]

SystemOne record.

option_orders

dict[str, Iterable[str]]

{question_id: option ids in model order}, in the order the model returns questions.

nemo_automodel.components.datasets.decision.records.validate_record(
    record: dict[str, typing.Any]
) -> None

Raise ValueError if a record is not a well-formed SystemOne example.

nemo_automodel.components.datasets.decision.records.QUESTION_TYPES = ('noul', 'choice', 'score')

nemo_automodel.components.datasets.decision.records.QUESTION_TYPE_IDS = {'noul': 0, 'choice': 1, 'score': 2}

nemo_automodel.components.datasets.decision.records._Record = TypeVar('_Record')