nemo_rl.algorithms.x_token.token_aligner#

Cross-tokenizer token alignment.

The student and teacher tokenize the same source text; this module pairs their tokens by the character spans each token covers (return_offsets_mapping=True on a fast HF tokenizer). Consecutive tokens that share an exact (char_start, char_end) collapse into a cluster, and a strict char-end walker pairs student and teacher clusters covering the same character range. Special tokens (BOS / EOS / pad, offset (0, 0)) are paired separately by role. The result feeds the cross-tokenizer distillation loss.

Public surface: - :class:AlignmentPair — per-pair record containing student/teacher token ranges and an is_correct flag filled by the aligner. - :class:AlignmentBatch — dense-padded per-batch alignment payload that covers all three loss modes (P-KL, gold_loss, xtoken_loss). - :class:TokenAligner — owns the two tokenizers and the projection matrix, exposes :meth:align and :meth:align_chat for the collator. - :func:align_by_offsets_cluster — the single-sample offset alignment kernel, also usable directly.

Module Contents#

Classes#

AlignmentPair

One aligned span between student and teacher token sequences.

NativeAlignmentPart

One source-validated answer piece on a model’s rendered chat surface.

NativeAlignmentRegions

Selected semantic character spans in one native assistant turn.

AlignmentBatch

Per-batch alignment payload covering all three loss modes.

TokenAligner

Aligns student and teacher tokenizations of the same source text.

Functions#

canonical_token

Return a canonical representation of a tokenizer token.

_canonicalize_sequence

Canonicalize every token in a sequence, including byte-merging.

_normalize_canonical_merge_offsets

Give every token in a canonical N-to-1 merge one offset envelope.

_merge_encoding_artifacts

Merge known multi-token mojibake patterns into single tokens.

_get_byte_value

Return the byte value (0..255) for a single character, or None.

_merge_consecutive_bytes

Merge consecutive byte-fallback tokens back into Unicode characters.

_try_merge_byte_buffer

Decode 2-4 buffered byte tokens as a single UTF-8 character.

_role_of

Classify a token id as a special-token role or "content".

_partition

Split a sequence into leading-specials / content / trailing-specials.

_pair_specials_by_role

Pair leading/trailing special tokens 1<->1 by matching role.

_cluster_same_span

Collapse consecutive tokens sharing the exact same (cs, ce) span.

_content_align_offset_cluster

Pair student/teacher clusters covering the same character range.

align_by_offsets_cluster

Cluster + strict char-end offset alignment for a single sample.

Data#

API#

nemo_rl.algorithms.x_token.token_aligner.VISUAL_BYTE_MAP#

None

nemo_rl.algorithms.x_token.token_aligner._MULTI_TOKEN_ARTIFACT_FIXES: list[tuple[list[str], list[str]]]#

[([‘ĠâĪ’, ‘ij’], [‘Ġ∑’]), ([‘âĪ’, ‘ij’], [’∑’]), ([‘ĠâĪ’, ‘ı’], [‘Ġ∏’]), ([‘âĪ’, ‘ı’], [’∏’]), ([‘ĠâĪ’…

nemo_rl.algorithms.x_token.token_aligner._MULTI_TOKEN_ARTIFACT_FIXES_BY_FIRST: dict[str, list[tuple[list[str], list[str]]]]#

None

nemo_rl.algorithms.x_token.token_aligner._UNICODE_FIXES#

None

nemo_rl.algorithms.x_token.token_aligner._SPECIAL_TOKEN_MAP#

None

class nemo_rl.algorithms.x_token.token_aligner.AlignmentPair#

One aligned span between student and teacher token sequences.

The alignment sets is_correct using decoded-text equality, NFC-validated native repair, or matching EOS roles. Insertions/deletions (orphan groups) use -1 for the empty side’s start/end indices.

.. attribute:: s_start

Inclusive start index into the student token sequence (-1 for teacher-only insertions).

.. attribute:: s_end

Exclusive end index into the student token sequence (-1 for teacher-only insertions).

.. attribute:: t_start

Inclusive start index into the teacher token sequence (-1 for student-only insertions).

.. attribute:: t_end

Exclusive end index into the teacher token sequence (-1 for student-only insertions).

.. attribute:: is_correct

True when the decoded student and teacher span texts are equal, when an NFC-validated native repair matches them, or when both spans are their tokenizer’s EOS token. Defaults to False until the aligner validates the pair.

s_start: int#

None

s_end: int#

None

t_start: int#

None

t_end: int#

None

is_correct: bool#

False

class nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentPart#

One source-validated answer piece on a model’s rendered chat surface.

Differently spelled Boolean/null values may retain CE without exact-text KD.

name: str#

None

span: tuple[int, int]#

None

text: str#

None

allow_native_difference: bool#

False

class nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentRegions#

Selected semantic character spans in one native assistant turn.

reasoning: tuple[int, int] | None#

None

close: tuple[int, int] | None#

None

answer: tuple[int, int] | None#

None

get(name: str) → tuple[int, int] | None#

Return the named semantic span, rejecting unknown region names.

class nemo_rl.algorithms.x_token.token_aligner.AlignmentBatch#

Per-batch alignment payload covering all three loss modes.

The collator hands this dataclass directly to the loss fn alongside the tokenized batch. Tensors are dense-padded to the batch maximum so DTensor V2 can shard on dim 0 without knowing about cross-tokenizer specifics.

.. attribute:: pair_valid

[B, max_pairs] bool. False on padding entries.

.. attribute:: pair_is_correct

[B, max_pairs] bool. True when decoded student and teacher span texts are equal, when an NFC-validated native repair matches them, or when both spans are their tokenizer’s EOS token.

.. attribute:: student_chunk_id

[B, T_s] long. Chunk index (= pair index) the student token belongs to; -1 if not in any chunk (insertion-only pair on student side).

.. attribute:: teacher_chunk_id

[B, T_t] long. Counterpart.

pair_valid: torch.Tensor#

None

pair_is_correct: torch.Tensor#

None

student_chunk_id: torch.Tensor#

None

teacher_chunk_id: torch.Tensor#

None

class nemo_rl.algorithms.x_token.token_aligner.TokenAligner(
student_tokenizer,
teacher_tokenizer,
projection_matrix_path: str,
)#

Aligns student and teacher tokenizations of the same source text.

Alignment is offset-based: each token carries the (char_start, char_end) span it covers in the shared source text (return_offsets_mapping=True). Consecutive tokens with the same span collapse into a cluster, a strict char-end walker pairs student and teacher clusters covering the same character range, and special tokens (offset (0, 0)) are paired by role.

Parameters:
  • student_tokenizer – HF tokenizer for the student model. Must be a fast tokenizer so the collator can emit offset_mapping.

  • teacher_tokenizer – HF tokenizer for the teacher model.

  • projection_matrix_path –

    Path retained on the aligner for downstream callers (e.g. the loss fn) that materialize the projection on their training device via

    func:

    nemo_rl.algorithms.x_token.loss_utils.get_sparse_projection_matrix or :func:nemo_rl.algorithms.x_token.loss_utils.get_topk_projection.

Initialization

align(
student_ids: torch.Tensor,
teacher_ids: torch.Tensor,
*,
student_offsets: torch.Tensor,
teacher_offsets: torch.Tensor,
student_attention_mask: torch.Tensor | None = None,
teacher_attention_mask: torch.Tensor | None = None,
) → nemo_rl.algorithms.x_token.token_aligner.AlignmentBatch#

Align a batch of student/teacher token id tensors by char offsets.

Parameters:
  • student_ids – [B, T_s] long tensor.

  • teacher_ids – [B, T_t] long tensor.

  • student_offsets – [B, T_s, 2] long tensor of (char_start, char_end) per token, as produced by a fast tokenizer with return_offsets_mapping=True. Special/padding tokens carry (0, 0).

  • teacher_offsets – [B, T_t, 2] counterpart.

  • student_attention_mask – optional [B, T_s] mask (1 = real token, 0 = padding). When given, padded positions are forced to chunk_id = -1 so tokenizer padding never forms a valid chunk.

  • teacher_attention_mask – optional [B, T_t] counterpart.

Returns:

An :class:AlignmentBatch with all fields populated for the three loss modes.

align_chat(
student_ids: torch.Tensor,
teacher_ids: torch.Tensor,
*,
student_offsets: torch.Tensor,
teacher_offsets: torch.Tensor,
student_asst_char_spans: list[list[tuple[int, int]]],
teacher_asst_char_spans: list[list[tuple[int, int]]],
student_asst_mask: torch.Tensor,
teacher_asst_mask: torch.Tensor,
student_eot_indices: list[list[int]],
teacher_eot_indices: list[list[int]],
student_attention_mask: torch.Tensor | None = None,
teacher_attention_mask: torch.Tensor | None = None,
student_alignment_regions: list[list[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentRegions]] | None = None,
teacher_alignment_regions: list[list[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentRegions]] | None = None,
student_answer_parts: list[list[list[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentPart] | None]] | None = None,
teacher_answer_parts: list[list[list[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentPart] | None]] | None = None,
student_rendered_texts: list[str] | None = None,
teacher_rendered_texts: list[str] | None = None,
drop_first_content_pair: bool = False,
included_regions: Collection[str] | None = None,
) → nemo_rl.algorithms.x_token.token_aligner.AlignmentBatch#

Align assistant turns in a batch of independently rendered chats.

Parameters:
  • student_ids – Padded [B, T_s] student token IDs.

  • teacher_ids – Padded [B, T_t] teacher token IDs.

  • student_offsets – [B, T_s, 2] offsets in each student render.

  • teacher_offsets – [B, T_t, 2] offsets in each teacher render.

  • student_asst_char_spans – Per-sample assistant content spans in the student renders. Turns must correspond to the teacher spans.

  • teacher_asst_char_spans – Counterpart in the teacher renders.

  • student_asst_mask – [B, T_s] mask selecting tokens within assistant content regions.

  • teacher_asst_mask – [B, T_t] counterpart.

  • student_eot_indices – Per-sample, per-turn EOT positions. Use -1 when the render has no terminator token after the content.

  • teacher_eot_indices – Counterpart for the teacher tokens.

  • student_attention_mask – Optional [B, T_s] real-token mask. Padding is excluded from the returned chunk IDs.

  • teacher_attention_mask – Optional [B, T_t] counterpart.

  • student_alignment_regions – Optional per-sample, per-turn named reasoning, close, and answer character spans.

  • teacher_alignment_regions – Counterpart for the teacher renders.

  • student_answer_parts – Optional per-sample, per-turn structured answer pieces with original text and character spans.

  • teacher_answer_parts – Counterpart for the teacher renders.

  • student_rendered_texts – Original student renders, required for source-validated native Unicode repair.

  • teacher_rendered_texts – Counterpart for the teacher renders.

  • drop_first_content_pair – Omit each turn’s first paired content chunk from KD while retaining its EOT pair.

  • included_regions – Optional subset of reasoning, close, answer, and eot to align when semantic regions are supplied.

Returns:

A dense-padded :class:AlignmentBatch, using the same token-index coordinates and padding sentinels as :meth:align.

static _drop_padding(
batch: nemo_rl.algorithms.x_token.token_aligner.AlignmentBatch,
*,
student_attention_mask: torch.Tensor | None,
teacher_attention_mask: torch.Tensor | None,
) → None#

Strip tokenizer padding out of the chunk-id tensors.

Mutates batch in place. For every position the attention mask marks as padding, reset *_chunk_id to -1. Gating per position (rather than trimming a contiguous span) keeps this correct under either left- or right-padding. A pair whose tokens are entirely padding on one side then has size 0 there and is dropped by

Func:

nemo_rl.algorithms.x_token.loss_utils.valid_chunk_mask; a pair straddling the real/pad boundary shrinks to its real tokens.

static _pairs_to_batch(
per_sample_pairs: List[List[nemo_rl.algorithms.x_token.token_aligner.AlignmentPair]],
*,
b: int,
t_s: int,
t_t: int,
) → nemo_rl.algorithms.x_token.token_aligner.AlignmentBatch#

Pack per-sample alignment lists into dense-padded tensors.

_align_single(
student_ids: List[int],
teacher_ids: List[int],
student_offsets: List[Tuple[int, int]],
teacher_offsets: List[Tuple[int, int]],
) → List[nemo_rl.algorithms.x_token.token_aligner.AlignmentPair]#

Use the shared strict decoded matcher for ordinary text.

_align_one_offset(
student_ids: List[int],
teacher_ids: List[int],
student_offsets: List[Tuple[int, int]],
teacher_offsets: List[Tuple[int, int]],
) → List[nemo_rl.algorithms.x_token.token_aligner.AlignmentPair]#

Run strict offset clustering, then validate pairs by decoded text.

align_one_offset_per_asst(
student_ids: List[int],
student_offsets: List[Tuple[int, int]],
student_asst_char_spans: List[Tuple[int, int]],
teacher_ids: List[int],
teacher_offsets: List[Tuple[int, int]],
teacher_asst_char_spans: List[Tuple[int, int]],
*,
student_asst_mask: List[int],
teacher_asst_mask: List[int],
student_eot_indices: List[int],
teacher_eot_indices: List[int],
student_alignment_regions: List[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentRegions] | None = None,
teacher_alignment_regions: List[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentRegions] | None = None,
student_answer_parts: List[List[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentPart] | None] | None = None,
teacher_answer_parts: List[List[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentPart] | None] | None = None,
student_rendered_text: str | None = None,
teacher_rendered_text: str | None = None,
drop_first_content_pair: bool = False,
included_regions: Collection[str] | None = None,
) → List[nemo_rl.algorithms.x_token.token_aligner.AlignmentPair]#

Align native chat-template surfaces one assistant region at a time.

Student and teacher chat templates generally place the same assistant content at different absolute character offsets. Native-thinking templates can also use different whitespace around </think>. This method aligns paired semantic regions after rebasing each region to character offset zero, then translates the resulting token spans back to full-sequence indices. Template scaffold and side-specific boundary whitespace are therefore excluded from the KD payload.

Structured calls additionally bind prose and each tool payload to separate native pieces. Template-only separators, differently rendered scalar values, and tokens crossing those piece boundaries keep CE but are not compared as exact-text KD events.

Both sides must supply an assistant mask with one entry per token and an EOT index for each assistant turn. Use -1 when a rendered turn has no terminator token after its content.

_coalesce_native_piece_pairs(
pairs: List[nemo_rl.algorithms.x_token.token_aligner.AlignmentPair],
*,
student_ids: List[int],
teacher_ids: List[int],
student_offsets: List[Tuple[int, int]],
teacher_offsets: List[Tuple[int, int]],
text: str,
) → List[nemo_rl.algorithms.x_token.token_aligner.AlignmentPair]#

Bind complete Unicode fragments on an identical native piece.

A fast tokenizer may place an NFC-composed token over just the base character’s offset, or split an emoji into several byte tokens with the same offset. Adjacent offset clusters must then be considered together. Every emitted pair must decode to the same NFC text as its original character envelope. Boundary orphans remain CE-only.

nemo_rl.algorithms.x_token.token_aligner.canonical_token(token: str, *, enabled: bool = True) → str#

Return a canonical representation of a tokenizer token.

Public helper consumed by the alignment pipeline AND by the projection-prep CLIs in tools/x_token/. The enabled flag is a passthrough toggle: when False the input is returned unchanged (lets CLI call sites gate canonicalization via a single flag without branching at every site).

nemo_rl.algorithms.x_token.token_aligner._canonicalize_sequence(
seq: List[str],
) → Tuple[List[str], List[Tuple[int, int]]]#

Canonicalize every token in a sequence, including byte-merging.

Returns:

(canon, canon_to_orig). canon_to_orig[k] is a half-open [orig_start, orig_end) range giving the original-token positions that canonical token k was built from. Ranges are non-overlapping, strictly increasing, and jointly cover range(len(seq)), preserving the original token-index coordinates when canonical groups are used to normalize byte-fragment offsets.

nemo_rl.algorithms.x_token.token_aligner._normalize_canonical_merge_offsets(
token_strings: List[str],
offsets: List[Tuple[int, int]],
*,
token_ids: List[int],
special_token_ids: Collection[int],
) → List[Tuple[int, int]]#

Give every token in a canonical N-to-1 merge one offset envelope.

Fast byte-level tokenizers can assign nested offsets to the pieces of one Unicode character. Strict offset clustering would consume the outer piece alone and emit the nested piece as an orphan. The canonicalizer already identifies which consecutive token ranges form one character; assigning those original positions the same connected offset envelope lets offset clustering retain the complete N-to-1 span.

Token ids, strings, order, and sequence length are unchanged. A range is left untouched when it contains a special/zero-width token or its offsets are non-monotonic or separated by a character gap.

nemo_rl.algorithms.x_token.token_aligner._merge_encoding_artifacts(
tokens: List[str],
) → Tuple[List[str], List[Tuple[int, int]]]#

Merge known multi-token mojibake patterns into single tokens.

Returns:

(merged, ranges) with one (orig_start, orig_end) entry per output token. Every entry in :data:_MULTI_TOKEN_ARTIFACT_FIXES rewrites to a single replacement token, so each merge contributes exactly one range covering the matched pattern.

nemo_rl.algorithms.x_token.token_aligner._get_byte_value(token_char: str) → int | None#

Return the byte value (0..255) for a single character, or None.

nemo_rl.algorithms.x_token.token_aligner._merge_consecutive_bytes(
tokens: List[str],
in_ranges: List[Tuple[int, int]],
) → Tuple[List[str], List[Tuple[int, int]]]#

Merge consecutive byte-fallback tokens back into Unicode characters.

Propagates in_ranges parallel to tokens: when a byte buffer collapses to one character, its parallel range slice is collapsed to a single (start, end); otherwise ranges pass through unchanged.

nemo_rl.algorithms.x_token.token_aligner._try_merge_byte_buffer(
byte_tokens: List[str],
byte_ranges: List[Tuple[int, int]],
) → Tuple[List[str], List[Tuple[int, int]]]#

Decode 2-4 buffered byte tokens as a single UTF-8 character.

Returns the merged single-character token plus a single collapsed range covering the whole buffer, or the unchanged buffer + ranges when no merge is possible.

nemo_rl.algorithms.x_token.token_aligner._PAD_EQUIVALENT_ROLES#

None

nemo_rl.algorithms.x_token.token_aligner._role_of(tok, token_id: int) → str#

Classify a token id as a special-token role or "content".

nemo_rl.algorithms.x_token.token_aligner._partition(
input_ids: List[int],
offsets: List[Tuple[int, int]],
) → Tuple[List[int], List[Tuple[int, int, int]], List[int]]#

Split a sequence into leading-specials / content / trailing-specials.

Content vs special is decided by (0, 0) offset; mid-stream specials are folded into trailing.

nemo_rl.algorithms.x_token.token_aligner._pair_specials_by_role(
s_positions: List[int],
s_ids: List[int],
s_tok,
t_positions: List[int],
t_ids: List[int],
t_tok,
) → List[Tuple[List[int], List[int]]]#

Pair leading/trailing special tokens 1<->1 by matching role.

nemo_rl.algorithms.x_token.token_aligner._cluster_same_span(
content: List[Tuple[int, int, int]],
) → List[Tuple[int, int, List[int]]]#

Collapse consecutive tokens sharing the exact same (cs, ce) span.

Each run of same-span tokens becomes one cluster; different-span tokens (even overlapping) stay separate.

nemo_rl.algorithms.x_token.token_aligner._content_align_offset_cluster(
s_content: List[Tuple[int, int, int]],
t_content: List[Tuple[int, int, int]],
) → List[Tuple[List[int], List[int]]]#

Pair student/teacher clusters covering the same character range.

Pre-merges same-span clusters, then runs a strict char-end walker with orphan emission. Every paired group satisfies::

min(cs over G_s) == min(cs over G_t)
max(ce over G_s) == max(ce over G_t)
nemo_rl.algorithms.x_token.token_aligner.align_by_offsets_cluster(
student_ids: List[int],
student_offsets: List[Tuple[int, int]],
student_tokenizer,
teacher_ids: List[int],
teacher_offsets: List[Tuple[int, int]],
teacher_tokenizer,
) → List[nemo_rl.algorithms.x_token.token_aligner.AlignmentPair]#

Cluster + strict char-end offset alignment for a single sample.

Parameters:
  • student_ids – list of student token ids (len = ctx_length, incl. pad).

  • student_offsets – list of (cs, ce) tuples per token, from return_offsets_mapping=True on a fast HF tokenizer.

  • student_tokenizer – HF tokenizer (used for special-token role lookup).

  • teacher_ids/offsets/tokenizer – same for teacher.

Returns:

Alignment pairs with contiguous token position ranges on both sides; orphan groups use start=end=-1 for the empty side. All pairs leave is_correct=False for the caller to fill after text validation.