nemo_rl.algorithms.x_token.token_aligner#
Cross-tokenizer token alignment.
The student and teacher tokenize the same source text; this module pairs their
tokens by the character spans each token covers (return_offsets_mapping=True
on a fast HF tokenizer). Consecutive tokens that share an exact
(char_start, char_end) collapse into a cluster, and a strict char-end
walker pairs student and teacher clusters covering the same character range.
Special tokens (BOS / EOS / pad, offset (0, 0)) are paired separately by
role. The result feeds the cross-tokenizer distillation loss.
Public surface:
- :class:AlignmentPair — per-pair record containing student/teacher
token ranges and an is_correct flag filled by the aligner.
- :class:AlignmentBatch — dense-padded per-batch alignment payload that
covers all three loss modes (P-KL, gold_loss, xtoken_loss).
- :class:TokenAligner — owns the two tokenizers and the projection
matrix, exposes :meth:align and :meth:align_chat for the collator.
- :func:align_by_offsets_cluster — the single-sample offset alignment
kernel, also usable directly.
Module Contents#
Classes#
One aligned span between student and teacher token sequences. |
|
One source-validated answer piece on a model’s rendered chat surface. |
|
Selected semantic character spans in one native assistant turn. |
|
Per-batch alignment payload covering all three loss modes. |
|
Aligns student and teacher tokenizations of the same source text. |
Functions#
Return a canonical representation of a tokenizer token. |
|
Canonicalize every token in a sequence, including byte-merging. |
|
Give every token in a canonical N-to-1 merge one offset envelope. |
|
Merge known multi-token mojibake patterns into single tokens. |
|
Return the byte value (0..255) for a single character, or None. |
|
Merge consecutive byte-fallback tokens back into Unicode characters. |
|
Decode 2-4 buffered byte tokens as a single UTF-8 character. |
|
Classify a token id as a special-token role or |
|
Split a sequence into leading-specials / content / trailing-specials. |
|
Pair leading/trailing special tokens 1<->1 by matching role. |
|
Collapse consecutive tokens sharing the exact same |
|
Pair student/teacher clusters covering the same character range. |
|
Cluster + strict char-end offset alignment for a single sample. |
Data#
API#
- nemo_rl.algorithms.x_token.token_aligner.VISUAL_BYTE_MAP#
None
- nemo_rl.algorithms.x_token.token_aligner._MULTI_TOKEN_ARTIFACT_FIXES: list[tuple[list[str], list[str]]]#
[([‘ĠâĪ’, ‘ij’], [‘Ġ∑’]), ([‘âĪ’, ‘ij’], [’∑’]), ([‘ĠâĪ’, ‘ı’], [‘Ġ∏’]), ([‘âĪ’, ‘ı’], [’∏’]), ([‘ĠâĪ’…
- nemo_rl.algorithms.x_token.token_aligner._MULTI_TOKEN_ARTIFACT_FIXES_BY_FIRST: dict[str, list[tuple[list[str], list[str]]]]#
None
- nemo_rl.algorithms.x_token.token_aligner._UNICODE_FIXES#
None
- nemo_rl.algorithms.x_token.token_aligner._SPECIAL_TOKEN_MAP#
None
- class nemo_rl.algorithms.x_token.token_aligner.AlignmentPair#
One aligned span between student and teacher token sequences.
The alignment sets
is_correctusing decoded-text equality, NFC-validated native repair, or matching EOS roles. Insertions/deletions (orphan groups) use-1for the empty side’s start/end indices... attribute:: s_start
Inclusive start index into the student token sequence (
-1for teacher-only insertions)... attribute:: s_end
Exclusive end index into the student token sequence (
-1for teacher-only insertions)... attribute:: t_start
Inclusive start index into the teacher token sequence (
-1for student-only insertions)... attribute:: t_end
Exclusive end index into the teacher token sequence (
-1for student-only insertions)... attribute:: is_correct
Truewhen the decoded student and teacher span texts are equal, when an NFC-validated native repair matches them, or when both spans are their tokenizer’s EOS token. Defaults toFalseuntil the aligner validates the pair.- s_start: int#
None
- s_end: int#
None
- t_start: int#
None
- t_end: int#
None
- is_correct: bool#
False
- class nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentPart#
One source-validated answer piece on a model’s rendered chat surface.
Differently spelled Boolean/null values may retain CE without exact-text KD.
- name: str#
None
- span: tuple[int, int]#
None
- text: str#
None
- allow_native_difference: bool#
False
- class nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentRegions#
Selected semantic character spans in one native assistant turn.
- reasoning: tuple[int, int] | None#
None
- close: tuple[int, int] | None#
None
- answer: tuple[int, int] | None#
None
- get(name: str) tuple[int, int] | None#
Return the named semantic span, rejecting unknown region names.
- class nemo_rl.algorithms.x_token.token_aligner.AlignmentBatch#
Per-batch alignment payload covering all three loss modes.
The collator hands this dataclass directly to the loss fn alongside the tokenized batch. Tensors are dense-padded to the batch maximum so DTensor V2 can shard on dim 0 without knowing about cross-tokenizer specifics.
.. attribute:: pair_valid
[B, max_pairs]bool. False on padding entries... attribute:: pair_is_correct
[B, max_pairs]bool. True when decoded student and teacher span texts are equal, when an NFC-validated native repair matches them, or when both spans are their tokenizer’s EOS token... attribute:: student_chunk_id
[B, T_s]long. Chunk index (= pair index) the student token belongs to;-1if not in any chunk (insertion-only pair on student side)... attribute:: teacher_chunk_id
[B, T_t]long. Counterpart.- pair_valid: torch.Tensor#
None
- pair_is_correct: torch.Tensor#
None
- student_chunk_id: torch.Tensor#
None
- teacher_chunk_id: torch.Tensor#
None
- class nemo_rl.algorithms.x_token.token_aligner.TokenAligner(
- student_tokenizer,
- teacher_tokenizer,
- projection_matrix_path: str,
Aligns student and teacher tokenizations of the same source text.
Alignment is offset-based: each token carries the
(char_start, char_end)span it covers in the shared source text (return_offsets_mapping=True). Consecutive tokens with the same span collapse into a cluster, a strict char-end walker pairs student and teacher clusters covering the same character range, and special tokens (offset(0, 0)) are paired by role.- Parameters:
student_tokenizer – HF tokenizer for the student model. Must be a fast tokenizer so the collator can emit
offset_mapping.teacher_tokenizer – HF tokenizer for the teacher model.
projection_matrix_path –
Path retained on the aligner for downstream callers (e.g. the loss fn) that materialize the projection on their training device via
- func:
nemo_rl.algorithms.x_token.loss_utils.get_sparse_projection_matrixor :func:nemo_rl.algorithms.x_token.loss_utils.get_topk_projection.
Initialization
- align(
- student_ids: torch.Tensor,
- teacher_ids: torch.Tensor,
- *,
- student_offsets: torch.Tensor,
- teacher_offsets: torch.Tensor,
- student_attention_mask: torch.Tensor | None = None,
- teacher_attention_mask: torch.Tensor | None = None,
Align a batch of student/teacher token id tensors by char offsets.
- Parameters:
student_ids –
[B, T_s]long tensor.teacher_ids –
[B, T_t]long tensor.student_offsets –
[B, T_s, 2]long tensor of(char_start, char_end)per token, as produced by a fast tokenizer withreturn_offsets_mapping=True. Special/padding tokens carry(0, 0).teacher_offsets –
[B, T_t, 2]counterpart.student_attention_mask – optional
[B, T_s]mask (1 = real token, 0 = padding). When given, padded positions are forced tochunk_id = -1so tokenizer padding never forms a valid chunk.teacher_attention_mask – optional
[B, T_t]counterpart.
- Returns:
An :class:
AlignmentBatchwith all fields populated for the three loss modes.
- align_chat(
- student_ids: torch.Tensor,
- teacher_ids: torch.Tensor,
- *,
- student_offsets: torch.Tensor,
- teacher_offsets: torch.Tensor,
- student_asst_char_spans: list[list[tuple[int, int]]],
- teacher_asst_char_spans: list[list[tuple[int, int]]],
- student_asst_mask: torch.Tensor,
- teacher_asst_mask: torch.Tensor,
- student_eot_indices: list[list[int]],
- teacher_eot_indices: list[list[int]],
- student_attention_mask: torch.Tensor | None = None,
- teacher_attention_mask: torch.Tensor | None = None,
- student_alignment_regions: list[list[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentRegions]] | None = None,
- teacher_alignment_regions: list[list[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentRegions]] | None = None,
- student_answer_parts: list[list[list[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentPart] | None]] | None = None,
- teacher_answer_parts: list[list[list[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentPart] | None]] | None = None,
- student_rendered_texts: list[str] | None = None,
- teacher_rendered_texts: list[str] | None = None,
- drop_first_content_pair: bool = False,
- included_regions: Collection[str] | None = None,
Align assistant turns in a batch of independently rendered chats.
- Parameters:
student_ids – Padded
[B, T_s]student token IDs.teacher_ids – Padded
[B, T_t]teacher token IDs.student_offsets –
[B, T_s, 2]offsets in each student render.teacher_offsets –
[B, T_t, 2]offsets in each teacher render.student_asst_char_spans – Per-sample assistant content spans in the student renders. Turns must correspond to the teacher spans.
teacher_asst_char_spans – Counterpart in the teacher renders.
student_asst_mask –
[B, T_s]mask selecting tokens within assistant content regions.teacher_asst_mask –
[B, T_t]counterpart.student_eot_indices – Per-sample, per-turn EOT positions. Use
-1when the render has no terminator token after the content.teacher_eot_indices – Counterpart for the teacher tokens.
student_attention_mask – Optional
[B, T_s]real-token mask. Padding is excluded from the returned chunk IDs.teacher_attention_mask – Optional
[B, T_t]counterpart.student_alignment_regions – Optional per-sample, per-turn named reasoning, close, and answer character spans.
teacher_alignment_regions – Counterpart for the teacher renders.
student_answer_parts – Optional per-sample, per-turn structured answer pieces with original text and character spans.
teacher_answer_parts – Counterpart for the teacher renders.
student_rendered_texts – Original student renders, required for source-validated native Unicode repair.
teacher_rendered_texts – Counterpart for the teacher renders.
drop_first_content_pair – Omit each turn’s first paired content chunk from KD while retaining its EOT pair.
included_regions – Optional subset of reasoning, close, answer, and eot to align when semantic regions are supplied.
- Returns:
A dense-padded :class:
AlignmentBatch, using the same token-index coordinates and padding sentinels as :meth:align.
- static _drop_padding(
- batch: nemo_rl.algorithms.x_token.token_aligner.AlignmentBatch,
- *,
- student_attention_mask: torch.Tensor | None,
- teacher_attention_mask: torch.Tensor | None,
Strip tokenizer padding out of the chunk-id tensors.
Mutates
batchin place. For every position the attention mask marks as padding, reset*_chunk_idto-1. Gating per position (rather than trimming a contiguous span) keeps this correct under either left- or right-padding. A pair whose tokens are entirely padding on one side then has size 0 there and is dropped by- Func:
nemo_rl.algorithms.x_token.loss_utils.valid_chunk_mask; a pair straddling the real/pad boundary shrinks to its real tokens.
- static _pairs_to_batch(
- per_sample_pairs: List[List[nemo_rl.algorithms.x_token.token_aligner.AlignmentPair]],
- *,
- b: int,
- t_s: int,
- t_t: int,
Pack per-sample alignment lists into dense-padded tensors.
- _align_single(
- student_ids: List[int],
- teacher_ids: List[int],
- student_offsets: List[Tuple[int, int]],
- teacher_offsets: List[Tuple[int, int]],
Use the shared strict decoded matcher for ordinary text.
- _align_one_offset(
- student_ids: List[int],
- teacher_ids: List[int],
- student_offsets: List[Tuple[int, int]],
- teacher_offsets: List[Tuple[int, int]],
Run strict offset clustering, then validate pairs by decoded text.
- align_one_offset_per_asst(
- student_ids: List[int],
- student_offsets: List[Tuple[int, int]],
- student_asst_char_spans: List[Tuple[int, int]],
- teacher_ids: List[int],
- teacher_offsets: List[Tuple[int, int]],
- teacher_asst_char_spans: List[Tuple[int, int]],
- *,
- student_asst_mask: List[int],
- teacher_asst_mask: List[int],
- student_eot_indices: List[int],
- teacher_eot_indices: List[int],
- student_alignment_regions: List[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentRegions] | None = None,
- teacher_alignment_regions: List[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentRegions] | None = None,
- student_answer_parts: List[List[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentPart] | None] | None = None,
- teacher_answer_parts: List[List[nemo_rl.algorithms.x_token.token_aligner.NativeAlignmentPart] | None] | None = None,
- student_rendered_text: str | None = None,
- teacher_rendered_text: str | None = None,
- drop_first_content_pair: bool = False,
- included_regions: Collection[str] | None = None,
Align native chat-template surfaces one assistant region at a time.
Student and teacher chat templates generally place the same assistant content at different absolute character offsets. Native-thinking templates can also use different whitespace around
</think>. This method aligns paired semantic regions after rebasing each region to character offset zero, then translates the resulting token spans back to full-sequence indices. Template scaffold and side-specific boundary whitespace are therefore excluded from the KD payload.Structured calls additionally bind prose and each tool payload to separate native pieces. Template-only separators, differently rendered scalar values, and tokens crossing those piece boundaries keep CE but are not compared as exact-text KD events.
Both sides must supply an assistant mask with one entry per token and an EOT index for each assistant turn. Use
-1when a rendered turn has no terminator token after its content.
- _coalesce_native_piece_pairs(
- pairs: List[nemo_rl.algorithms.x_token.token_aligner.AlignmentPair],
- *,
- student_ids: List[int],
- teacher_ids: List[int],
- student_offsets: List[Tuple[int, int]],
- teacher_offsets: List[Tuple[int, int]],
- text: str,
Bind complete Unicode fragments on an identical native piece.
A fast tokenizer may place an NFC-composed token over just the base character’s offset, or split an emoji into several byte tokens with the same offset. Adjacent offset clusters must then be considered together. Every emitted pair must decode to the same NFC text as its original character envelope. Boundary orphans remain CE-only.
- nemo_rl.algorithms.x_token.token_aligner.canonical_token(token: str, *, enabled: bool = True) str#
Return a canonical representation of a tokenizer token.
Public helper consumed by the alignment pipeline AND by the projection-prep CLIs in
tools/x_token/. Theenabledflag is a passthrough toggle: whenFalsethe input is returned unchanged (lets CLI call sites gate canonicalization via a single flag without branching at every site).
- nemo_rl.algorithms.x_token.token_aligner._canonicalize_sequence(
- seq: List[str],
Canonicalize every token in a sequence, including byte-merging.
- Returns:
(canon, canon_to_orig).canon_to_orig[k]is a half-open[orig_start, orig_end)range giving the original-token positions that canonical tokenkwas built from. Ranges are non-overlapping, strictly increasing, and jointly coverrange(len(seq)), preserving the original token-index coordinates when canonical groups are used to normalize byte-fragment offsets.
- nemo_rl.algorithms.x_token.token_aligner._normalize_canonical_merge_offsets(
- token_strings: List[str],
- offsets: List[Tuple[int, int]],
- *,
- token_ids: List[int],
- special_token_ids: Collection[int],
Give every token in a canonical N-to-1 merge one offset envelope.
Fast byte-level tokenizers can assign nested offsets to the pieces of one Unicode character. Strict offset clustering would consume the outer piece alone and emit the nested piece as an orphan. The canonicalizer already identifies which consecutive token ranges form one character; assigning those original positions the same connected offset envelope lets offset clustering retain the complete N-to-1 span.
Token ids, strings, order, and sequence length are unchanged. A range is left untouched when it contains a special/zero-width token or its offsets are non-monotonic or separated by a character gap.
- nemo_rl.algorithms.x_token.token_aligner._merge_encoding_artifacts(
- tokens: List[str],
Merge known multi-token mojibake patterns into single tokens.
- Returns:
(merged, ranges)with one(orig_start, orig_end)entry per output token. Every entry in :data:_MULTI_TOKEN_ARTIFACT_FIXESrewrites to a single replacement token, so each merge contributes exactly one range covering the matched pattern.
- nemo_rl.algorithms.x_token.token_aligner._get_byte_value(token_char: str) int | None#
Return the byte value (0..255) for a single character, or None.
- nemo_rl.algorithms.x_token.token_aligner._merge_consecutive_bytes(
- tokens: List[str],
- in_ranges: List[Tuple[int, int]],
Merge consecutive byte-fallback tokens back into Unicode characters.
Propagates
in_rangesparallel totokens: when a byte buffer collapses to one character, its parallel range slice is collapsed to a single(start, end); otherwise ranges pass through unchanged.
- nemo_rl.algorithms.x_token.token_aligner._try_merge_byte_buffer(
- byte_tokens: List[str],
- byte_ranges: List[Tuple[int, int]],
Decode 2-4 buffered byte tokens as a single UTF-8 character.
Returns the merged single-character token plus a single collapsed range covering the whole buffer, or the unchanged buffer + ranges when no merge is possible.
- nemo_rl.algorithms.x_token.token_aligner._PAD_EQUIVALENT_ROLES#
None
- nemo_rl.algorithms.x_token.token_aligner._role_of(tok, token_id: int) str#
Classify a token id as a special-token role or
"content".
- nemo_rl.algorithms.x_token.token_aligner._partition(
- input_ids: List[int],
- offsets: List[Tuple[int, int]],
Split a sequence into leading-specials / content / trailing-specials.
Content vs special is decided by
(0, 0)offset; mid-stream specials are folded into trailing.
- nemo_rl.algorithms.x_token.token_aligner._pair_specials_by_role(
- s_positions: List[int],
- s_ids: List[int],
- s_tok,
- t_positions: List[int],
- t_ids: List[int],
- t_tok,
Pair leading/trailing special tokens 1<->1 by matching role.
- nemo_rl.algorithms.x_token.token_aligner._cluster_same_span(
- content: List[Tuple[int, int, int]],
Collapse consecutive tokens sharing the exact same
(cs, ce)span.Each run of same-span tokens becomes one cluster; different-span tokens (even overlapping) stay separate.
- nemo_rl.algorithms.x_token.token_aligner._content_align_offset_cluster(
- s_content: List[Tuple[int, int, int]],
- t_content: List[Tuple[int, int, int]],
Pair student/teacher clusters covering the same character range.
Pre-merges same-span clusters, then runs a strict char-end walker with orphan emission. Every paired group satisfies::
min(cs over G_s) == min(cs over G_t) max(ce over G_s) == max(ce over G_t)
- nemo_rl.algorithms.x_token.token_aligner.align_by_offsets_cluster(
- student_ids: List[int],
- student_offsets: List[Tuple[int, int]],
- student_tokenizer,
- teacher_ids: List[int],
- teacher_offsets: List[Tuple[int, int]],
- teacher_tokenizer,
Cluster + strict char-end offset alignment for a single sample.
- Parameters:
student_ids – list of student token ids (len = ctx_length, incl. pad).
student_offsets – list of
(cs, ce)tuples per token, fromreturn_offsets_mapping=Trueon a fast HF tokenizer.student_tokenizer – HF tokenizer (used for special-token role lookup).
teacher_ids/offsets/tokenizer – same for teacher.
- Returns:
Alignment pairs with contiguous token position ranges on both sides; orphan groups use
start=end=-1for the empty side. All pairs leaveis_correct=Falsefor the caller to fill after text validation.