nemo_automodel.shared.retrieval_ids

View as Markdown

Stable passage identities shared by retrieval text and image collators.

Module Contents

Functions

NameDescription
document_id_to_int64Encode a corpus ID consistently across processes and input modalities.

API

nemo_automodel.shared.retrieval_ids.document_id_to_int64(
document_id: str
) -> int

Encode a corpus ID consistently across processes and input modalities.

Parameters:

document_id
str

Corpus document identifier encoded as UTF-8.

Returns: int

Nonnegative 63-bit integer, preserving the existing text-collator encoding.