ModelSpec

class nemo_microservices.types.shared_params.ModelSpec

Bases: TypedDict

context_size: Required[int]

The maximum number of tokens to process together in a single forward pass through the model.

is_chat: Required[bool]

Indicates if the model is designed for multi-turn conversation rather than single-prompt completion.

num_parameters: Required[int]

The total number of trainable parameters in the model’s neural network architecture.

num_virtual_tokens: Required[int]

The number of virtual tokens the model can support for techniques such as prompt tuning, where special trainable embeddings are prepended to inputs.