`nemo_microservices.types.shared.model_spec`#

Module Contents#

class nemo_microservices.types.shared.model_spec.ModelSpec(/, **data: typing.Any)#

Bases: nemo_microservices._models.BaseModel

context_size: int#

None

The maximum number of tokens to process together in a single forward pass through the model.

is_chat: bool#

None

Indicates if the model is designed for multi-turn conversation rather than single-prompt completion.

num_parameters: int#

None

The total number of trainable parameters in the model’s neural network architecture.

num_virtual_tokens: int#

None

The number of virtual tokens the model can support for techniques such as prompt tuning, where special trainable embeddings are prepended to inputs.