`nemo_microservices.types.shared.inference_params`#

Module Contents#

Classes#

InferenceParams

API#

class nemo_microservices.types.shared.inference_params.InferenceParams(/, **data: Any)#

Bases: nemo_microservices._models.BaseModel

max_tokens: int | None#: None

stop: List[str] | None#: None

temperature: float | None#: None

top_k: int | None#: None

top_p: float | None#: None