Is this page helpful?

`nemo_microservices.types.shared.inference_params`#

Module Contents#

Classes#

InferenceParams

API#

class nemo_microservices.types.shared.inference_params.InferenceParams(/, **data: typing.Any)#

Bases: nemo_microservices._models.BaseModel

max_tokens: Optional[int]#: None

stop: Optional[List[str]]#: None

temperature: Optional[float]#: None

top_k: Optional[int]#: None

top_p: Optional[float]#: None