nemo_microservices.types.evaluation_params#
Module Contents#
Classes#
API#
- class nemo_microservices.types.evaluation_params.EvaluationParams(/, **data: Any)#
Bases:
nemo_microservices._models.BaseModel- extra: Dict[str, object] | None#
None
Any other custom parameters.
- limit_samples: int | None#
None
Limit number of evaluation samples
- max_retries: int | None#
None
Maximum number of retries for failed requests.
- max_tokens: int | None#
None
The maximum number of tokens to generate.
- parallelism: int | None#
None
Parallelism to be used for the evaluation job.
Typically, this represents the maximum number of concurrent requests made to the model.
- request_timeout: int | None#
None
The timeout to be used for requests made to the model.
- stop: str | List[str] | None#
None
Up to 4 sequences where the API will stop generating further tokens.
- temperature: float | None#
None
Float value between 0 and 1.
temp of 0 indicates greedy decoding, where the token with highest prob is chosen. Temperature can’t be set to 0.0 currently.
- top_p: float | None#
None
Float value between 0 and 1; limits to the top tokens within a certain probability. top_p=0 means the model will only consider the single most likely token for the next prediction.