nemo_microservices.types.evaluation_params_param#
Module Contents#
Classes#
API#
- class nemo_microservices.types.evaluation_params_param.EvaluationParamsParam#
Bases:
typing_extensions.TypedDict- extra: Dict[str, object]#
None
Any other custom parameters.
- limit_samples: int#
None
Limit number of evaluation samples
- max_retries: int#
None
Maximum number of retries for failed requests.
- max_tokens: int#
None
The maximum number of tokens to generate.
- parallelism: int#
None
Parallelism to be used for the evaluation job.
Typically, this represents the maximum number of concurrent requests made to the model.
- request_timeout: int#
None
The timeout to be used for requests made to the model.
- stop: str | nemo_microservices._types.SequenceNotStr[str]#
None
Up to 4 sequences where the API will stop generating further tokens.
- temperature: float#
None
Float value between 0 and 1.
temp of 0 indicates greedy decoding, where the token with highest prob is chosen. Temperature can’t be set to 0.0 currently.
- top_p: float#
None
Float value between 0 and 1; limits to the top tokens within a certain probability. top_p=0 means the model will only consider the single most likely token for the next prediction.