`nemo_microservices.types.beta.safe_synthesizer.generate_parameters`#

Module Contents#

class nemo_microservices.types.beta.safe_synthesizer.generate_parameters.GenerateParameters(/, **data: Any)#

guided_decoding_backend: str | None#

None

The backend used for guided decoding when use_structured_generation=True.

Supported backends (from vllm) are ‘outlines’, ‘guidance’, ‘xgrammar’, and ‘auto’.

invalid_fraction_threshold: float | None#

None

The fraction of invalid records that will stop generation after the patience limit is reached.

num_records: int | None#

None

Number of records to generate.

patience: int | None#

None

Number of consecutive generations where the invalid_fraction_threshold is reached before stopping generation.

repetition_penalty: float | None#

None

The value used to control the likelihood of the model repeating the same token.

temperature: float | None#

None

Sampling temperature.

top_p: float | None#

None

Nucleus sampling probability.

use_structured_generation: bool | None#

None

Use structured generation.