nemo_microservices.types.guardrail_check_params

Module Contents

Classes

GuardrailCheckParams

Data

Message

API

class nemo_microservices.types.guardrail_check_params.GuardrailCheckParams

Bases: typing_extensions.TypedDict

best_of: int

None

Not supported.

Generates best_of completions server-side and returns the “best” (the one with the highest log probability per token). Results cannot be streamed. When used with n, best_of controls the number of candidate completions and n specifies how many to return - best_of must be greater than n.

echo: bool

None

Not supported.

If echo is true, the response will include the prompt and optionally its tokens ids and logprobs.

frequency_penalty: float

None

Positive values penalize new tokens based on their existing frequency in the text.

function_call: str | Dict[str, object]

None

Not Supported.

Deprecated in favor of tool_choice. ‘none’ means the model will not call a function and instead generates a message. ‘auto’ means the model can pick between generating a message or calling a function. Specifying a particular function via {‘name’: ‘my_function’} forces the model to call that function.

guardrails: nemo_microservices.types.guardrails_data_param.GuardrailsDataParam

None

Guardrails specific options for the request.

ignore_eos: bool

None

Ignore the eos when running

logit_bias: Dict[str, float]

None

Not Supported.

Modify the likelihood of specified tokens appearing in the completion.

logprobs: bool

None

Whether to return log probabilities of the output tokens or not.

If true, returns the log probabilities of each output token returned in the content of message

max_tokens: int

None

The maximum number of tokens that can be generated in the chat completion.

messages: typing_extensions.Required[Iterable[Message]]

None

A list of messages comprising the conversation so far

model: typing_extensions.Required[str]

None

The model to use for completion. Must be one of the available models.

n: int

None

How many chat completion choices to generate for each input message.

presence_penalty: float

None

Positive values penalize new tokens based on whether they appear in the text so far.

response_format: Dict[str, str]

None

Format of the response, can be ‘json_object’ to force the model to output valid JSON.

seed: int

None

If specified, attempts to sample deterministically.

stop: str | nemo_microservices._types.SequenceNotStr[str]

None

Up to 4 sequences where the API will stop generating further tokens.

stream: bool

None

If set, partial message deltas will be sent, like in ChatGPT.

suffix: str

None

Not supported. If echo is set, the prompt is returned with the completion.

system_fingerprint: str

None

Represents the backend configuration that the model runs with.

Used with seed for determinism.

temperature: float

None

What sampling temperature to use, between 0 and 2.

tool_choice: str | Dict[str, object]

None

Not Supported.

Favored over function_call. Controls which (if any) function is called by the model.

tools: nemo_microservices._types.SequenceNotStr[str]

None

A list of tools the model may call.

top_logprobs: int

None

The number of most likely tokens to return at each token position.

top_p: float

None

An alternative to sampling with temperature, called nucleus sampling.

user: str

None

Not Supported. A unique identifier representing your end-user.

vision: bool

None

Whether this is a vision-capable request with image inputs.

nemo_microservices.types.guardrail_check_params.Message: typing_extensions.TypeAlias

None