PromptDataDeParam

class nemo_microservices.types.PromptDataDeParam

Bases: TypedDict

icl_few_shot_examples: str

A string including a set of examples. These are pre-pended to the prompt.

inference_params: InferenceParams

Parameters that influence the inference of a model.

system_prompt: str

The system prompt that should be applied during inference.

system_prompt_template: str

The template which will be used to compile the final prompt used for prompting the LLM. Currently supports only {{icl_few_shot_examples}}