nemo_rl.models.generation.vllm.utils

Module Contents

Functions

format_prompt_for_vllm_generation

Format a list of prompts for vllm generation (which requires a specific format for its own generate method).

API

nemo_rl.models.generation.vllm.utils.format_prompt_for_vllm_generation(

data: nemo_rl.distributed.batched_data_dict.BatchedDataDict[nemo_rl.models.generation.interfaces.GenerationDatumSpec],

sample_idx: Optional[int] = None,

) → list[dict[str, Any]]

Format a list of prompts for vllm generation (which requires a specific format for its own generate method).

See https://docs.vllm.ai/en/v0.9.1/features/multimodal_inputs.html for prompt format for multimodal inputs.