`nemo_rl.environments.nemo_gym_video`#

Module Contents#

Functions#

`_require_video_config_value`
`_get_content_part_url`	Return a string media source from a Responses/Chat content part.
`_resolve_local_video_path`	Resolve a local video source and reject unsupported remote schemes.
`normalize_video_urls_in_examples`	Convert bare local video paths to file URLs before Gym dispatch.
`_extract_static_video_messages`	Convert one-video Responses input into HF multimodal chat messages.
`_json_mapping`
`_metadata_extra_body`
`_chat_template_kwargs_for_processor`
`_deep_merge_dict`
`_inject_vllm_mm_processor_kwargs`
`_remove_vllm_mm_processor_kwargs`
`_replace_cached_video_frames_with_native_video`	Replace cached image parts with one lossless native-video manifest.
`_strip_local_media_metadata`
`_compute_dynamic_prompt_length`
`_video_to_image_content`
`_make_overlength_filtered_video_example`
`nemo_gym_example_to_video_datum_spec`	Preprocess static Gym video with vLLM-equivalent frame sampling.

Data#

`_VideoConfigValue`
`_LOCAL_VIDEO_METADATA_KEYS`

API#

nemo_rl.environments.nemo_gym_video._VideoConfigValue#: ‘TypeVar(…)’

nemo_rl.environments.nemo_gym_video._LOCAL_VIDEO_METADATA_KEYS#: ‘frozenset(…)’

nemo_rl.environments.nemo_gym_video._require_video_config_value( value: nemo_rl.environments.nemo_gym_video._VideoConfigValue | None, field_name: str, ) → nemo_rl.environments.nemo_gym_video._VideoConfigValue#

nemo_rl.environments.nemo_gym_video._get_content_part_url(part: dict[str, Any], *keys: str) → str#: Return a string media source from a Responses/Chat content part.

nemo_rl.environments.nemo_gym_video._resolve_local_video_path(source: str) → str#: Resolve a local video source and reject unsupported remote schemes.

nemo_rl.environments.nemo_gym_video.normalize_video_urls_in_examples( examples: list[dict[str, Any]], ) → None#: Convert bare local video paths to file URLs before Gym dispatch.

nemo_rl.environments.nemo_gym_video._extract_static_video_messages( nemo_gym_example: dict[str, Any], ) → tuple[list[dict[str, Any]], str | None] | None#

Convert one-video Responses input into HF multimodal chat messages.

A video may be represented either by one native video content part or by a sequence of cached input_image parts carrying _is_video_frame. The latter is the on-disk frame-cache format used by the video Gym recipes.

nemo_rl.environments.nemo_gym_video._json_mapping( value: Any, *, field_name: str, ) → dict[str, Any]#

nemo_rl.environments.nemo_gym_video._metadata_extra_body( nemo_gym_example: dict[str, Any], ) → dict[str, Any]#

nemo_rl.environments.nemo_gym_video._chat_template_kwargs_for_processor( nemo_gym_example: dict[str, Any], ) → dict[str, Any]#

nemo_rl.environments.nemo_gym_video._deep_merge_dict( base: dict[str, Any], update: dict[str, Any], ) → dict[str, Any]#

nemo_rl.environments.nemo_gym_video._inject_vllm_mm_processor_kwargs( nemo_gym_example: dict[str, Any], mm_processor_kwargs: dict[str, Any], ) → None#

nemo_rl.environments.nemo_gym_video._remove_vllm_mm_processor_kwargs( nemo_gym_example: dict[str, Any], names: set[str], ) → None#

nemo_rl.environments.nemo_gym_video._replace_cached_video_frames_with_native_video( nemo_gym_example: dict[str, Any], ) → None#: Replace cached image parts with one lossless native-video manifest.

nemo_rl.environments.nemo_gym_video._strip_local_media_metadata( nemo_gym_example: dict[str, Any], ) → None#

nemo_rl.environments.nemo_gym_video._compute_dynamic_prompt_length( processor: Any, messages: list[dict[str, Any]], template_kwargs: dict[str, Any], ) → int | None#

nemo_rl.environments.nemo_gym_video._video_to_image_content( video_path: str, *, num_frames: int, temporal_patch_size: int, sampling_style: nemo_rl.models.generation.vllm.video_utils.VideoSamplingStyle, ) → tuple[list[dict[str, Any]], dict[str, Any]]#

nemo_rl.environments.nemo_gym_video._make_overlength_filtered_video_example( nemo_gym_example: dict[str, Any], ) → dict[str, Any]#

nemo_rl.environments.nemo_gym_video.nemo_gym_example_to_video_datum_spec( nemo_gym_example: dict[str, Any], *, processor: Any, max_seq_length: int | None, idx: int, task_name: str, data_config: nemo_rl.data.interfaces.TaskDataSpec | None = None, ) → dict[str, Any] | None#

Preprocess static Gym video with vLLM-equivalent frame sampling.

The raw video remains in the outbound Gym request. Cached frames are sent as one native-video manifest so vLLM consumes the same lossless RGB frames as policy preprocessing. Those tensors are reattached to vLLM-authored prompt token IDs after the rollout.

nemo_rl.environments.nemo_gym_video#

Module Contents#

Functions#

Data#

API#

`nemo_rl.environments.nemo_gym_video`#