> This page is for version 0.2.1.
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.

# nemo_gym.core.media

Media carried in Responses messages, such as an `input_image` part holding a base64 data URI.

`split_text_and_media` decodes a message into its text and its media files,
and `MediaFile` converts between bytes and data URIs.
`nemo_gym.sandbox.stage_media` writes the decoded files into a sandbox for a harness CLI to read.

Images are the supported media today.
Supporting another kind means adding its media types to `_MEDIA_TYPE_EXTENSIONS` and `_MEDIA_TYPE_SIGNATURES`
and its Responses part type to `_PART_URL_FIELDS`, or a decoder for a part that does not hold a data URI.

## Module Contents

### Classes

| Name                                                                      | Description                                                                  |
| ------------------------------------------------------------------------- | ---------------------------------------------------------------------------- |
| [`MediaFile`](#nemo_gym-core-media-MediaFile)                             | Decoded media content and its media type, such as `image/png`.               |
| [`UnsupportedContentError`](#nemo_gym-core-media-UnsupportedContentError) | A message holds content that cannot be passed on as text or as a media file. |

### Functions

| Name                                                                | Description                                                                        |
| ------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |
| [`_media_type_of`](#nemo_gym-core-media-_media_type_of)             | -                                                                                  |
| [`_part_field`](#nemo_gym-core-media-_part_field)                   | -                                                                                  |
| [`split_text_and_media`](#nemo_gym-core-media-split_text_and_media) | Return a message's text, joined across text parts, and its decoded media in order. |

### Data

[`_MEDIA_TYPE_ALIASES`](#nemo_gym-core-media-_MEDIA_TYPE_ALIASES)

[`_MEDIA_TYPE_EXTENSIONS`](#nemo_gym-core-media-_MEDIA_TYPE_EXTENSIONS)

[`_MEDIA_TYPE_SIGNATURES`](#nemo_gym-core-media-_MEDIA_TYPE_SIGNATURES)

[`_PART_URL_FIELDS`](#nemo_gym-core-media-_PART_URL_FIELDS)

[`_TEXT_PART_TYPES`](#nemo_gym-core-media-_TEXT_PART_TYPES)

### API

```python
class nemo_gym.core.media.MediaFile(
    media_type: str,
    data: bytes
)
```

Dataclass

Decoded media content and its media type, such as `image/png`.

The content always starts with the bytes that identify its media type.

**`data`** `bytes = field(repr=False)`

---

**`extension`** `str`

The file extension, without the dot, that files of this media type are written with.

---

**`media_type`** `str`

---

```python
nemo_gym.core.media.MediaFile.__post_init__() -> None
```

```python
nemo_gym.core.media.MediaFile.from_data_uri(
    uri: str
) -> nemo_gym.core.media.MediaFile
```

classmethod

Decode a `data:<media type>[;<parameter>]...;base64,<payload>` URI.

The URI must name a supported media type, and the payload must hold data of a supported media type.
When the two differ, as for a PNG labeled `image/jpeg`, the payload's own type wins,
so the file gets the extension its content needs.

```python
nemo_gym.core.media.MediaFile.to_data_uri() -> str
```

Encode as a `data:<media type>;base64,<payload>` URI.

```python
class nemo_gym.core.media.UnsupportedContentError()
```

**Bases:** `ValueError`

A message holds content that cannot be passed on as text or as a media file.

The caller should reject the request rather than drop the content.

```python
nemo_gym.core.media._media_type_of(
    data: bytes
) -> str | None
```

```python
nemo_gym.core.media._part_field(
    part: object,
    name: str
) -> object
```

```python
nemo_gym.core.media.split_text_and_media(
    content: str | collections.abc.Iterable[object]
) -> tuple[str, list[nemo_gym.core.media.MediaFile]]
```

Return a message's text, joined across text parts, and its decoded media in order.

Each part may be a mapping or a Responses content part model.
Raises `UnsupportedContentError` for a part that is neither text nor supported media.

```python
nemo_gym.core.media._MEDIA_TYPE_ALIASES = {'image/jpg': 'image/jpeg'}
```

```python
nemo_gym.core.media._MEDIA_TYPE_EXTENSIONS = {'image/png': 'png', 'image/jpeg': 'jpg', 'image/gif': 'gif', 'image/webp': 'web...
```

```python
nemo_gym.core.media._MEDIA_TYPE_SIGNATURES = {'image/png': re.compile(b'\\x89PNG\\r\\n\\x1a\\n.{4}IHDR', re.DOTALL), 'image/j...
```

```python
nemo_gym.core.media._PART_URL_FIELDS = {'input_image': 'image_url'}
```

```python
nemo_gym.core.media._TEXT_PART_TYPES = {'input_text', 'text'}
```