***

layout: overview
slug: nemo-curator/nemo\_curator/utils/writer\_utils
title: nemo\_curator.utils.writer\_utils
----------------------------------------

## Module Contents

### Classes

| Name                                                                      | Description                                                            |
| ------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| [`JsonEncoderCustom`](#nemo_curator-utils-writer_utils-JsonEncoderCustom) | Custom JSON encoder that handles types that are not JSON serializable. |

### Functions

| Name                                                              | Description                  |
| ----------------------------------------------------------------- | ---------------------------- |
| [`write_bytes`](#nemo_curator-utils-writer_utils-write_bytes)     | Write bytes to local path.   |
| [`write_csv`](#nemo_curator-utils-writer_utils-write_csv)         | Write csv to local path.     |
| [`write_json`](#nemo_curator-utils-writer_utils-write_json)       | Write json to local path.    |
| [`write_parquet`](#nemo_curator-utils-writer_utils-write_parquet) | Write parquet to local path. |

### API

<Anchor id="nemo_curator-utils-writer_utils-JsonEncoderCustom">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    class nemo_curator.utils.writer_utils.JsonEncoderCustom()
    ```
  </CodeBlock>
</Anchor>

<Indent>
  **Bases:** `JSONEncoder`

  Custom JSON encoder that handles types that are not JSON serializable.

  Example:

  <CodeBlock showLineNumbers={false}>
    ```python
    json.dumps(data, cls=JsonEncoderClass)
    ```
  </CodeBlock>

  <Anchor id="nemo_curator-utils-writer_utils-JsonEncoderCustom-default">
    <CodeBlock showLineNumbers={false} wordWrap={true}>
      ```python
      nemo_curator.utils.writer_utils.JsonEncoderCustom.default(
          obj: object
      ) -> str | object
      ```
    </CodeBlock>
  </Anchor>

  <Indent>
    Encode an object for JSON serialization.

    **Parameters:**

    <ParamField path="obj" type="object">
      Object to encode.
    </ParamField>

    **Returns:** `str | object`

    Encoded object.
  </Indent>
</Indent>

<Anchor id="nemo_curator-utils-writer_utils-write_bytes">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    nemo_curator.utils.writer_utils.write_bytes(
        buffer: bytes,
        dest: pathlib.Path,
        desc: str,
        source_video: str,
        verbose: bool,
        backup_and_overwrite: bool = False,
        overwrite: bool = False
    ) -> None
    ```
  </CodeBlock>
</Anchor>

<Indent>
  Write bytes to local path.

  **Parameters:**

  <ParamField path="buffer" type="bytes">
    Bytes to write.
  </ParamField>

  <ParamField path="dest" type="pathlib.Path">
    Destination to write.
  </ParamField>

  <ParamField path="desc" type="str">
    Description of the write.
  </ParamField>

  <ParamField path="source_video" type="str">
    Source video.
  </ParamField>

  <ParamField path="verbose" type="bool">
    Verbosity.
  </ParamField>

  <ParamField path="client">
    Storage client.
  </ParamField>

  <ParamField path="backup_and_overwrite" type="bool" default="False">
    Backup and overwrite.
  </ParamField>

  <ParamField path="overwrite" type="bool" default="False">
    Overwrite.
  </ParamField>
</Indent>

<Anchor id="nemo_curator-utils-writer_utils-write_csv">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    nemo_curator.utils.writer_utils.write_csv(
        dest: pathlib.Path,
        desc: str,
        source_video: str,
        data: list[list[str]],
        verbose: bool,
        backup_and_overwrite: bool = False
    ) -> None
    ```
  </CodeBlock>
</Anchor>

<Indent>
  Write csv to local path.

  **Parameters:**

  <ParamField path="dest" type="pathlib.Path">
    Destination to write.
  </ParamField>

  <ParamField path="desc" type="str">
    Description of the write.
  </ParamField>

  <ParamField path="source_video" type="str">
    Source video.
  </ParamField>

  <ParamField path="data" type="list[list[str]]">
    Data to write.
  </ParamField>

  <ParamField path="verbose" type="bool">
    Verbosity.
  </ParamField>

  <ParamField path="client">
    Storage client.
  </ParamField>

  <ParamField path="backup_and_overwrite" type="bool" default="False">
    Backup and overwrite.
  </ParamField>
</Indent>

<Anchor id="nemo_curator-utils-writer_utils-write_json">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    nemo_curator.utils.writer_utils.write_json(
        data: dict[str, typing.Any],
        dest: pathlib.Path,
        desc: str,
        source_video: str,
        verbose: bool,
        backup_and_overwrite: bool = False,
        overwrite: bool = False
    ) -> None
    ```
  </CodeBlock>
</Anchor>

<Indent>
  Write json to local path.

  **Parameters:**

  <ParamField path="data" type="dict[str, Any]">
    Data to write.
  </ParamField>

  <ParamField path="dest" type="pathlib.Path">
    Destination to write.
  </ParamField>

  <ParamField path="desc" type="str">
    Description of the write.
  </ParamField>

  <ParamField path="source_video" type="str">
    Source video.
  </ParamField>

  <ParamField path="verbose" type="bool">
    Verbosity.
  </ParamField>

  <ParamField path="client">
    Storage client.
  </ParamField>

  <ParamField path="backup_and_overwrite" type="bool" default="False">
    Backup and overwrite.
  </ParamField>

  <ParamField path="overwrite" type="bool" default="False">
    Overwrite.
  </ParamField>
</Indent>

<Anchor id="nemo_curator-utils-writer_utils-write_parquet">
  <CodeBlock showLineNumbers={false} wordWrap={true}>
    ```python
    nemo_curator.utils.writer_utils.write_parquet(
        data: list[dict[str, str]],
        dest: pathlib.Path,
        desc: str,
        source_video: str,
        verbose: bool,
        backup_and_overwrite: bool = False,
        overwrite: bool = False
    ) -> None
    ```
  </CodeBlock>
</Anchor>

<Indent>
  Write parquet to local path.

  **Parameters:**

  <ParamField path="data" type="list[dict[str, str]]">
    Data to write.
  </ParamField>

  <ParamField path="dest" type="pathlib.Path">
    Destination to write.
  </ParamField>

  <ParamField path="desc" type="str">
    Description of the write.
  </ParamField>

  <ParamField path="source_video" type="str">
    Source video.
  </ParamField>

  <ParamField path="verbose" type="bool">
    Verbosity.
  </ParamField>

  <ParamField path="client">
    Storage client.
  </ParamField>

  <ParamField path="backup_and_overwrite" type="bool" default="False">
    Whether to backup existing file before overwriting.
  </ParamField>

  <ParamField path="overwrite" type="bool" default="False">
    Whether to overwrite existing file.
  </ParamField>
</Indent>