> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/aistore/llms.txt.
> For full documentation content, see https://docs.nvidia.com/aistore/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/aistore/_mcp/server.

# aistore.pytorch.multishard_dataset

Multishard Stream Dataset for AIS.

Copyright (c) 2024-2025, NVIDIA CORPORATION. All rights reserved.

## Module Contents

### Classes

| Name                                                                             | Description                                                                                      |
| -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| [`AISMultiShardStream`](#aistore-pytorch-multishard_dataset-AISMultiShardStream) | An iterable-style dataset that iterates over multiple shard streams and yields combined samples. |

### API

```python
class aistore.pytorch.multishard_dataset.AISMultiShardStream(
    data_sources: typing.List[aistore.sdk.DataShard]
)
```

**Bases:** `IterableDataset`

An iterable-style dataset that iterates over multiple shard streams and yields combined samples.

**Parameters:**

List of DataShard objects

**Returns:**

Iterable over the combined samples, where each sample is a tuple of
one object bytes from each shard stream

```python
aistore.pytorch.multishard_dataset.AISMultiShardStream.__iter__() -> typing.Iterator
```

```python
aistore.pytorch.multishard_dataset.AISMultiShardStream._get_shard_objects_iterator(
    bucket: aistore.sdk.Bucket,
    prefix: str = '',
    etl_name: str = ''
) -> typing.Iterable[bytes]
```

Create an iterable over all the objects in the given shards.

**Parameters:**

Bucket containing the shards

Prefix of the object names

ETL name to apply on each object

**Returns:** `Iterable[bytes]`

Iterable\[Object]: Iterable over all the objects in the given shards,
with each iteration returning a combined sample