> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/v26.09/reference/infra/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

# Infra

## Docs

- [Memory Management](https://docs.nvidia.com/nemo/curator/v26.09/reference/infra/memory-management.md): Strategies and best practices for managing memory when processing large datasets with NeMo Curator
- [Monitoring](https://docs.nvidia.com/nemo/curator/v26.09/reference/infra/monitoring.md): Set up Prometheus and Grafana to monitor NeMo Curator pipelines on single-user and multi-user Ray clusters
- [GPU Processing](https://docs.nvidia.com/nemo/curator/v26.09/reference/infra/gpu-processing.md): Guide to leveraging NVIDIA GPU acceleration in NeMo Curator for faster data processing and memory optimization
- [Resumable Processing](https://docs.nvidia.com/nemo/curator/v26.09/reference/infra/resumable-processing.md): Resume interrupted NeMo Curator pipelines with source-level completion checkpoints
- [Execution Backends](https://docs.nvidia.com/nemo/curator/v26.09/reference/infra/execution-backends.md): Choose and configure execution backends for NeMo Curator pipelines
- [Stage Worker Sizing](https://docs.nvidia.com/nemo/curator/v26.09/reference/infra/stage-worker-sizing.md): Configure fixed and autoscaling worker pools and backend stage overrides for Ray Data, Xenna, and Ray Actor Pool executors
- [Per-Stage Runtime Environments](https://docs.nvidia.com/nemo/curator/v26.09/reference/infra/per-stage-runtime.md): Isolate Python dependencies per pipeline stage using Ray runtime environments
- [Container Environments](https://docs.nvidia.com/nemo/curator/v26.09/reference/infra/container-environments.md): Build, configure, and use NeMo Curator containers in local and cluster environments


> **Note:** This page contains both a page directory (above) and the landing page content (below). The page directory is generated for agent use and does not appear on the landing page.

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/curator/v26.09/reference/infra/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/curator/_mcp/server.

> Technical reference for NeMo Curator's infrastructure components including distributed computing, memory management, and GPU acceleration

# Infrastructure References

Browse these references to configure distributed execution, memory use, GPU processing, checkpoints, workers, and containers across NeMo Curator modalities.

---

## Infrastructure Components

#### [Memory Management](/reference/infra/memory-management)

Optimize memory usage when processing large datasets
partitioning
batching
monitoring

#### [GPU Acceleration](/reference/infra/gpu-processing)

Use NVIDIA GPUs to accelerate data processing
cuda
rmm
performance

#### [Resumable Processing](/reference/infra/resumable-processing)

Continue interrupted operations across large datasets
checkpoints
recovery
batching

#### [Stage Worker Sizing](/reference/infra/stage-worker-sizing)

Configure fixed and autoscaling worker pools across Ray Data, Xenna, and Ray Actor Pool backends
workers
autoscaling
stage-overrides

#### [Container Environments](/reference/infra/container-environments)

Build your own NeMo Curator container and configure its dependencies, build arguments, and runtime environment.
docker
uv
environments