About Setup & Deployment#
The administration section provides comprehensive information for deployment, infrastructure management, monitoring, and scaling NeMo Curator. Use these resources to efficiently set up and maintain your NeMo Curator environment at any scale, from development workstations to production clusters.
Installation & Configuration#
Install NeMo Curator with system requirements, package extras, and verification steps. Covers PyPI, source, and container installation methods.
Configure NeMo Curator for deployment environments, storage access, credentials, and environment variables for operational management.
Deployment Options#
Deploy NeMo Curator on Kubernetes clusters using Dask Operator, GPU Operator, and PVC storage. Includes setup, storage, cluster creation, module execution, and cleanup.
Run NeMo Curator on Slurm clusters with shared filesystems. Covers job scripts, Dask cluster setup, module execution, monitoring, and advanced Python-based job submission.
Integration Options#
Integrate NeMo Curator with Apache Spark for distributed processing