Skip to main content
Ctrl+K
NVIDIA NeMo Microservices - Home NVIDIA NeMo Microservices - Home

NVIDIA NeMo Microservices

NVIDIA NeMo Microservices - Home NVIDIA NeMo Microservices - Home

NVIDIA NeMo Microservices

Table of Contents

About NeMo Microservices

  • Overview
  • Key Features
  • Concepts
    • Platform
    • Entities
    • Customization
    • Evaluation
    • Inference
    • Guardrails
  • Release Notes

Get Started

  • About Getting Started
  • Demo Cluster Setup on Minikube
    • Requirements
    • Set Up Using Deployment Scripts
    • Set Up Manually
  • Beginner Tutorials
    • Deploy NIM
    • Customize and Evaluate LLMs
    • Add Safety Checks to LLMs
    • Use Llama Stack APIs

Jupyter Notebooks

  • Data Flywheel and Tool Calling

Manage Entities

  • About Managing Entities
  • Tutorials
    • Set Up Organizational Entities
    • Create Dataset Files
  • Namespaces
    • Create Namespace
    • Update Namespace
    • List Namespaces
    • Delete Namespace
  • Projects
    • Create Project
    • Update Project
    • List Projects
    • Delete Project
  • Datasets
    • Create Dataset
    • Get Dataset
    • Update Dataset
    • List Datasets
    • Delete Dataset
  • Models
    • Get Model
    • Update Model
    • List Models
    • Delete Model
  • Entity Fields Reference

Fine-Tune

  • About Fine-Tuning
  • Tutorials
    • Format Training Dataset
    • Start a LoRA Model Customization Job
    • Start a Full SFT Customization Job
    • Start a Knowledge Distillation (KD) Customization Job
    • Checking Your Customization Job Metrics
    • Optimize for Tokens/GPU Throughput
  • Model Catalog
    • Llama Models
    • Llama Nemotron Models
    • Phi Models
  • Manage Targets
    • Create Target
    • Get Target Details
    • List Targets
    • Update Target
    • Delete Target
    • Target Values
  • Manage Configs
    • Create Config
    • Get Config Details
    • List Configs
    • Update Config
  • Manage Jobs
    • Create Job
    • Get Job Status
    • List Active Jobs
    • Cancel Job
    • Hyperparameter Options

Evaluate

  • About Evaluating
  • Tutorials
    • Run a Simple Job
  • Evaluation Types
    • Agentic
    • BFCL
    • BigCode
    • Custom
    • LM Harness
    • RAG
    • Retriever
    • Similarity Metrics
  • Targets
    • Create Target
    • Delete Target
    • Data Source Targets
    • LLM Model Targets
    • Retriever Pipeline Targets
    • RAG Pipeline Targets
    • Target Schema
  • Configurations
    • Create Config
    • Delete Config
    • System Prompts for Advanced Reasoning
    • Config Schema
  • Jobs
    • Create Job
    • Get Job Details
    • Get Job Status
    • List Jobs
    • Get Job Results
    • Download Detailed Results
    • Get Job Logs
    • Delete Job
    • Job Target & Config Matrix
    • Job Durations
    • Job Schema
  • Live Evaluations
  • Custom Evaluations
    • Data Format
    • Task Templating
    • Metrics
    • Output Format & Results
  • Results
  • Filter and Sort Responses
  • Support Matrix

Deploy NIM and Run Inference

  • About Deploying and Running Inference on NIM
  • Tutorials
    • Deploy NIM
    • Run Inference on NIM
  • Manage NIM Deployments
    • Deploy NIM Microservice
    • Get NIM Deployment Details
    • List Deployments
    • Update Deployment
    • Delete NIM Deployment
    • Create Configuration
    • Get Configuration
    • List Configurations
    • Update Configuration
    • Delete Configuration
  • Run Inference on NIM
    • Health Check
    • List Models
    • Chat Completions
    • Completions

Manage Guardrails

  • About Guardrails
  • Terminology
  • Tutorials
    • Demo Configuration
    • Multiple NIM for LLMs
    • NemoGuard NIM
    • Multimodal Data
    • Injection Detection
    • Custom Dependencies
    • Custom HTTP Headers
    • Custom LLM Providers
    • Deploying with Docker
  • Manage Configurations
    • Configuration Store
    • Creating a Configuration
    • Listing Configurations
    • Getting a Configuration
    • Updating a Configuration
    • Deleting a Configuration
  • Manage Access to Models
  • Check a Guardrail
  • Inference with Guardrails
  • Streaming Output
  • Reference

Admin Setup

  • About Admin Setup
  • Installation Scenarios
  • Helm Installation Overview
  • Install as Platform
    • Install
    • Ingress Setup
    • Upgrade
    • Uninstall
  • Install Individually
    • NeMo Data Store
    • NeMo Entity Store
    • NeMo Operator
    • NeMo Customizer
    • NeMo Evaluator
      • Helm Chart
      • Docker Compose
      • Chart Config Options
    • NeMo Guardrails
    • DGX Cloud Admission Controller
    • NeMo Deployment Management
    • NeMo NIM Proxy
  • Configure Models
  • Manage GPUs
    • Configure Cluster GPUs
    • Model Configurations Matrix
    • Troubleshooting GPU Jobs
  • Custom Resource Definitions
  • Manage Storage
    • Databases
      • PostgreSQL
      • Milvus
    • PVCs
      • AWS Peristent Volumes
      • Oracle Persistent Volumes
    • Object Storage
      • Amazon S3
    • Backup and Restore
  • Manage Secrets
    • Secrets for Accessing NGC Catalog
    • External Database Secrets
    • JSON Web Token Secrets
    • Object Store Secrets
    • MLFlow Customizer Secrets
    • Weights & Biases Keys
  • Open Telemetry Setup
  • Tenant Configuration Options
  • Security for NeMo Microservices

Reference

  • NeMo Microservice APIs
    • Platform
    • Entity Store
    • Data Store
    • Customizer
    • Evaluator
    • Guardrails
    • NIM Proxy
    • Deployment Management
  • Requirements
  • NeMo Helm Chart Values
    • Platform
    • Entity Store
    • Data Store
    • Customizer
    • Evaluator
    • Guardrails
    • NIM Proxy
    • Deployment Management
    • DGX Cloud Admission Controller
  • Troubleshooting Guide
    • Troubleshoot Customizer
    • Troubleshoot Evaluator
    • Troubleshoot Guardrails
    • Troubleshoot Setup
  • NVIDIA Distribution in Llama Stack
  • Governing Terms

Resources

  • OSS License Acknowledgements
  • Manage Storage
  • Databases
  • Milvus

Milvus#

NeMo Evaluator uses Milvus for vector database storage for Retriever Evaluation Type and RAG Evaluation Type.

To learn how to configure the NeMo Evaluator microservice to use Milvus, see Milvus in the NeMo Evaluator setup guide.

previous

PostgreSQL

next

ReadWriteMany Persistent Volumes

NVIDIA NVIDIA
Privacy Policy | Manage My Privacy | Do Not Sell or Share My Data | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2024-2025, NVIDIA Corporation.

Last updated on Jun 11, 2025.