Skip to main content
Ctrl+K
NVIDIA NeMo Microservices - Home NVIDIA NeMo Microservices - Home

NVIDIA NeMo Microservices

NVIDIA NeMo Microservices - Home NVIDIA NeMo Microservices - Home

NVIDIA NeMo Microservices

Table of Contents

About NeMo Microservices

  • Overview
  • Key Features
  • Concepts
    • Platform
    • Entities
    • Customization
    • Evaluation
    • Inference
    • Guardrails
  • Release Notes

Get Started

  • About Getting Started
  • Beginner Tutorial Prerequisites
  • Beginner Tutorials
    • Deploy NIM for LLMs
    • Customize and Evaluate LLMs
    • Add Safety Checks to LLMs

Jupyter Notebooks

  • Data Flywheel and Tool Calling

Manage Entities

  • About Managing Entities
  • Tutorials
    • Set Up Organizational Entities
    • Create Dataset Files
  • Namespaces
    • Create Namespace
    • Update Namespace
    • List Namespaces
    • Delete Namespace
  • Projects
    • Create Project
    • Update Project
    • List Projects
    • Delete Project
  • Datasets
    • Create Dataset
    • Get Dataset
    • Update Dataset
    • List Datasets
    • Delete Dataset
  • Models
    • Get Model
    • Update Model
    • List Models
    • Delete Model
  • Entity Fields Reference

Fine-Tune

  • About Fine-Tuning
  • Tutorials
    • Format Training Dataset
    • Start a LoRA Model Customization Job
    • Start a Full SFT Customization Job
    • Checking Your Customization Job Metrics
    • Optimize for Tokens/GPU Throughput
  • Model Catalog
    • Llama Models
    • Phi Models
  • Manage Customization Configs
    • Get Customization Configs
    • Customization Config
  • Manage Customization Jobs
    • Create Job
    • Get Job Status
    • List Active Jobs
    • Cancel Job
    • Hyperparameter Options

Evaluate

  • About Evaluating
  • Tutorials
    • Run a Simple Evaluation
  • Evaluation Types
  • Evaluation Jobs
  • Evaluation Targets
  • Evaluation Configurations
  • Custom Evaluations
  • Evaluation Results
  • Use Custom Data
  • Filter and Sort Responses
  • Support Matrix

Deploy and Proxy NIMs

  • About Deploying and Proxying NIM for LLMs
  • Deploy NVIDIA NIM for large language models (LLMs)
  • Proxy Deployed NIM for LLMs

Manage Guardrails

  • About Guardrails
  • Terminology
  • Tutorials
    • Demo Configuration
    • Multiple NIM for LLMs
    • NemoGuard NIM
    • Multimodal Data
    • Custom Dependencies
    • Custom HTTP Headers
    • Custom LLM Providers
    • Deploying with Docker
  • Manage Configurations
    • Configuration Store
    • Creating a Configuration
    • Listing Configurations
    • Getting a Configuration
    • Updating a Configuration
    • Deleting a Configuration
  • Check a Guardrail
  • Inference with Guardrails
  • Streaming Output
  • Reference

Admin Setup

  • About Admin Setup
  • Installation Scenarios
  • Helm Installation Overview
  • Install as Platform
    • Install
    • Ingress Setup
    • Upgrade
    • Uninstall
  • Install Individually
    • NeMo Data Store
    • NeMo Entity Store
    • NeMo Operator
    • NeMo Customizer
    • NeMo Evaluator
    • NeMo Guardrails
    • DGX Cloud Admission Controller
    • NeMo Deployment Management
    • NeMo NIM Proxy
  • Model Configurations
  • Custom Resource Definitions
  • Manage Storage
    • Databases
      • PostgreSQL
      • Milvus
    • ReadWriteMany Persistent Volumes
      • AWS Peristent Volumes
      • Oracle Persistent Volumes
    • Object Storage
      • Amazon S3
    • Backup and Restore
  • Manage Secrets
    • Secrets for Accessing NGC Catalog
    • External Database Secrets
    • JSON Web Token Secrets
    • Object Store Secrets
    • MLFlow Customizer Secrets
    • Weights & Biases Keys
  • Open Telemetry Setup
  • Tenant Configuration Options
  • Security for NeMo Microservices

Reference

  • NeMo Microservice APIs
    • Platform
    • Entity Store
    • Data Store
    • Customizer
    • Evaluator
    • Guardrails
    • NIM Proxy
    • Deployment Management
  • Requirements
  • NeMo Helm Chart Values
    • Platform
    • Entity Store
    • Data Store
    • Customizer
    • Evaluator
    • Guardrails
    • NIM Proxy
    • Deployment Management
    • DGX Cloud Admission Controller
  • Troubleshooting Guide
    • Troubleshoot Customizer
    • Troubleshoot Evaluator
    • Troubleshoot Guardrails
    • Troubleshoot Setup
  • Governing Terms

Resources

  • OSS License Acknowledgements
  • Manage Storage
  • Databases
  • Milvus

Milvus#

NeMo Evaluator uses Milvus for vector database storage for Retriever Pipelines and RAG Pipelines.

To learn how to configure the NeMo Evaluator microservice to use Milvus, see Configure Milvus in the NeMo Evaluator setup guide.

previous

PostgreSQL

next

ReadWriteMany Persistent Volumes

NVIDIA NVIDIA
Privacy Policy | Manage My Privacy | Do Not Sell or Share My Data | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2024-2025, NVIDIA Corporation.

Last updated on May 23, 2025.