Skip to main content
Ctrl+K
NVIDIA NeMo Microservices - Home NVIDIA NeMo Microservices - Home

NVIDIA NeMo Microservices

NVIDIA NeMo Microservices - Home NVIDIA NeMo Microservices - Home

NVIDIA NeMo Microservices

Table of Contents

About NeMo Microservices

  • Overview
  • Key Features
  • Concepts
    • Platform
    • Entities
    • Customization
    • Evaluation
    • Inference
    • Guardrails
  • Release Notes

Get Started

  • About Getting Started
  • Demo Cluster Setup on Minikube
    • Requirements
    • Set Up Using Deployment Scripts
    • Set Up Manually
  • Beginner Tutorials
    • Deploy NIM
    • Customize and Evaluate LLMs
    • Add Safety Checks to LLMs
    • Use Llama Stack APIs

Jupyter Notebooks

  • Data Flywheel and Tool Calling

Manage Entities

  • About Managing Entities
  • Tutorials
    • Set Up Organizational Entities
    • Create Dataset Files
  • Namespaces
    • Create Namespace
    • Update Namespace
    • List Namespaces
    • Delete Namespace
  • Projects
    • Create Project
    • Update Project
    • List Projects
    • Delete Project
  • Datasets
    • Create Dataset
    • Get Dataset
    • Update Dataset
    • List Datasets
    • Delete Dataset
  • Models
    • Get Model
    • Update Model
    • List Models
    • Delete Model
  • Entity Fields Reference

Fine-Tune

  • About Fine-Tuning
  • Tutorials
    • Format Training Dataset
    • Start a LoRA Model Customization Job
    • Start a Full SFT Customization Job
    • Start a Knowledge Distillation (KD) Customization Job
    • Checking Your Customization Job Metrics
    • Optimize for Tokens/GPU Throughput
  • Model Catalog
    • Llama Models
    • Llama Nemotron Models
    • Phi Models
  • Manage Targets
    • Create Target
    • Get Target Details
    • List Targets
    • Update Target
    • Delete Target
    • Target Values
  • Manage Configs
    • Create Config
    • Get Config Details
    • List Configs
    • Update Config
  • Manage Jobs
    • Create Job
    • Get Job Status
    • List Active Jobs
    • Cancel Job
    • Hyperparameter Options

Evaluate

  • About Evaluating
  • Tutorials
    • Run a Simple Job
  • Evaluation Types
    • Agentic
    • BFCL
    • BigCode
    • Custom
    • LM Harness
    • RAG
    • Retriever
    • Similarity Metrics
  • Targets
    • Create Target
    • Delete Target
    • Data Source Targets
    • LLM Model Targets
    • Retriever Pipeline Targets
    • RAG Pipeline Targets
    • Target Schema
  • Configurations
    • Create Config
    • Delete Config
    • System Prompts for Advanced Reasoning
    • Config Schema
  • Jobs
    • Create Job
    • Get Job Details
    • Get Job Status
    • List Jobs
    • Get Job Results
    • Download Detailed Results
    • Get Job Logs
    • Delete Job
    • Job Target & Config Matrix
    • Job Durations
    • Job Schema
  • Live Evaluations
  • Custom Evaluations
    • Data Format
    • Task Templating
    • Metrics
    • Output Format & Results
  • Results
  • Filter and Sort Responses
  • Support Matrix

Deploy NIM and Run Inference

  • About Deploying and Running Inference on NIM
  • Tutorials
    • Deploy NIM
    • Run Inference on NIM
  • Manage NIM Deployments
    • Deploy NIM Microservice
    • Get NIM Deployment Details
    • List Deployments
    • Update Deployment
    • Delete NIM Deployment
    • Create Configuration
    • Get Configuration
    • List Configurations
    • Update Configuration
    • Delete Configuration
  • Run Inference on NIM
    • Health Check
    • List Models
    • Chat Completions
    • Completions

Manage Guardrails

  • About Guardrails
  • Terminology
  • Tutorials
    • Demo Configuration
    • Multiple NIM for LLMs
    • NemoGuard NIM
    • Multimodal Data
    • Injection Detection
    • Custom Dependencies
    • Custom HTTP Headers
    • Custom LLM Providers
    • Deploying with Docker
  • Manage Configurations
    • Configuration Store
    • Creating a Configuration
    • Listing Configurations
    • Getting a Configuration
    • Updating a Configuration
    • Deleting a Configuration
  • Manage Access to Models
  • Check a Guardrail
  • Inference with Guardrails
  • Streaming Output
  • Reference

Admin Setup

  • About Admin Setup
  • Installation Scenarios
  • Helm Installation Overview
  • Install as Platform
    • Install
    • Ingress Setup
    • Upgrade
    • Uninstall
  • Install Individually
    • NeMo Data Store
    • NeMo Entity Store
    • NeMo Operator
    • NeMo Customizer
    • NeMo Evaluator
      • Helm Chart
      • Docker Compose
      • Chart Config Options
    • NeMo Guardrails
    • DGX Cloud Admission Controller
    • NeMo Deployment Management
    • NeMo NIM Proxy
  • Configure Models
  • Manage GPUs
    • Configure Cluster GPUs
    • Model Configurations Matrix
    • Troubleshooting GPU Jobs
  • Custom Resource Definitions
  • Manage Storage
    • Databases
      • PostgreSQL
      • Milvus
    • PVCs
      • AWS Peristent Volumes
      • Oracle Persistent Volumes
    • Object Storage
      • Amazon S3
    • Backup and Restore
  • Manage Secrets
    • Secrets for Accessing NGC Catalog
    • External Database Secrets
    • JSON Web Token Secrets
    • Object Store Secrets
    • MLFlow Customizer Secrets
    • Weights & Biases Keys
  • Open Telemetry Setup
  • Tenant Configuration Options
  • Security for NeMo Microservices

Reference

  • NeMo Microservice APIs
    • Platform
    • Entity Store
    • Data Store
    • Customizer
    • Evaluator
    • Guardrails
    • NIM Proxy
    • Deployment Management
  • Requirements
  • NeMo Helm Chart Values
    • Platform
    • Entity Store
    • Data Store
    • Customizer
    • Evaluator
    • Guardrails
    • NIM Proxy
    • Deployment Management
    • DGX Cloud Admission Controller
  • Troubleshooting Guide
    • Troubleshoot Customizer
    • Troubleshoot Evaluator
    • Troubleshoot Guardrails
    • Troubleshoot Setup
  • NVIDIA Distribution in Llama Stack
  • Governing Terms

Resources

  • OSS License Acknowledgements
  • Governing Terms

Governing Terms#

The software and materials are governed by the NVIDIA Software License Agreement and the Product-Specific Terms for NVIDIA AI Products.

previous

Troubleshooting NeMo Microservices Deployment on Kubernetes

next

OSS License Acknowledgements
NVIDIA NVIDIA
Privacy Policy | Manage My Privacy | Do Not Sell or Share My Data | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2024-2025, NVIDIA Corporation.

Last updated on Jun 11, 2025.