Skip to main content
Ctrl+K
NVIDIA NeMo Microservices - Home NVIDIA NeMo Microservices - Home

NVIDIA NeMo Microservices

NVIDIA NeMo Microservices - Home NVIDIA NeMo Microservices - Home

NVIDIA NeMo Microservices

Table of Contents

About NeMo Microservices

  • Overview
  • Key Features
  • Concepts
    • Platform
    • Entities
    • Customization
    • Evaluation
    • Inference
    • Guardrails
    • Synthetic Data Generation
  • Release Notes

Get Started

  • About Getting Started
  • Demo Cluster Setup
    • Requirements
    • Minikube
      • Set Up Using Deployment Scripts
      • Set Up Manually
    • Kubernetes
      • Install
      • Ingress Setup
      • Upgrade
      • Uninstall
  • Install NeMo Microservices Python SDK
  • Beginner Platform Tutorials
    • Deploy NIM
    • Customize and Evaluate LLMs
    • Add Safety Checks to LLMs
    • Use Llama Stack APIs
  • Jupyter Notebooks

Manage Entities

  • About Managing Entities
  • Tutorials
    • Set Up Organizational Entities
    • Create Dataset Files
  • Namespaces
    • Create Namespace
    • Update Namespace
    • Get Namespace
    • List Namespaces
    • Delete Namespace
  • Projects
    • Create Project
    • Update Project
    • Get Project
    • List Projects
    • Delete Project
  • Datasets
    • Create Dataset
    • Get Dataset
    • Update Dataset
    • List Datasets
    • Delete Dataset
  • Models
    • Get Details of a Model
    • Update Model
    • List Models
    • Delete Model
  • Entity Fields Reference

Generate Synthetic Data (Beta)

  • About Generating Synthetic Data
  • Quickstart
  • Tutorials
  • Configure Models
  • Seeding SDG with External Data
  • Generate Realistic Personal Details
  • Define Data Columns
    • Column Types
      • Sampling-Based Columns
      • Expression Columns
      • LLM-Based Columns
    • Multi-Modal Context
    • Add Constraints
    • Using Jinja Templates
    • Structured Outputs
  • Generate Data
    • Data Generation Workflow
    • Manage Jobs
      • Create Data Generation Job
      • Get Job Status
      • List Data Generation Jobs
      • Get Job Logs
      • Get Job Results
      • Download Job Results
  • Data Designer Configuration
    • Save Configurations
    • Load Configurations
  • Data Quality
    • Code Validation
    • Data Evaluation
    • Add Custom Validation

Fine-Tune

  • About Fine-Tuning
  • Tutorials
    • Format Training Dataset
    • LoRA Customization
    • SFT Customization
    • DPO Customization
    • Distillation Customization
    • Check Job Metrics
    • Optimize Throughput
    • Import Private HF Model
  • Model Catalog
    • Llama Models
    • Llama Nemotron Models
    • Phi Models
    • Embedding Models
    • GPT-OSS Models
    • Gemma Models
    • Dataset Format Requirements
  • Manage Targets
    • Create Target
    • Get Target Details
    • List Targets
    • Update Target
    • Delete Target
    • Target Values
  • Manage Configs
    • Create Config
    • Get Config Details
    • List Configs
    • Update Config
  • Manage Jobs
    • Create Job
    • Get Job Status
    • List Active Jobs
    • Cancel Job
    • Hyperparameter Options

Evaluate

  • About Evaluating
  • Quickstart
  • Tutorials
    • Run an LLM Harness Eval
    • Run an LLM Judge Eval
  • Evaluation Flows
    • Academic Benchmarks
      • BigCode
      • BFCL
      • LM Harness
      • Safety Harness
      • Simple Evals
    • Retrieval
    • RAG
    • Agentic
    • LLM-as-a-Judge
    • Template
    • Prompt Optimization
  • Targets
    • Create Target
    • Delete Target
    • Data Source Targets
    • LLM Model Targets
    • Retriever Pipeline Targets
    • RAG Pipeline Targets
    • Target Schema
  • Configurations
    • Create Config
    • Delete Config
    • Config Schema
  • Jobs
    • Create Job
    • Get Job Details
    • Get Job Status
    • List Jobs
    • Get Job Results
    • Download Detailed Results
    • Get Job Logs
    • Delete Job
    • Job Target & Config Matrix
    • Job Durations
    • Job Schema
  • Live Evaluations
  • API Key Authentication
  • Results
  • Filter and Sort Responses
  • Support Matrix

Audit Model Safety (Beta)

  • About Auditing Models
  • Quickstart
  • Tutorials
    • Run a Simple Job
    • Run with Local NIM
    • Notebook
  • Manage Targets
    • Basic Target
    • NVIDIA Hosted NIM
    • NIM Proxy
    • Local NIM
    • OpenAI
    • List Targets
    • Update and Delete a Target
  • Manage Configs
    • Create Config
    • Get a Config
    • List Configs
    • Update and Delete a Config
    • Probes and Plugins
  • Run and Manage Audit Jobs
  • Viewing Audit Job Results
  • Reference

Deploy NIM and Run Inference

  • About Deploying and Running Inference on NIM
  • Tutorials
    • Deploy NIM
    • Run Inference on NIM
  • Manage NIM Deployments
    • Deploy NIM Microservices
    • Get NIM Deployment Details
    • List Deployments
    • Update Deployment
    • Delete NIM Deployment
    • Create Configuration
    • Get Configuration
    • List Configurations
    • Update Configuration
    • Delete Configuration
  • Run Inference on NIM
    • Health Check
    • List Models
    • Chat Completions
    • Completions
    • Embeddings

Manage Guardrails

  • About Guardrails
  • Terminology
  • Quickstart
  • Tutorials
    • Integrate with NemoGuard NIM
    • Parallel Rails
    • Multimodal Data
    • Injection Detection
    • Custom HTTP Headers
    • Custom LLM Providers
  • Manage Configurations
    • Creating a Configuration
    • Listing Configurations
    • Getting a Configuration
    • Updating a Configuration
    • Deleting a Configuration
    • Configuration Store
  • Manage Access to Models
  • Check a Guardrail
  • Inference with Guardrails
  • Streaming Output
  • Reference

Admin Setup

  • About Admin Setup
  • Helm Installation Overview
  • Helm Installation Options
  • Install as Platform
    • Install
    • Ingress Setup
    • Upgrade
    • Uninstall
  • Install Individually
    • NeMo Auditor
      • Helm Chart
    • NeMo Customizer
    • NeMo Data Store
    • NeMo Deployment Management
    • NeMo Entity Store
    • NeMo Evaluator
      • Helm Chart
      • Chart Config Options
    • NeMo Guardrails
      • Basic Installation
      • Integrate with NIM Deployed in Cluster
      • Integrate with NIM from build.nvidia.com
      • High Availability
      • Configuration Store
      • Custom Dependencies
    • NeMo NIM Proxy
    • NeMo Operator
  • Configure Models
  • Manage GPUs
    • Configure Cluster GPUs
    • Model Configurations Matrix
    • Troubleshooting GPU Jobs
  • Custom Resource Definitions
  • Manage Storage
    • Databases
      • PostgreSQL
      • Milvus
    • PVCs
      • AWS Peristent Volumes
      • Oracle Persistent Volumes
    • Object Storage
      • Amazon S3
    • Backup and Restore
  • Manage Secrets
    • Secrets for Accessing NGC Catalog
    • External Database Secrets
    • JSON Web Token Secrets
    • Object Store Secrets
    • MLFlow Customizer Secrets
    • Weights & Biases Keys
    • Hugging Face API Key Secret
  • Open Telemetry Setup
  • Tenant Configuration Options
  • Security for NeMo Microservices

Reference

  • System Requirements
  • NeMo Microservice API Reference
    • Platform
    • Auditor
    • Customizer
    • Data Designer
    • Deployment Management
    • Entity Store
    • Evaluator
    • Guardrails
    • NIM Proxy
  • NeMo Microservices Python SDK Reference
    • Client APIs
    • Resource APIs
      • Audit Safety
        • Audit Configuration Resource
        • Audit Targets Resource
        • Auditor Jobs Resource
        • Auditor Results Resource
      • Customization
        • Customization Configs Resource
        • Customization Resource
        • Customization Jobs Resource
        • Customization Targets Resource
      • Data Designer
        • Data Designer Resource
        • Data Designer Jobs Resource
        • Data Designer Results Resource
      • Deployment
        • Deployment Resource
        • Deployment Configs Resource
        • Deployment Model Deployments Resource
      • Entity
        • Entity Datasets Resource
        • Entity Models Resource
        • Entity Namespaces Resource
        • Entity Projects Resource
      • Evaluation
        • Evaluation Resource
        • Evaluation Configs Resource
        • Evaluation Jobs Resource
        • Evaluation Results Resource
        • Evaluation Targets Resource
      • Guardrails
        • Guardrail Resource
        • Guardrail Configs Resource
        • Guardrail Models Resource
        • Guardrail Completions Resource
      • Inference
        • Inference Resource
        • Inference Models Resource
        • Inference Chat Resource
        • Inference Chat Completions Resource
        • Inference Completions Resource
        • Inference Embeddings Resource
    • Type APIs
      • Beta Types
        • Audit Types
        • Data Designer Types
        • AuditPlugin
        • AuditTarget
        • AuditRunData
        • Audit Report Data
        • Audit System Data
        • Audit Config Param
        • Audit Plugins Page
        • Audit Target Param
        • Audit Plugin Filter
      • Chat Types
        • ChatCompletionResponse
        • ChatCompletionStreamResponse
        • CompletionCreateParams
      • Customization Types
        • ConfigCreateParams
        • ConfigUpdateParams
        • CustomizationJob
        • DatasetCu
        • DatasetParameters
        • Hyperparameters
        • JobEntry
        • JobCreateParams
        • LoraParameters
        • SftParameters
        • WandBIntegration
      • Deployment Types
        • ConfigCreateParams for Deployment
        • ConfigListParams
        • ConfigUpdateParams for Deployment
        • DeploymentConfigFilterParam
        • DeploymentConfigsPage
        • ModelDeployment
        • ModelDeploymentCreateParams
        • ModelDeploymentFilterParam
        • ModelDeploymentListParams
        • ModelDeploymentStatusDetails
        • ModelDeploymentUpdateParams
        • ModelDeploymentsPage
      • Evaluation Types
        • ConfigCreateParams for Evaluation
        • ConfigListParams for Evaluation
        • ConfigUpdateParams for Evaluation
        • EvaluationConfigsPage
        • EvaluationJob
        • EvaluationJobFilter
        • EvaluationJobFilterParam
        • EvaluationJobsPage
        • EvaluationResultFilter
        • EvaluationResultFilterParam
        • EvaluationResultsPage
        • EvaluationTargetsPage
        • GroupResultParam
        • JobCreateParams for Evaluation
        • JobListParams
        • MetricResultParam
        • ResultCreateParams
        • ResultListParams
        • ResultUpdateParams
        • TargetCreateParams
        • TargetListParams
        • TargetUpdateParams
        • TaskResultParam
      • Guardrail Types
        • CompletionCreateParams
        • ConfigCreateParams
        • ConfigListParams
        • ConfigUpdateParams
        • GuardrailCompletionResponse
        • GuardrailCompletionStreamResponse
        • GuardrailConfigFilterParam
        • GuardrailConfigsPage
        • ModelListParams
        • ModelsResponse
        • ModelsResponseEntry
      • Shared Types
        • ActionRails
        • APIEndpointData
        • APIEndpointFormat
        • ArtifactStatus
        • AutoAlignOptions
        • AutoAlignRailConfig
        • BackendEngineType
        • ChatCompletionAssistantMessageParam
        • ChatCompletionContentPartImageParam
        • ChatCompletionContentPartTextParam
        • ChatCompletionFunctionMessageParam
        • ChatCompletionMessageToolCallParam
        • ChatCompletionMessageToolCall
        • ChatCompletionMessage
        • ChatCompletionResponseChoice
        • ChatCompletionResponseStreamChoice
        • ChatCompletionSystemMessageParam
        • ChatCompletionTokenLogprob
        • ChatCompletionToolMessageParam
        • ChatCompletionUserMessageParam
        • ChoiceDeltaFunctionCall
        • ChoiceDeltaToolCallFunction
        • ChoiceDeltaToolCall
        • ChoiceLogprobs
        • ClavataRailConfig
        • ClavataRailOptions
        • CompletionResponseChoice
        • CompletionResponseStreamChoice
        • ConfigDataParam
        • ConfigData
        • DeleteResponse
        • DeltaMessage
        • DialogRails
        • ErrorResponse
        • FactCheckingRailConfig
        • FiddlerGuardrails
        • FinetuningType
        • FunctionCall
        • Function
        • GenericSortField
        • GuardrailConfigParam
        • GuardrailConfig
        • GuardrailModel
        • HTTPValidationError
        • ImageURL
        • InferenceParams
        • InjectionDetection
        • InputRails
        • Instruction
        • JailbreakDetectionConfig
        • JobStatus
        • LogProbs
        • LoraFinetuningData
        • MessageTemplate
        • ModelArtifact
        • ModelPrecision
        • ModelSpec
        • OutputRailsStreamingConfig
        • OutputRails
        • Ownership
        • PTuningFinetuningData
        • PaginationData
        • ParameterEfficientFinetuningData
        • PatronusEvaluateAPIParams
        • PatronusEvaluateConfigParam
        • PatronusEvaluateConfig
        • PatronusEvaluationSuccessStrategy
        • PatronusRailConfigParam
        • PatronusRailConfig
        • PrivateAIDetectionOptions
        • PrivateAIDetection
        • PromptData
        • RailsConfigDataParam
        • RailsConfigData
        • RailsParam
        • Rails
        • ReasoningModelConfig
        • ReasoningParams
        • RetrievalRails
        • SensitiveDataDetectionOptions
        • SensitiveDataDetection
        • SingleCallConfig
        • TaskPrompt
        • TopLogprob
        • UsageInfo
        • UserMessagesConfig
        • ValidationError
        • VersionTag
      • Shared Params Types
        • ActionRails
        • APIEndpointData
        • APIEndpointFormat
        • ArtifactStatus
        • AutoAlignOptions
        • AutoAlignRailConfig
        • BackendEngineType
        • ChatCompletionAssistantMessageParam
        • ChatCompletionContentPartImageParam
        • ChatCompletionContentPartTextParam
        • ChatCompletionFunctionMessageParam
        • ChatCompletionMessageToolCallParam
        • ChatCompletionSystemMessageParam
        • ChatCompletionToolMessageParam
        • ChatCompletionUserMessageParam
        • ClavataRailConfig
        • ClavataRailOptions
        • ConfigDataParam
        • DialogRails
        • FactCheckingRailConfig
        • FiddlerGuardrails
        • FinetuningType
        • Function
        • FunctionCall
        • GenericSortField
        • GuardrailConfigParam
        • GuardrailModel
        • ImageURL
        • InferenceParams
        • InjectionDetection
        • InputRails
        • Instruction
        • JailbreakDetectionConfig
        • JobStatus
        • LoraFinetuningData
        • MessageTemplate
        • ModelArtifact
        • ModelPrecision
        • ModelSpec
        • OutputRails
        • OutputRailsStreamingConfig
        • Ownership
        • ParameterEfficientFinetuningData
        • PatronusEvaluateAPIParams
        • PatronusEvaluateConfigParam
        • PatronusEvaluationSuccessStrategy
        • PatronusRailConfigParam
        • PTuningFinetuningData
        • PrivateAIDetection
        • PrivateAIDetectionOptions
        • PromptData
        • RailsConfigDataParam
        • RailsParam
        • ReasoningModelConfig
        • ReasoningParams
        • RetrievalRails
        • SensitiveDataDetection
        • SensitiveDataDetectionOptions
        • SingleCallConfig
        • TaskPrompt
        • UserMessagesConfig
        • VersionTag
      • Common Types
        • ActivatedRail
        • ArtifactStatusDe
        • BackendEngineTypeDe
        • BaseModelFilterParam
        • BaseModelFilter
        • CachedOutputsDataParam
        • CachedOutputsData
        • CompletionCreateParams
        • CompletionResponse
        • CompletionStreamResponse
        • CreateEmbeddingResponse
        • CreatedAtFilterParam
        • CreatedAtFilter
        • CustomizationConfigParam
        • CustomizationConfig
        • CustomizationTargetParam
        • CustomizationTarget
        • CustomizationTrainingOptionParam
        • CustomizationTrainingOption
        • DatasetCreateParams
        • DatasetEvParam
        • DatasetEv
        • DatasetFilterParam
        • DatasetFilter
        • DatasetListParams
        • DatasetSortField
        • DatasetUpdateParams
        • Dataset
        • DatasetsPage
        • DateTimeFilterParam
        • DateTimeFilter
        • DeploymentConfigParam
        • DeploymentConfig
        • EmbeddingCreateParams
        • Embedding
        • EvaluationConfigFilterParam
        • EvaluationConfigFilter
        • EvaluationConfigParam
        • EvaluationConfig
        • EvaluationLiveParams
        • EvaluationParamsParam
        • EvaluationParams
        • EvaluationResult
        • EvaluationStatusDetailsParam
        • EvaluationStatusDetails
        • EvaluationTargetFilterParam
        • EvaluationTargetFilter
        • EvaluationTargetParam
        • EvaluationTarget
        • ExecutedAction
        • ExternalEndpointConfigParam
        • ExternalEndpointConfig
        • FinetuningTypeDe
        • GenerationLogOptionsParam
        • GenerationLog
        • GenerationOptionsParam
        • GenerationRailsOptionsParam
        • GenerationStats
        • GroupConfigParam
        • GroupConfig
        • GroupResult
        • GuardrailCheckParams
        • GuardrailCheckResponse
        • GuardrailConfigDeParam
        • GuardrailConfigDe
        • GuardrailsDataParam
        • GuardrailsData
        • LabelSelectorRequirementParam
        • LabelSelectorRequirement
        • LabelSelectorTermParam
        • LabelSelectorTerm
        • LiveEvaluation
        • LlmCallInfo
        • MetricConfigParam
        • MetricConfig
        • MetricResult
        • ModelArtifactDeParam
        • ModelArtifactDe
        • ModelCreateParams
        • ModelDeParam
        • ModelDe
        • ModelEvParam
        • ModelEv
        • ModelFilterParam
        • ModelFilter
        • ModelListParams
        • ModelParam
        • ModelPeftFilterParam
        • ModelPeftFilter
        • ModelPrecisionDe
        • ModelSortField
        • ModelSpecDeParam
        • ModelSpecDe
        • ModelUpdateParams
        • Model
        • ModelsPage
        • NamespaceCreateParams
        • NamespaceListParams
        • NamespaceUpdateParams
        • Namespace
        • NamespacesPage
        • NIMDeploymentConfigParam
        • NIMDeploymentConfig
        • NodeAffinityParam
        • NodeAffinity
        • NodeSelectorParam
        • NodeSelectorTermParam
        • NodeSelectorTerm
        • NodeSelector
        • ParameterEfficientFinetuningDataDeParam
        • ParameterEfficientFinetuningDataDe
        • PreferredSchedulingTermParam
        • PreferredSchedulingTerm
        • ProjectCreateParams
        • ProjectFilterParam
        • ProjectFilter
        • ProjectListParams
        • ProjectSortField
        • ProjectUpdateParams
        • Project
        • ProjectsPage
        • PromptDataDeParam
        • PromptDataDe
        • RagPipelineDataParam
        • RagPipelineData
        • RagTargetParam
        • RagTarget
        • RailStatus
        • RetrieverPipelineDataParam
        • RetrieverPipelineData
        • RetrieverTargetParam
        • RetrieverTarget
        • ScoreParam
        • ScoreStatsParam
        • ScoreStats
        • Score
        • StatusEnum
        • TargetStatus
        • TargetType
        • TaskConfigParam
        • TaskConfig
        • TaskResult
        • TaskStatus
        • TolerationParam
        • Toleration
        • TrainingPodSpecParam
        • TrainingPodSpec
        • TrainingType
  • NeMo Microservices Helm Chart
  • Troubleshooting
    • Troubleshoot Auditor
    • Troubleshoot Customizer
    • Troubleshoot Evaluator
    • Troubleshoot Guardrails
    • Troubleshoot Data Designer
    • Troubleshoot Setup
  • NVIDIA Distribution in Llama Stack
  • EULA

Resources

  • OSS License Acknowledgements
  • NeMo Microservices Python SDK Reference
  • Types APIs
  • Beta Types
  • Audit Types

Audit Types#

  • AuditConfig
  • AuditJobHandle
  • AuditJobStatus
  • JobCreateParams
  • JobListResponse
  • ConfigCreateParams
  • ConfigListResponse
  • ConfigUpdateParams
  • TargetCreateParams
  • TargetListResponse
  • TargetUpdateParams
  • AuditPluginsDataOutput
  • ConfigListVersionsParams
  • TargetListVersionsParams

previous

Beta Types

next

AuditConfig
NVIDIA NVIDIA
Privacy Policy | Manage My Privacy | Do Not Sell or Share My Data | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2024-2025, NVIDIA Corporation.

Last updated on Sep 10, 2025.