Create and Manage Evaluation Targets#
When you run an evaluation in NVIDIA NeMo Evaluator, you create a separate target and configuration for the evaluation.
Tip
Because NeMo Evaluator separates the target and the configuration, you can create a target once, and reuse it multiple times with different configurations (for example, to make a model scorecard). To see what targets and configurations are supported together, refer to Job Target and Configuration Matrix.
NeMo Evaluator provides evaluation capabilities for the following different target types:
Data Sources
LLM Models
Retriever Pipelines
RAG Pipelines
Task Guides#
Perform common evaluation target tasks.
Create and submit a new evaluation target
Delete an existing evaluation target
References#
Review detailed specifications for configuring evaluation targets, including data sources, LLM models, retrieval pipelines, and RAG integrations.
Configure evaluation targets using direct data input through rows or datasets for quick evaluations and testing
Set up evaluation targets for LLM models, including NIM endpoints, chat endpoints, and offline pre-generated outputs
Configure retriever pipeline targets using embedding models and optional reranking for document retrieval
Set up RAG pipeline targets combining retrieval and generation capabilities for comprehensive evaluations
Reference documentation for the JSON schema used to define evaluation targets