Evaluation Tutorials | NVIDIA NeMo Helix

Use these tutorials to become familiar with evaluation with NeMo Helix.

Before You Start

Set up a local instance of the platform for the following tutorials.

Learn how to evaluate a fine-tuned model using the LLM Judge metric with a custom dataset.

Learn how to write a domain-specific Python metric, test it locally, and run it through the Evaluator service.

For the conceptual overview of how Evaluator separates definition (library) from execution (platform), see About Evaluating → How evaluation runs. For runnable SDK examples, see SDK Resources.