Benchmarks#

Learn how to run, build, and integrate benchmarks into NeMo Gym.

Tip

Looking for benchmark design tutorials? See the Building Environments.

Run a benchmark or benchmark suite

Tutorial to run a benchmark or benchmark suite using NeMo Gym.

Run a benchmark or benchmark suite
Add a benchmark to NeMo Gym

Best practices for benchmark integration into NeMo Gym.

Adding a benchmark to NeMo Gym
Design a customer evaluation

Best practices for design and iteration on a customer evaluation benchmark.

Designing a customer evaluation benchmark
Add aggregate metrics for your benchmark

Implementation tutorial for aggregate metrics.

Aggregate Metrics