Benchmarks

Learn how to run, build, and integrate benchmarks into NeMo Gym.

Tip

Looking for benchmark design tutorials? See the Building Environments.

Run a benchmark or benchmark suite

Tutorial to run a benchmark or benchmark suite using NeMo Gym.

benchmark

Run a benchmark or benchmark suite

Add a benchmark to NeMo Gym

Best practices for benchmark integration into NeMo Gym.

benchmark

Adding a benchmark to NeMo Gym

Design a customer evaluation

Best practices for design and iteration on a customer evaluation benchmark.

benchmark

Designing a customer evaluation benchmark

Add aggregate metrics for your benchmark

Implementation tutorial for aggregate metrics.

benchmark

Aggregate Metrics