Evaluation Techniques#
Follow step-by-step guides for different evaluation scenarios and methodologies in NeMo Evaluator.
Before You Start#
Ensure you have:
Completed the initial getting started guides for Installation Guide and Quickstart.
Have your endpoint and API key ready or prepared for the checkpoint you wish to deploy.
Prepared your Hugging Face token for accessing gated datasets.
Evaluations#
Select an evaluation type tailored to your model capabilities.
Text Generation
Measure model performance through natural language generation for academic benchmarks, reasoning tasks, and general knowledge assessment.
Log-Probability
Assess model confidence and uncertainty using log-probabilities for multiple-choice scenarios without text generation.