ReferenceFull Library ReferenceNemo Voice AgentNemo Voice AgentEvaluationScenariosData

nemo_voice_agent.evaluation.scenarios.data.qa

View as Markdown

Module Contents

Classes

NameDescription
QABaseScenarioBase class for QA evaluation scenarios.
QACapitalFranceQA scenario asking about the capital of France.
QALargestOceanQA scenario asking about the largest ocean.
QAMathAdditionQA scenario asking a basic arithmetic question.
QAMathWordProblemQA scenario with a math word problem.
QAMoonLandingQA scenario asking about the first moon landing.
QARomeoAndJulietQA scenario asking who wrote Romeo and Juliet.
QASpeedOfLightQA scenario asking about the speed of light.
QATelephoneInventorQA scenario asking who invented the telephone.
QAWeatherNewYorkQA scenario asking about the weather in New York using GetCityWeatherTool.
QAWeatherSanFranciscoQA scenario asking about the weather in San Francisco using GetCityWeatherTool.

API

class nemo_voice_agent.evaluation.scenarios.data.qa.QABaseScenario()

Bases: Scenario

Base class for QA evaluation scenarios. Provides sensible defaults for all 8 required properties. Not registered as an eval scenario itself.

_user_background
str
_user_name
str = 'Alex'
_user_personality
str
_user_question
str = ''
agent_actions
Actions
agent_persona
Persona
agent_resources
Resources
agent_task
Task
max_duration
= 60
success_signals
user_actions
Actions
user_persona
Persona
user_resources
Resources
user_task
Task
class nemo_voice_agent.evaluation.scenarios.data.qa.QACapitalFrance()

Bases: QABaseScenario

QA scenario asking about the capital of France.

_user_background
_user_name
= 'John'
_user_personality
_user_question
= 'What is the capital of France?'
description
= 'QA scenario: What is the capital of France?'
name
= 'qa__capital_france'
reference_answer
class nemo_voice_agent.evaluation.scenarios.data.qa.QALargestOcean()

Bases: QABaseScenario

QA scenario asking about the largest ocean.

_user_background
_user_name
= 'Yuki'
_user_personality
_user_question
= 'What is the largest ocean on Earth?'
description
= 'QA scenario: What is the largest ocean on Earth?'
name
= 'qa__largest_ocean'
reference_answer
class nemo_voice_agent.evaluation.scenarios.data.qa.QAMathAddition()

Bases: QABaseScenario

QA scenario asking a basic arithmetic question.

_user_background
= 'You are a student who is practicing basic math.'
_user_name
= 'Maria'
_user_personality
_user_question
= 'What is 15 plus 27?'
description
= 'QA scenario: What is 15 plus 27?'
name
= 'qa__math_addition'
reference_answer
class nemo_voice_agent.evaluation.scenarios.data.qa.QAMathWordProblem()

Bases: QABaseScenario

QA scenario with a math word problem.

_user_background
_user_name
= 'Tom'
_user_personality
_user_question
description
name
= 'qa__math_word_problem'
reference_answer
class nemo_voice_agent.evaluation.scenarios.data.qa.QAMoonLanding()

Bases: QABaseScenario

QA scenario asking about the first moon landing.

_user_background
_user_name
= 'Sarah'
_user_personality
_user_question
= 'When was the first moon landing?'
description
= 'QA scenario: When was the first moon landing?'
name
= 'qa__moon_landing'
reference_answer
class nemo_voice_agent.evaluation.scenarios.data.qa.QARomeoAndJuliet()

Bases: QABaseScenario

QA scenario asking who wrote Romeo and Juliet.

_user_background
_user_name
= 'Emily'
_user_personality
_user_question
= 'Who wrote Romeo and Juliet?'
description
= 'QA scenario: Who wrote Romeo and Juliet?'
name
= 'qa__romeo_and_juliet'
reference_answer
class nemo_voice_agent.evaluation.scenarios.data.qa.QASpeedOfLight()

Bases: QABaseScenario

QA scenario asking about the speed of light.

_user_background
_user_name
= 'David'
_user_personality
_user_question
= 'What is the speed of light?'
description
= 'QA scenario: What is the speed of light?'
name
= 'qa__speed_of_light'
reference_answer
class nemo_voice_agent.evaluation.scenarios.data.qa.QATelephoneInventor()

Bases: QABaseScenario

QA scenario asking who invented the telephone.

_user_background
_user_name
= 'Kevin'
_user_personality
_user_question
= 'Who invented the telephone?'
description
= 'QA scenario: Who invented the telephone?'
name
= 'qa__telephone_inventor'
reference_answer
class nemo_voice_agent.evaluation.scenarios.data.qa.QAWeatherNewYork()

Bases: QABaseScenario

QA scenario asking about the weather in New York using GetCityWeatherTool.

_user_background
_user_name
= 'Priya'
_user_personality
_user_question
= 'What is the weather in New York?'
agent_resources
Resources
description
= 'QA scenario: What is the weather in New York?'
name
= 'qa__weather_new_york'
reference_answer
class nemo_voice_agent.evaluation.scenarios.data.qa.QAWeatherSanFrancisco()

Bases: QABaseScenario

QA scenario asking about the weather in San Francisco using GetCityWeatherTool.

_user_background
_user_name
= 'Carlos'
_user_personality
_user_question
= 'What is the weather in San Francisco?'
agent_resources
Resources
description
name
= 'qa__weather_san_francisco'
reference_answer