Configuration Model

NeMo Labs Voice Agent is configured by YAML. One top-level server config selects the models and pipeline behavior. The speech-to-text (STT), large language model (LLM), and text-to-speech (TTS) components can also load a per-model sub-config. ConfigManager (nemo_voice_agent/utils/config_manager.py) merges these files before the example server builds any pipeline stage.

Where the Files Live

The demo server and evaluation harness keep their top-level configuration in separate directories.

examples/generic_voice_agent/server/
├── model_registry.yaml          # model id -> sub-config filename
└── server_configs/
    ├── default.yaml             # the shipped top-level config
    ├── default_nvidia.yaml      # top-level config for hosted NIM endpoints
    ├── stt_configs/             # per-ASR-model sub-configs
    ├── llm_configs/             # per-LLM sub-configs
    └── tts_configs/             # per-TTS-model sub-configs

ConfigManager takes a server base path — the directory holding server_configs/ and model_registry.yaml — plus an optional path to the top-level file. The example server passes its own directory as the base path and reads the file path from the SERVER_CONFIG_PATH environment variable, falling back to <base>/server_configs/default.yaml. A relative SERVER_CONFIG_PATH is resolved against the current working directory, so cd first:

$
cd examples/generic_voice_agent/server
$
SERVER_CONFIG_PATH=server_configs/default_nvidia.yaml python server.py

The Three Layers

ConfigManager combines three configuration layers before a builder reads its component block:

Layers 2 and 3 are alternatives, not additions: model_config: short-circuits the registry lookup entirely.

The Sub-Config Wins

For STT, LLM, and TTS, the merge loop copies every sub-config key over the corresponding key in the top-level block. The sub-config overrides the top-level file.

The shipped default demonstrates it. default.yaml declares llm.type: auto, but llm_configs/nemotron_nano_v3.yaml (selected by llm.model_config:) declares type: vllm, so the effective backend is vllm. ConfigManager logs every such replacement:

LLM config field `type` is overridden from `auto` to `vllm` by .../llm_configs/nemotron_nano_v3.yaml

Practical consequence: editing a key in default.yaml that the sub-config also sets has no effect. Change it in the sub-config, or point model_config: at a copy you own. Keys the sub-config does not mention (model, enable_reasoning, function_call_timeout_secs, system_prompt, …) keep their top-level values. device is not one of them — nemotron_nano_v3.yaml sets device: "cuda" itself, so a device you edit in default.yaml is silently replaced by the sub-config’s value.

The merge is a shallow, per-key replacement within the block. If a sub-config defines a nested mapping such as vllm_generation_params, the whole mapping replaces the top-level one rather than merging into it.

Only the Filename of model_config: Matters

ConfigManager takes os.path.basename() of model_config: and looks the file up in <base>/server_configs/<component>_configs/. The directory portion of the shipped values (./server_configs/llm_configs/nemotron_nano_v3.yaml) is decorative — a sub-config must physically live in the component’s own directory next to the top-level file. A missing file raises FileNotFoundError at startup.

Registry Auto-Resolution

When a block has no model_config: and server.use_model_registry is true, ConfigManager looks model: up in model_registry.yaml and uses the entry’s yaml_id as the sub-config filename. Refer to Model Registry for the file’s structure. Two follow-on behaviors apply:

• Reasoning swap. If the model was resolved through the registry, its entry sets reasoning_supported: true, and llm.enable_reasoning: true, the loader substitutes the sibling file whose name ends in _think.yaml. Because an explicit model_config: bypasses the registry, the swap does not fire for the shipped default — point model_config: at the _think.yaml by hand. Refer to Reasoning.
• No match. If the model is in neither model_config: nor the registry, no sub-config is loaded, and the system logs a warning. The top-level block must then be complete on its own. Nothing is silently substituted.

OmegaConf Interpolation

Configs use OmegaConf, so a value can reference another key. Resolution happens in two distinct phases, and the difference matters:

• Top-level file: resolved eagerly at load. ConfigManager calls OmegaConf.to_container(..., resolve=True) before any sub-config is merged. An interpolation here can only reference keys present in the same file. Referencing one that only a sub-config supplies raises InterpolationKeyError at startup.
• Sub-config: resolved lazily against the merged root. Sub-config values are copied over verbatim and resolved on access, against the final merged server config. That is why llm_configs/nemotron_nano_v3.yaml can write the following and get 0.6 and 1024 — the values its own file contributed to llm.temperature and llm.max_new_tokens:

1
vllm_generation_params:
2
  temperature: ${llm.temperature}
3
  top_p: ${llm.top_p}
4
  max_completion_tokens: ${llm.max_new_tokens}

Paths in interpolations are absolute from the config root (llm.temperature), not relative to the sub-config.

System Prompt

llm.system_prompt is path-or-literal: ConfigManager runs os.path.isfile() on the value and reads the file when it exists, otherwise treats the string as the prompt itself. Relative paths resolve against the current working directory, not the server base path. Reusable prompts ship in examples/generic_voice_agent/server/example_prompts/.

llm.system_prompt_suffix is appended to whichever prompt was chosen, separated by a newline. The shipped default combines a literal prompt from default.yaml with a tool-usage suffix from the LLM sub-config. Omit system_prompt entirely and a short built-in default is used. Details in Prompts.

Which Block Feeds Which Component

Every stage is constructed by a builder in nemo_voice_agent/pipecat/services/nemo/builders.py. Each builder reads one block of the merged config:

Note on the server block: server.log_file, server.log_level, server.create_new_log, server.overwrite_existing_log, and server.talk_first are honored by the evaluation bot server (evaluation/bot_server.py). The example server in examples/generic_voice_agent/server/server.py hardcodes talk-first and calls logging setup with its defaults, writing bot_server.log at DEBUG.

llm.type: auto is resolved at service-construction time, not by ConfigManager: the LLM factory probes whether vLLM can load the model and falls back to the Hugging Face backend if not. Refer to LLM Backends.

Inspecting the Merged Result

The startup log is the source of truth. ConfigManager emits Final STT config:, Final LLM config:, and Final TTS config: lines after each merge. It also emits one ... is overridden from ... by ... line per replaced key. The example server then logs the fully resolved config as Server config:. Check those lines whenever a setting appears to be ignored — the override log names the file that won.

Gotchas

Keep these merge and path behaviors in mind when a configuration edit does not take effect:

• The shipped llm_configs/nemotron_nano_v3.yaml sets start_vllm_on_init: false, so python server.py alone does not work by itself. Start vLLM first, or change that key. Refer to vLLM Backend.
• turn_taking.backchannel_phrases_path is tried against the working directory first, then against the server base path, and raises FileNotFoundError naming both if neither exists. An inline list or null is also accepted — null lets any speech interrupt the bot.
• Only one client can be connected at a time. A second connection is rejected with WebSocket close code 1013, and the incumbent is kept. No config key changes this.

Next

Continue with the configuration guide that matches the setting you need to change:

• Server Config Reference — block-by-block walkthrough of the top-level file.
• Model Registry — adding a model so it auto-resolves.
• Audio Logging — the transport.record_audio_data path.
• Config Schema — full key listing.