Build Voice Agents

Build Voice Agents

View as Markdown

Start with configuration when the shipped pipeline already supports the behavior you need. Add tools or custom pipeline code only when configuration is not enough.

Choose an Area

Choose the smallest surface that supports the behavior you want to add.

AreaUse It to
ConfigureSelect component configs, prompts, model-registry entries, and audio logging.
Model servingServe with vLLM, configure vLLM plugins, or use NVIDIA NIM and Riva.
ToolsEnable tool calling or write direct and component-owned tools.
ExtendAdd processors, assemble a custom pipeline, or integrate through RTVI.

Use this order to move from a working example to the smallest maintainable customization.

  1. Confirm the current behavior in the Quickstart.
  2. Make the smallest configuration change that satisfies the requirement.
  3. Add a tool for a discrete action or a processor for a new pipeline transformation.
  4. Build a custom pipeline only when the shipped assembly cannot express the design.

Key Concepts

The extension surface you choose determines where the change belongs and how much pipeline behavior you own:

  • Configuration selects existing components and behavior without changing Python code.
  • Tools let supported language models invoke application functions during a conversation.
  • Processors transform frames at a focused point in the shipped pipeline.
  • Custom pipelines replace the shipped assembly when stage ordering or ownership must change.