Build Voice AgentsConfigure

Audio Capture and Logging

View as Markdown

NeMo Labs Voice Agent writes two independent outputs. Per-session audio and transcript capture is off by default. The pipeline also emits a rotating text log at runtime. This page explains how to enable audio capture and interpret both outputs.

Enable Audio Capture

Audio capture is controlled by two keys in the transport: block of the top-level server config (examples/generic_voice_agent/server/server_configs/default.yaml, and the same block in default_nvidia.yaml):

1transport:
2  audio_out_10ms_chunks: 8
3  record_audio_data: true          # default: false
4  audio_log_dir: "./audio_logs"    # default: "./audio_logs"
KeyDefaultEffect
transport.record_audio_datafalseEnables capture. When false, no capture code runs.
transport.audio_log_dir"./audio_logs"Base directory, resolved relative to the server’s working directory.

build_audio_logger in nemo_voice_agent/pipecat/services/nemo/builders.py reads those two keys. When record_audio_data is false it returns None. Every downstream service—speech-to-text (STT), text-to-speech (TTS), turn-taking, and diarization—is then constructed with audio_logger=None, so capture adds no runtime work.

The session ID is generated at server start, not per client connection: session_YYYYMMDD_HHMMSS. Because the server keeps one pipeline alive across reconnects, as described in Server Config, a client that disconnects and reconnects keeps appending to the same session directory.

audio_logs/, audio_logs_user/, and audio_logs_agent/ are gitignored.

What Lands on Disk

Each capture session produces per-speaker audio and metadata plus a whole-session stereo mixdown.

audio_logs/
└── session_20260806_141200/
    ├── user/
    │   ├── 00001_141207.wav
    │   ├── 00001_141207.json
    │   └── ...
    ├── agent/
    │   ├── 00001_141203.wav
    │   ├── 00001_141203.json
    │   └── ...
    ├── session_metadata.json
    └── conversation_stereo.wav
ArtifactContents
user/NNNNN_HHMMSS.wavOne user turn, mono 16-bit at 16 kHz, sliced out of the continuous input buffer by the turn’s start/end times.
user/NNNNN_HHMMSS.jsonMetadata and the final transcription for that turn.
agent/NNNNN_HHMMSS.wavOne synthesized TTS segment (mono 16-bit) at the TTS service’s output sample rate. A single agent turn is usually several segments.
agent/NNNNN_HHMMSS.jsonMetadata and the text that was synthesized.
session_metadata.jsonRolling index of every entry; rewritten after each save and again at finalize.
conversation_stereo.wavWhole-session mixdown, left channel = agent, right channel = user, 16 kHz. Written only at finalize.

The NNNNN prefix is a per-speaker counter (the user and agent counters are separate), and HHMMSS is wall-clock time at save. User turns get a 0.8-second pre-roll prepended, clamped so a turn never overlaps the previous entry’s end time.

Metadata Fields

Both sides share base_name, counter, turn_index, speaker, timestamp (ISO 8601), start_time / end_time (float seconds from the first audio frame of the session), audio_file, sample_rate, num_channels, and audio_duration_sec. Beyond that:

FieldSideMeaning
transcriptionuserFinal automatic speech recognition (ASR) text for the turn.
is_backchanneluserTrue when turn-taking classified the utterance as a backchannel, so it did not interrupt the bot. Refer to Turn Taking.
num_audio_chunks, num_transcription_chunksuserHow many streaming chunks were merged into this turn.
model, backenduserASR model name and backend that produced the transcription.
textagentThe text handed to TTS for this segment.
modelagentTTS model name.
cutoff_timeagentnull when the segment played to completion; a float (seconds from session start) when the user interrupted. Interruption stamps the same value on every segment of the current turn and zeroes the agent channel of the stereo mix after that point.

session_metadata.json holds session_id, start_time, last_updated, a flat user_entries list, and an agent_entries list where each element is itself the list of segments for one agent turn. finalize_session adds end_time, total_user_entries, total_agent_segments, and total_agent_turns.

How Capture Is Wired

AudioLogger (nemo_voice_agent/pipecat/services/nemo/audio_logger.py) is a plain object passed into the services by the builders. Each service pushes data into it at the right moment.

SourceWhat It Contributes
STT serviceStamps the session’s first-audio timestamp and appends every input chunk to the continuous user buffer without voice activity detection (VAD) gating. It also stages the turn’s audio and transcription. The user channel therefore includes silence.
Turn-taking serviceMarks backchannel turns, records when the bot begins speaking, sets cutoff_time on interruption, and advances the turn index for user turns.
TTS serviceCalls log_agent_audio one time per synthesized segment and advances the turn index for agent turns.
RTVIAudioLoggerObserverA pipeline observer that flushes the staged user turn to disk when a TranscriptionFrame is pushed. It is added to the task’s observer list unconditionally and no-ops when the logger is None.
run_bot_websocket_serverCalls finalize_session on client disconnect, on session timeout, and on pipeline shutdown — this is what writes conversation_stereo.wav.

Two limitations worth knowing:

  • build_audio_logger passes only the directory, session ID, and enabled flag. AudioLogger’s other constructor arguments (user sample rate, pre-roll seconds, rounding precision) are not exposed through YAML, so their defaults always apply. Change them by constructing AudioLogger yourself. Refer to Builders.
  • The AudioLogger docstring records a known issue with conversation_stereo.wav: the two channels need roughly a -0.8 s offset applied to sound in sync. The per-turn WAV files are unaffected.

Quick check after a session:

$ls audio_logs/session_*/
$python -m json.tool audio_logs/session_*/session_metadata.json | head -40

Server Log File

Logging is configured by setup_logging in nemo_voice_agent/utils/misc.py, which installs a colorized stderr sink plus a file sink with rotation="1 day" at DEBUG level. Each day, loguru rolls the active file aside into a timestamped sibling, so when debugging a failure check the newest bot_server.*.log and not only bot_server.log.

setup_rotating_log wraps that with a rename-or-delete step for a pre-existing file: it either removes the old log or renames it to bot_server.<YYYYmmdd_HHMMSS>.log before installing the sinks.

KeyDefaultRead by
server.log_file"bot_server.log"evaluation/bot_server.py
server.log_level"DEBUG"evaluation/bot_server.py
server.create_new_logfalseevaluation/bot_server.py (roll the existing log aside at startup)
server.overwrite_existing_logfalseevaluation/bot_server.py (delete instead of rename)

Gotcha: the example server (examples/generic_voice_agent/server/server.py) calls setup_logging() with no arguments, so it always writes bot_server.log at DEBUG regardless of the server.log_file / server.log_level values in default.yaml. Those keys take effect for the evaluation bot servers, which resolve them through resolve_log_file_path and setup_rotating_log. The call is repeated after service construction because model libraries reconfigure loguru during import.

Evaluation Runs

The evaluation role configs (evaluation/server_configs/agent.yaml and user.yaml) also ship with record_audio_data: false, and give each role its own audio_log_dir (./audio_logs_agent, ./audio_logs_user) so the two bots do not collide. Independently of this, the bridge writes conversation_log.wav (stereo), conversation_log.txt, and conversation_log.seglst.json into each scenario’s output directory. For output details, refer to Evaluation Results.

Use these pages to continue configuring or inspecting the runtime: