ReferenceFull Library ReferenceNemo Voice AgentNemo Voice AgentPipecatServicesNemo

nemo_voice_agent.pipecat.services.nemo.diar

View as Markdown

Module Contents

Classes

NameDescription
NeMoDiarInputParamsStreaming diarization parameters for the NeMo diarization service.
NemoDiarServicePipecat STT service wrapping NeMo’s streaming diarization model.

API

class nemo_voice_agent.pipecat.services.nemo.diar.NeMoDiarInputParams()

Bases: BaseModel

Streaming diarization parameters for the NeMo diarization service.

buffer_size
Optional[int] = 30
config_path
Optional[str] = None
frame_len_in_secs
Optional[float] = 0.08
language
Optional[Language] = Language.EN_US
raw_audio_frame_len_in_secs
Optional[float] = 0.016
threshold
Optional[float] = 0.4
class nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService(
    model: typing.Optional[str] = '',
    device: typing.Optional[str] = 'cuda:0',
    sample_rate: typing.Optional[int] = 16000,
    params: typing.Optional[nemo_voice_agent.pipecat.services.nemo.diar.NeMoDiarInputParams] = None,
    use_vad: bool = True,
    audio_passthrough: bool = True,
    backend: typing.Optional[str] = 'legacy',
    enabled: bool = True,
    kwargs = {}
)

Bases: STTService

Pipecat STT service wrapping NeMo’s streaming diarization model.

_audio_buffer
= []
_queue
= asyncio.Queue()
_response_queue
= asyncio.Queue()
nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService._diarization_processor()

Background processor that handles diarization calls.

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService._get_dominant_speaker_id(
    spk_pred: numpy.ndarray
)
nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService._handle_diarization_result(
    diar_result
)
async

Handle diarization result from background processing.

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService._handle_transcription(
    transcript: str,
    is_final: bool,
    language: typing.Optional[str] = None
)
async

Handle a transcription result.

Parameters:

transcript
str

The transcribed text

is_final
bool

Whether this is a final transcription

language
Optional[str]" default="None

The language of the transcription

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService._load_model()
nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService._processing_task_handler()
async

Handler for background processing task.

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService._response_task_handler()
async

Handler for processing diarization results.

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService._stop_tasks()
async

Stop background processing tasks.

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService.can_generate_metrics() -> bool

Only report initial metrics, no need to spam metrics every 80ms

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService.cancel(
    frame: pipecat.frames.frames.CancelFrame
)
async

Handle service cancellation.

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService.process_frame(
    frame: pipecat.frames.frames.Frame,
    direction: pipecat.processors.frame_processor.FrameDirection
)
async

Process audio data and generate transcription frames.

Parameters:

audio

Raw audio bytes to transcribe

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService.reset()

Reset the diarization service.

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService.run_stt(
    audio: bytes
) -> typing.AsyncGenerator[pipecat.frames.frames.Frame, None]
async

Process audio data and generate transcription frames.

Parameters:

audio
bytes

Raw audio bytes to transcribe

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService.service_metadata_frame()

Broadcast nothing: this service diarizes, it does not transcribe.

STTService would otherwise publish an STTMetadataFrame carrying a time-to-final-transcript figure, which is meaningless here. It would also be actively harmful: SpeechTimeoutUserTurnStopStrategy keeps the last value it sees, and this service sits downstream of the real STT in the pipeline, so its placeholder would override the STT’s measured latency and skew end-of-turn timing.

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService.set_language(
    language: pipecat.transcriptions.language.Language
)
async

Update the service’s recognition language.

Parameters:

language
Language

New language for recognition

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService.set_model(
    model: str
)
async

Update the service’s model.

Parameters:

model
str

New model name/path to use

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService.start(
    frame: pipecat.frames.frames.StartFrame
)
async

Handle service start.

nemo_voice_agent.pipecat.services.nemo.diar.NemoDiarService.stop(
    frame: pipecat.frames.frames.EndFrame
)
async

Handle service stop.