Automatic Speech Recognition (Latest)

NVIDIA Docs Hub NVIDIA NIM Automatic Speech Recognition (Latest) nim/riva/asr/latest/asr-performance-table.html

Automatic Speech Recognition (Latest)

nim/riva/asr/latest/asr-performance-table.html

Chunk size (ms): 160
Language model: n-gram
Maximum effective # of streams with n-gram language model: 104

# of streams	Latency (ms)					Throughput (RTFX)
	avg	p50	p90	p95	p99
1	18	16.7	17.4	20	40	0.999
8	21	19.8	21	30	50.4	7.99
16	29	26	40	41	70	15.96
32	41	45	51	55	110	31.9
48	53	58	71	73	160	47.75
64	65.5	72	83	85	210	63.6

Chunk size (ms): 960
Language model: n-gram
Maximum effective # of streams with n-gram language model: 490

# of streams	Latency (ms)					Throughput (RTFX)
	avg	p50	p90	p95	p99
1	20	18.4	20	36	100	0.999
64	80	83	95	140	200	63.7
128	119	100	155	230	296	126.8
256	190	170	270	400	530	252
384	260	245	378	580	800	374.5
512	346	330	496	850	1200	494

Language model: n-gram

# of streams	Throughput (RTFX)
1	200
32	2000

Chunk size (ms): 160
Language model: n-gram
Maximum effective # of streams with n-gram language model: 104

# of streams	Latency (ms)					Throughput (RTFX)
	avg	p50	p90	p95	p99
1	24	22.7	23.7	25	50	0.999
8	32.7	31	33	51	72.7	7.98
16	44	40.8	50	63	110	15.94
32	59	60	73	75	180	31.8
48	79	90	93	100	240	47.6
64	100	109	114	160	310	63.4

Chunk size (ms): 960
Language model: n-gram
Maximum effective # of streams with n-gram language model: 490

# of streams	Latency (ms)					Throughput (RTFX)
	avg	p50	p90	p95	p99
1	30	29.1	40	50	100	0.999
64	123	130	160	240	260	63.5
128	185	165	240	360	430	126.4
256	300	266	430	630	830	249.4
384	460	445	770	1100	1560	368
512	720	650	1400	1550	2150	483

Language model: n-gram

# of streams	Throughput (RTFX)
1	180
32	1330

Chunk size (ms): 160
Language model: n-gram
Maximum effective # of streams with n-gram language model: 104

# of streams	Latency (ms)					Throughput (RTFX)
	avg	p50	p90	p95	p99
1	25	23	29	30	50	0.999
8	31	29	35.5	46	70	7.98
16	44	40	56	60	100	15.95
32	60	62	76	80	150	31.84
48	80	86	100	112	227	47.7

Chunk size (ms): 960
Language model: n-gram
Maximum effective # of streams with n-gram language model: 490

# of streams	Latency (ms)					Throughput (RTFX)
	avg	p50	p90	p95	p99
1	30	27	40	50	100	0.999
64	120	130	150	200	240	63.5
128	170	150	220	310	380	126.5
256	270	250	390	540	700	250.5

Language model: n-gram

# of streams	Throughput (RTFX)
1	180
32	1440

© Copyright © 2024, NVIDIA Corporation. Last updated on Aug 6, 2024.