Performance
Contents
Performance#
Evaluation Process#
This section shows the latency and throughput numbers for streaming and offline configurations of the Riva ASR service on different GPUs. These numbers were captured after the preconfigured ASR pipelines from our Quick Start scripts were deployed. The Jasper, QuartzNet, Conformer, and Citrinet-1024 acoustic models were tested.
In streaming mode, the client and the server used audio chunks of the same duration (100ms, 160ms, and 800ms depending on the server configuration). Refer to the Results section for the chunk size value to use.
The Riva streaming client riva_streaming_asr_client
,
provided in the Riva image, was used with the --simulate_realtime
flag to
simulate transcription from a microphone, where each stream was doing three iterations
over a sample audio file (1272-135031-0000.wav
) from the LibriSpeech dev-clean dataset.
The Librispeech datasets can be obtained from https://www.openslr.org/12.
The source code for the riva_streaming_asr_client
can be obtained from https://github.com/nvidia-riva/cpp-clients.
The command used to measure performance was:
riva_streaming_asr_client \
--chunk_duration_ms=<chunk_duration> \
--simulate_realtime=true \
--automatic_punctuation=true \
--num_parallel_requests=<num_streams> \
--word_time_offsets=true \
--print_transcripts=false \
--interim_results=false \
--num_iterations=<3*num_streams> \
--audio_file=1272-135031-0000.wav \
--output_filename=/tmp/output.json
The riva_streaming_asr_client
returns the following latency measurements:
intermediate latency
: latency of responses returned withis_final == false
final latency
: latency of responses returned withis_final == true
latency
: the overall latency of all returned responses. This is what is tabulated in the following tables.
Refer to the following diagram for a schematic representation of the different latencies measured by the Riva streaming ASR client.

In offline mode, the command used to measure maximum throughput was:
riva_asr_client \
--automatic_punctuation=true \
--num_parallel_requests=32 \
--word_time_offsets=true \
--print_transcripts=false \
--num_iterations=96 \
--audio_file=1272-135031-0000x5.wav \
--output_filename=/tmp/output.json
where 1272-135031-0000x5.wav
is simply the 1272-135031-0000.wav
audio file concatenated five times.
The source code for the riva_asr_client
can be obtained from: https://github.com/nvidia-riva/cpp-clients
Results#
Latencies and throughput measurements for streaming and offline configurations are reported in the following tables. Throughput is measured in RTFX (duration of audio transcribed / computation time).
Note
Audio files were iterated 1 time for Xavier AGX, Xavier NX, and Orin AGX and 3 times for all other experiments.
Note
If the language model is none
, the inference is performed with a greedy decoder. If the language model is n-gram
,
then a beam decoder was used.
Note
The values in the tables are average values over 3 trials. The values in the table are rounded to the last significant digit according to standard deviation calculated on 3 trials. If a standard deviation is less than 0.001 of the average, then the corresponding value is rounded as if standard deviation equals 0.001 of the value.
For specifications of the hardware on which these measurements were collected, refer to the Hardware Specifications section. Please notice, that
results on AWS and GCP are computed using Riva 2.4.0
results On-Prem on GPUs A40, T4, L40, H100 on Riva 2.9.0.
other On-Prem results on Riva 2.8.0
Cloud instance descriptions for AWS and GCP.
160
Language model |
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
|||
n-gram |
1 |
13.9 |
13.57 |
13.9 |
14.2 |
28 |
1 |
n-gram |
8 |
26.2 |
25.5 |
26 |
27 |
60 |
7.98 |
n-gram |
16 |
37 |
35 |
45 |
45.4 |
90 |
15.93 |
n-gram |
32 |
52 |
56 |
63 |
64 |
140 |
31.8 |
n-gram |
48 |
62 |
65 |
70 |
72 |
150 |
47.6 |
n-gram |
64 |
76 |
76 |
81 |
83 |
230 |
63.3 |
none |
1 |
13.1 |
12.8 |
12.96 |
13.2 |
22 |
1 |
none |
8 |
25 |
24.5 |
25 |
25 |
60 |
7.98 |
none |
16 |
34 |
30 |
44 |
44 |
80 |
15.94 |
none |
32 |
50 |
50 |
61 |
63 |
120 |
31.8 |
none |
48 |
56 |
62 |
66 |
70 |
140 |
47.6 |
none |
64 |
70 |
73 |
76 |
77 |
190 |
63.3 |
800
Language model |
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
|||
n-gram |
1 |
16 |
15.4 |
16.3 |
24 |
30 |
1 |
n-gram |
64 |
97 |
96 |
107 |
170 |
300 |
63.3 |
n-gram |
128 |
140 |
130 |
168 |
270 |
530 |
125.7 |
n-gram |
256 |
230 |
220 |
270 |
530 |
1170 |
247 |
n-gram |
384 |
330 |
330 |
380 |
740 |
2000 |
365 |
n-gram |
512 |
470 |
440 |
500 |
1240 |
3060 |
478 |
none |
1 |
13 |
12.67 |
13 |
17.1 |
18 |
1 |
none |
64 |
86 |
88 |
93 |
160 |
290 |
63.4 |
none |
128 |
120 |
100 |
140 |
250 |
500 |
125.9 |
none |
256 |
200 |
200 |
240 |
470 |
1090 |
247.7 |
none |
384 |
300 |
304 |
340 |
690 |
1700 |
365.6 |
none |
512 |
420 |
410 |
460 |
1040 |
2870 |
480 |
Speaker Diarization |
Language model |
# of streams |
Throughput (RTFX) |
---|---|---|---|
False |
n-gram |
32 |
1500 |
False |
none |
32 |
1600 |
True |
n-gram |
32 |
120 |
True |
none |
32 |
120 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.8 |
13.7 |
14.3 |
14.6 |
20 |
1 |
8 |
26.6 |
26.3 |
27.7 |
30 |
40 |
7.98 |
16 |
37 |
36 |
46 |
47 |
50 |
15.95 |
32 |
57 |
63 |
70 |
70 |
73 |
31.84 |
48 |
67 |
73 |
80 |
80 |
90 |
47.7 |
64 |
80 |
87 |
94 |
95 |
100 |
63.6 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15 |
14.5 |
15.6 |
20 |
20 |
0.999 |
64 |
95 |
103 |
111 |
114 |
117 |
63.4 |
128 |
150 |
150 |
200 |
200 |
200 |
126.2 |
256 |
220 |
235 |
270 |
300 |
300 |
250.5 |
384 |
300 |
340 |
400 |
410 |
440 |
372 |
512 |
400 |
470 |
540 |
560 |
650 |
492 |
768 |
660 |
730 |
1000 |
1000 |
1200 |
720 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1500 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.5 |
13.3 |
13.8 |
14 |
20 |
1 |
8 |
25.7 |
25 |
25.7 |
27 |
52 |
8 |
16 |
40 |
30 |
44 |
45 |
80 |
15.97 |
32 |
50 |
50 |
62 |
63 |
127 |
31.95 |
48 |
61.6 |
65 |
70 |
74 |
183 |
47.9 |
64 |
74.6 |
75 |
81 |
87.7 |
270 |
63.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15 |
14 |
20 |
21 |
20 |
1 |
64 |
93 |
94 |
110 |
175 |
300 |
63.8 |
128 |
140 |
130 |
160 |
300 |
530 |
127.4 |
256 |
228 |
210 |
270 |
585 |
1190 |
253.5 |
384 |
390 |
320 |
780 |
1300 |
2100 |
371 |
512 |
620 |
430 |
1720 |
2440 |
3400 |
483 |
768 |
1500 |
660 |
4400 |
7300 |
10000 |
682 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1000 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.9 |
13.9 |
14.3 |
14.4 |
17 |
1 |
8 |
27.5 |
27.3 |
28.3 |
30 |
40 |
7.99 |
16 |
40 |
37 |
40 |
50 |
50 |
15.98 |
32 |
57 |
64 |
70 |
71 |
74 |
31.9 |
48 |
70 |
76 |
81.4 |
83 |
90 |
47.9 |
64 |
86.6 |
93 |
99.4 |
101 |
110 |
63.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15 |
14.8 |
16 |
17 |
20 |
1 |
64 |
90 |
100 |
110 |
115 |
140 |
63.7 |
128 |
140 |
135 |
180 |
200 |
270 |
127 |
256 |
230 |
230 |
330 |
400 |
430 |
252 |
384 |
340 |
340 |
646 |
700 |
800 |
374 |
512 |
460 |
470 |
800 |
970 |
1200 |
494 |
768 |
920 |
740 |
1900 |
2300 |
3000 |
727 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1900 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.5 |
13.37 |
13.7 |
13.8 |
20 |
1 |
8 |
27 |
25 |
30 |
30 |
30 |
7.99 |
16 |
34 |
30 |
44 |
44.5 |
50 |
15.97 |
32 |
50.3 |
57 |
62 |
63 |
69 |
31.9 |
48 |
59 |
65 |
68 |
70 |
80 |
47.9 |
64 |
70 |
76 |
80 |
81 |
90 |
63.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15 |
14.3 |
15 |
16 |
20 |
1 |
64 |
85 |
93 |
103 |
108 |
120 |
63.8 |
128 |
130 |
120 |
160 |
200 |
230 |
127.2 |
256 |
197 |
200 |
260 |
400 |
500 |
252.3 |
384 |
285 |
310 |
400 |
650 |
670 |
375.5 |
512 |
390 |
420 |
600 |
860 |
1000 |
496 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
2800 |
160
none
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15.1 |
14.9 |
15.4 |
16 |
20 |
0.999 |
8 |
34 |
34 |
40 |
40 |
56 |
7.97 |
16 |
48 |
48 |
60 |
60 |
90 |
15.9 |
32 |
73 |
77 |
90 |
90 |
150 |
31.6 |
48 |
92 |
100 |
106 |
110 |
200 |
47.25 |
64 |
110 |
115 |
125 |
140 |
270 |
62.8 |
800
none
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
18.3 |
17.6 |
20 |
20 |
20 |
0.999 |
64 |
130 |
124 |
160 |
240 |
300 |
62.7 |
128 |
220 |
200 |
300 |
500 |
700 |
123 |
256 |
400 |
370 |
700 |
1000 |
1300 |
237 |
384 |
680 |
556 |
1350 |
2000 |
2600 |
337 |
512 |
2000 |
2000 |
4000 |
6000 |
8000 |
400 |
none
# of streams |
Throughput (RTFX) |
---|---|
32 |
870 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.6 |
13.48 |
14 |
14.5 |
20 |
1 |
8 |
25.5 |
25.4 |
26.2 |
26.7 |
34 |
7.99 |
16 |
35 |
32 |
44.4 |
45 |
48 |
15.97 |
32 |
50 |
56 |
63 |
63 |
70 |
31.9 |
48 |
58 |
64 |
69 |
70 |
97 |
47.8 |
64 |
70 |
74 |
79 |
82 |
118 |
63.7 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15 |
14.5 |
15 |
16 |
20 |
1 |
64 |
85 |
92 |
100 |
100 |
110 |
63.7 |
128 |
110 |
110 |
140 |
150 |
170 |
127.2 |
256 |
172 |
184 |
237 |
243 |
255 |
253.4 |
384 |
246 |
295 |
341 |
350 |
380 |
378.6 |
512 |
340 |
410 |
460 |
494 |
540 |
502 |
768 |
540 |
608 |
800 |
900 |
1150 |
740 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
2000 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.5 |
13.13 |
13.7 |
17 |
20 |
1 |
8 |
26 |
24.7 |
26 |
32 |
57 |
7.98 |
16 |
34 |
30 |
44 |
46 |
94.5 |
15.93 |
32 |
50 |
50 |
62 |
80 |
160 |
31.8 |
48 |
63 |
63 |
70 |
82 |
243 |
47.6 |
64 |
78 |
75 |
83 |
128 |
300 |
63.3 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15 |
14 |
20 |
20 |
20 |
1 |
64 |
108 |
86 |
230 |
284 |
323 |
63.3 |
128 |
170 |
120 |
400 |
500 |
570 |
125.6 |
256 |
310 |
223 |
720 |
1080 |
1250 |
247 |
384 |
530 |
340 |
1350 |
1640 |
2150 |
364.4 |
512 |
1200 |
600 |
3000 |
3600 |
4300 |
468 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1000 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9.9 |
9.7 |
10 |
10.15 |
16.7 |
1 |
8 |
14.5 |
14 |
15 |
15.5 |
34 |
7.99 |
16 |
23 |
23 |
25 |
25.8 |
60 |
15.95 |
32 |
35 |
34 |
36 |
37 |
100 |
31.8 |
48 |
46 |
44 |
50 |
50 |
140 |
47.6 |
64 |
55 |
53 |
58 |
60 |
200 |
63.3 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11 |
10.2 |
11 |
16 |
20 |
1 |
64 |
64 |
56 |
60 |
135 |
260 |
63.4 |
128 |
108 |
94 |
110 |
240 |
470 |
126 |
256 |
172 |
148 |
174 |
400 |
960 |
248.3 |
384 |
240 |
207 |
250 |
585 |
1430 |
367 |
512 |
320 |
270 |
320 |
800 |
2100 |
482 |
768 |
484 |
400 |
480 |
1400 |
3900 |
703 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
4000 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9.53 |
9.53 |
9.8 |
9.9 |
13 |
1 |
8 |
13.7 |
13.6 |
14.4 |
15 |
19 |
8 |
16 |
20 |
20 |
24 |
25 |
27 |
16 |
32 |
34.5 |
34.6 |
36 |
37 |
41 |
31.96 |
48 |
43 |
43 |
47 |
48 |
51 |
47.9 |
64 |
54 |
54 |
60 |
60 |
68 |
63.9 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10.3 |
10.3 |
11 |
11.5 |
20 |
1 |
64 |
55 |
55 |
62 |
65 |
70 |
63.9 |
128 |
96 |
96 |
108 |
112 |
120 |
127.5 |
256 |
150 |
160 |
190 |
200 |
220 |
254.4 |
384 |
190 |
203 |
240 |
240 |
260 |
381 |
512 |
240 |
265 |
306 |
316 |
340 |
506 |
768 |
350 |
397 |
460 |
480 |
520 |
755 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1000 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10 |
9.9 |
10.3 |
11 |
14 |
1 |
8 |
16 |
16 |
17 |
20 |
23 |
8 |
16 |
24 |
25 |
30 |
29.5 |
32 |
15.98 |
32 |
42.5 |
42.4 |
44.5 |
46 |
50 |
31.94 |
48 |
53 |
53 |
60 |
63 |
66 |
47.9 |
64 |
60 |
60 |
66 |
70 |
74 |
63.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11 |
10.7 |
11.4 |
12 |
20 |
1 |
64 |
69 |
70 |
80 |
80 |
86 |
63.8 |
128 |
100 |
104 |
140 |
150 |
200 |
127.2 |
256 |
176 |
184 |
260 |
290 |
330 |
253 |
384 |
260 |
260 |
400 |
460 |
600 |
377 |
512 |
310 |
334 |
500 |
610 |
645 |
499 |
768 |
500 |
500 |
800 |
1060 |
1400 |
737 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1000 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.2 |
11.07 |
11.3 |
11.4 |
15 |
1 |
8 |
20 |
20 |
20.8 |
21 |
30 |
8 |
16 |
30 |
30 |
37 |
38 |
45 |
15.98 |
32 |
52 |
53 |
55 |
56 |
64 |
31.9 |
48 |
64 |
64 |
70 |
71.7 |
75 |
47.9 |
64 |
70 |
72 |
78 |
79 |
85 |
63.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.4 |
12.2 |
13 |
13.5 |
20 |
1 |
64 |
88 |
90 |
97 |
100 |
110 |
63.8 |
128 |
140 |
130 |
170 |
200 |
240 |
127 |
256 |
210 |
220 |
270 |
423 |
460 |
252 |
384 |
290 |
314 |
380 |
620 |
670 |
375.6 |
512 |
382 |
410 |
530 |
900 |
960 |
496 |
768 |
670 |
626 |
1350 |
1700 |
2200 |
733 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1000 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.2 |
11.7 |
17 |
29 |
35.6 |
1 |
8 |
26 |
23 |
30 |
56 |
80 |
8 |
16 |
38 |
37 |
45 |
70 |
94 |
15.97 |
32 |
67 |
65 |
74 |
90 |
114 |
31.9 |
48 |
87 |
85 |
100.8 |
120 |
160 |
47.8 |
64 |
100 |
97 |
110 |
130 |
180 |
63.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
20.5 |
17.7 |
34 |
35 |
40 |
1 |
64 |
116 |
110 |
160 |
170 |
200 |
63.8 |
128 |
190 |
180 |
260 |
270 |
300 |
127 |
256 |
300 |
310 |
400 |
420 |
600 |
252.5 |
384 |
460 |
470 |
700 |
750 |
900 |
374 |
512 |
600 |
610 |
900 |
1100 |
1200 |
492 |
768 |
1300 |
1300 |
2000 |
2300 |
3000 |
725 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
600 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.1 |
11.05 |
11.46 |
11.85 |
20 |
1 |
8 |
19.2 |
19 |
20.1 |
20.4 |
25 |
8 |
16 |
30 |
30 |
38 |
38 |
43 |
15.97 |
32 |
50 |
50 |
55 |
57 |
70 |
31.9 |
48 |
63 |
61.6 |
70 |
73 |
90 |
47.85 |
64 |
70 |
70 |
78 |
80 |
100 |
63.7 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12 |
11.9 |
12.5 |
12.7 |
20 |
1 |
64 |
86 |
88 |
97 |
100 |
105 |
63.8 |
128 |
120 |
120 |
150 |
150 |
160 |
127.3 |
256 |
184 |
210 |
236 |
240 |
260 |
253.4 |
384 |
255 |
300 |
340 |
344 |
356 |
379 |
512 |
343 |
400 |
460 |
510 |
600 |
502 |
768 |
570 |
620 |
900 |
1000 |
1400 |
740 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
900 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.14 |
12.85 |
13.8 |
15 |
21 |
1 |
8 |
15.2 |
14.6 |
16.4 |
18.1 |
37 |
7.99 |
16 |
20 |
20 |
25 |
27 |
59 |
15.95 |
32 |
34 |
33.5 |
39 |
42 |
98 |
31.84 |
48 |
43 |
41 |
48 |
52 |
165 |
47.7 |
64 |
50 |
46 |
56 |
60 |
235 |
63.5 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
17.5 |
17 |
22 |
25 |
27 |
1 |
64 |
72 |
57 |
130 |
210 |
280 |
63.4 |
128 |
123 |
96 |
230 |
380 |
500 |
126 |
256 |
200 |
140 |
400 |
700 |
1040 |
248 |
384 |
279 |
186 |
640 |
1140 |
1500 |
367.4 |
512 |
390 |
240 |
1110 |
1500 |
2180 |
483 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
2000 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
7.1 |
6.8 |
8 |
9 |
16 |
1 |
8 |
10 |
9.2 |
11 |
12 |
37 |
7.99 |
16 |
14 |
13 |
16 |
18.5 |
60 |
15.96 |
32 |
18 |
16 |
20 |
24 |
100 |
31.85 |
48 |
23 |
19 |
26 |
30 |
154 |
47.7 |
64 |
28 |
23 |
30 |
37 |
196 |
63.5 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12 |
11.75 |
16.5 |
17 |
20 |
1 |
64 |
55 |
40 |
110 |
197 |
263 |
63.4 |
128 |
85 |
56 |
190 |
350 |
470 |
126 |
256 |
146 |
86 |
350 |
656 |
940 |
248.6 |
384 |
207 |
110 |
550 |
993 |
1360 |
368 |
512 |
280 |
146 |
840 |
1320 |
1900 |
484 |
768 |
500 |
200 |
1600 |
2330 |
3200 |
700 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
2000 |
160
Language model |
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
|||
n-gram |
1 |
12.8 |
12.6 |
13 |
13 |
24.8 |
1 |
n-gram |
8 |
26 |
25.3 |
26.2 |
28 |
53 |
7.98 |
n-gram |
16 |
37 |
35 |
43 |
44 |
87 |
15.94 |
n-gram |
32 |
60 |
64 |
68.3 |
70 |
160 |
31.8 |
n-gram |
48 |
92.5 |
96 |
99.4 |
101 |
270 |
47.5 |
none |
1 |
11.9 |
11.7 |
11.9 |
12 |
18.2 |
1 |
none |
8 |
26 |
25.3 |
30 |
30 |
46 |
7.98 |
none |
16 |
34 |
32 |
40 |
41 |
74 |
15.94 |
none |
32 |
55.4 |
61 |
65 |
66 |
140 |
31.8 |
none |
48 |
84 |
90 |
93 |
94.2 |
240 |
47.6 |
none |
64 |
107 |
111 |
115 |
117 |
320 |
63.2 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15 |
14.7 |
15.5 |
22 |
23 |
1 |
64 |
130 |
126 |
140 |
200 |
320 |
63.3 |
128 |
203 |
200 |
225 |
330 |
560 |
125.6 |
256 |
354 |
360 |
420 |
640 |
1400 |
246.6 |
n-gram
Speaker Diarization |
Language model |
# of streams |
Throughput (RTFX) |
---|---|---|---|
False |
n-gram |
32 |
1200 |
False |
none |
32 |
1200 |
True |
n-gram |
32 |
100 |
True |
none |
32 |
100 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13 |
13 |
13.6 |
14 |
16 |
0.999 |
8 |
27 |
26 |
30 |
30 |
34 |
7.98 |
16 |
36 |
36 |
43 |
44 |
46 |
15.95 |
32 |
59 |
65 |
72 |
73 |
76 |
31.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13 |
12.7 |
13.4 |
13.6 |
20.7 |
1 |
8 |
30 |
30 |
33 |
34 |
46 |
7.99 |
16 |
42 |
40 |
50 |
51 |
66 |
15.97 |
32 |
74 |
81 |
84 |
86 |
118 |
31.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.2 |
12 |
12.7 |
13 |
15 |
0.998 |
8 |
25.1 |
24.8 |
26 |
30 |
33 |
7.97 |
16 |
33 |
32 |
40.4 |
41.4 |
43 |
15.94 |
32 |
57.4 |
64 |
67 |
68 |
70 |
31.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.4 |
13.5 |
14 |
14.1 |
16 |
1 |
8 |
27 |
26.5 |
28.4 |
30 |
33 |
7.99 |
16 |
36 |
35.6 |
40 |
44 |
46 |
15.97 |
32 |
60 |
67 |
74 |
75 |
80 |
31.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.1 |
12.1 |
12.3 |
12.5 |
15 |
1 |
8 |
26 |
25.8 |
30 |
30 |
30 |
7.99 |
16 |
36 |
34.4 |
42.5 |
43 |
46 |
15.97 |
32 |
60 |
67 |
70 |
71 |
80 |
31.9 |
48 |
88 |
96 |
100 |
101 |
110 |
47.8 |
160
none
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.8 |
14.6 |
15 |
15.2 |
20 |
0.999 |
8 |
34 |
35 |
38 |
40 |
50 |
7.97 |
16 |
50 |
50 |
60 |
60 |
80 |
15.9 |
32 |
80 |
87 |
100 |
100 |
140 |
31.6 |
48 |
120 |
130 |
140 |
150 |
250 |
47.2 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.2 |
12.2 |
12.7 |
13 |
15 |
1 |
8 |
27 |
27 |
30 |
30 |
32 |
7.99 |
16 |
34 |
33 |
41 |
42 |
43 |
15.97 |
32 |
56 |
62 |
66 |
67 |
74 |
31.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
19.2 |
16.5 |
27 |
35.3 |
45 |
0.997 |
8 |
45 |
42 |
60 |
70 |
95 |
7.95 |
16 |
76 |
75 |
90 |
100 |
126 |
15.85 |
32 |
130 |
130 |
155 |
170 |
190 |
31.55 |
48 |
3000 |
3000 |
5000 |
6000 |
7000 |
40 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
32 |
35.6 |
62 |
62 |
62 |
0.997 |
64 |
293 |
290 |
380 |
400 |
400 |
62.4 |
128 |
600 |
530 |
800 |
900 |
1000 |
120 |
256 |
5000 |
5000 |
7000 |
8000 |
9000 |
200 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
650 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.1 |
12 |
12.6 |
12.8 |
15.6 |
1 |
8 |
25.7 |
25 |
30 |
30 |
32 |
7.99 |
16 |
35 |
35 |
41 |
41.3 |
45 |
15.97 |
32 |
54 |
60 |
64 |
65 |
70 |
31.9 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13 |
12.9 |
14 |
15 |
16 |
1 |
64 |
110 |
120 |
134 |
136 |
150 |
63.7 |
128 |
190 |
200 |
220 |
230 |
250 |
126.8 |
256 |
330 |
370 |
410 |
420 |
440 |
252 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1400 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.63 |
13.9 |
17.5 |
19.6 |
19.84 |
0.997 |
8 |
32 |
30 |
38 |
43 |
45 |
7.94 |
16 |
47.8 |
48.8 |
60 |
80 |
83 |
15.8 |
32 |
87 |
85 |
160 |
170 |
172 |
31.2 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15 |
14 |
18 |
18 |
18 |
0.997 |
64 |
150 |
160 |
179 |
190 |
190 |
62.3 |
128 |
260 |
280 |
320 |
330 |
330 |
122 |
256 |
470 |
520 |
580 |
600 |
600 |
235 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1000 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14 |
13.7 |
16.7 |
17 |
18 |
0.996 |
8 |
29 |
28.8 |
32 |
34 |
37 |
7.94 |
16 |
42 |
40 |
50 |
54 |
60 |
15.8 |
32 |
80 |
80 |
130 |
140 |
160 |
31 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.2 |
12 |
12.7 |
14 |
15 |
0.998 |
8 |
25.3 |
25 |
26 |
28 |
30 |
7.98 |
16 |
35.6 |
34.4 |
41.8 |
42.5 |
46 |
15.93 |
32 |
56 |
63 |
67 |
68 |
74 |
31.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.1 |
11.9 |
12.4 |
15 |
18 |
1 |
8 |
26 |
25 |
26 |
31 |
52 |
7.98 |
16 |
35.7 |
33 |
41.6 |
44 |
82 |
15.94 |
32 |
56 |
60 |
64.4 |
80 |
154 |
31.8 |
48 |
90 |
90 |
95 |
123 |
297 |
47.5 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
28 |
27.3 |
28 |
33.5 |
52 |
0.998 |
4 |
53 |
51 |
52 |
62 |
108 |
3.986 |
8 |
104 |
100 |
102 |
124 |
240 |
7.94 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
29.7 |
29.2 |
30 |
37 |
40 |
0.999 |
4 |
55 |
53.2 |
60 |
70 |
75 |
3.99 |
8 |
104 |
101.6 |
110 |
125 |
140 |
7.97 |
n-gram
Speaker Diarization |
# of streams |
Throughput (RTFX) |
---|---|---|
False |
1 |
142 |
False |
4 |
190 |
False |
8 |
195 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10 |
9.8 |
10.1 |
10.3 |
15.6 |
1 |
8 |
17.44 |
17 |
18.5 |
19 |
37 |
7.99 |
16 |
28 |
27 |
30 |
30 |
60 |
15.95 |
32 |
42 |
40 |
44 |
46 |
108 |
31.8 |
48 |
53 |
50 |
60 |
60 |
150 |
47.6 |
64 |
69 |
68 |
73 |
75 |
210 |
63.4 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9.84 |
9.9 |
10.22 |
10.29 |
12 |
1 |
8 |
16 |
16 |
17 |
18 |
20 |
8 |
16 |
26.2 |
26.5 |
29 |
29 |
30 |
15.98 |
32 |
40 |
40 |
45 |
47 |
50 |
31.95 |
48 |
52 |
52 |
56 |
58 |
60 |
47.9 |
64 |
63 |
63 |
68.6 |
70 |
74 |
63.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10.6 |
10.6 |
11 |
11.4 |
13 |
0.999 |
8 |
22 |
22 |
23 |
24 |
26 |
7.98 |
16 |
36 |
40 |
40 |
40 |
42 |
15.95 |
32 |
60 |
60 |
62 |
63 |
66 |
31.8 |
48 |
82 |
82 |
85.3 |
86.6 |
100 |
47.6 |
64 |
103 |
101 |
110 |
120 |
180 |
63.2 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.3 |
11.3 |
11.7 |
11.8 |
14 |
1 |
8 |
20.4 |
20 |
22 |
23 |
25 |
8 |
16 |
31.6 |
32.7 |
34 |
35 |
37 |
15.98 |
32 |
50 |
50 |
53 |
54.5 |
57 |
31.94 |
48 |
66 |
70 |
78 |
80 |
83 |
47.9 |
64 |
81 |
90 |
95 |
97 |
100 |
63.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10.6 |
10.6 |
10.8 |
10.9 |
12.8 |
1 |
8 |
20.5 |
20.3 |
21 |
22 |
25 |
8 |
16 |
30 |
30 |
36 |
36 |
37 |
15.98 |
32 |
55.5 |
56.1 |
57.5 |
58 |
62 |
31.9 |
48 |
76 |
78 |
80.6 |
82 |
86.7 |
47.85 |
64 |
97 |
98 |
102 |
104 |
120 |
63.7 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14 |
12.7 |
18 |
30 |
37 |
1 |
8 |
30 |
26 |
35 |
50 |
75 |
7.99 |
16 |
38 |
36 |
50 |
64 |
89 |
15.97 |
32 |
65 |
62 |
80 |
90 |
120 |
31.9 |
48 |
79 |
77 |
90 |
104 |
144 |
47.85 |
64 |
100 |
100 |
120 |
140 |
170 |
63.7 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11 |
11 |
11.2 |
11.4 |
12 |
1 |
8 |
23 |
22.6 |
25 |
25 |
27 |
7.99 |
16 |
35 |
35 |
39 |
40 |
41 |
15.98 |
32 |
59 |
59 |
61 |
62 |
72 |
31.9 |
48 |
83 |
83 |
90 |
90 |
130 |
47.8 |
64 |
104 |
104 |
110 |
110 |
170 |
63.6 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11 |
10.8 |
11.5 |
12 |
13 |
0.997 |
8 |
22 |
21 |
24 |
25 |
26 |
7.95 |
16 |
40 |
40 |
50 |
50 |
60 |
15.83 |
32 |
74 |
72 |
90 |
100 |
100 |
31.3 |
48 |
105 |
98 |
147 |
150 |
160 |
46.4 |
64 |
141 |
126 |
210 |
250 |
280 |
61.1 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.1 |
11 |
11.7 |
12 |
14 |
0.999 |
8 |
23 |
22.5 |
25 |
26 |
29 |
7.98 |
16 |
35 |
35 |
39 |
39.6 |
44 |
15.93 |
32 |
58 |
59 |
61 |
63 |
67 |
31.8 |
48 |
80 |
81 |
84 |
86 |
90 |
47.6 |
64 |
100 |
100 |
105 |
106 |
120 |
63.3 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11 |
10.7 |
11.22 |
15.6 |
16.4 |
1 |
8 |
24 |
23 |
25 |
30 |
48 |
7.99 |
16 |
36 |
34 |
38.5 |
50 |
80 |
15.95 |
32 |
61 |
57 |
60.4 |
90 |
150 |
31.8 |
48 |
87 |
79.7 |
86 |
130 |
278 |
47.55 |
64 |
115 |
100.3 |
122.4 |
264 |
447 |
63.2 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13 |
12.6 |
13.6 |
14.6 |
21 |
1 |
8 |
20 |
20 |
22 |
24 |
40 |
7.99 |
16 |
27 |
27 |
30 |
32 |
56 |
15.96 |
32 |
46 |
45 |
50 |
53 |
96 |
31.85 |
48 |
61 |
58 |
64 |
67 |
180 |
47.7 |
64 |
78 |
72 |
80 |
84 |
330 |
63.5 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
7.6 |
7.2 |
8.3 |
9 |
17 |
1 |
8 |
11 |
10 |
12 |
13 |
36 |
7.99 |
16 |
14 |
13 |
15 |
17 |
50 |
15.96 |
32 |
20 |
18 |
22 |
24 |
90 |
31.9 |
48 |
24 |
22 |
27 |
30 |
130 |
47.7 |
64 |
32 |
28 |
35 |
38 |
170 |
63.5 |
160
Language model |
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
|||
n-gram |
1 |
18.93 |
18.52 |
18.83 |
19 |
39 |
0.999 |
n-gram |
8 |
40 |
37 |
50 |
52.8 |
87 |
7.97 |
n-gram |
16 |
53 |
50 |
66 |
67 |
130 |
15.9 |
n-gram |
32 |
66 |
69 |
72.3 |
74 |
170 |
31.7 |
n-gram |
48 |
91 |
94 |
100 |
103.6 |
280 |
47.4 |
none |
1 |
18 |
17.6 |
17.7 |
17.85 |
33.6 |
0.999 |
none |
8 |
38 |
36 |
40 |
40 |
82 |
7.97 |
none |
16 |
48 |
44 |
62 |
64 |
120 |
15.9 |
none |
32 |
60.5 |
66 |
69.4 |
70 |
160 |
31.73 |
none |
48 |
84.7 |
89 |
93 |
96 |
280 |
47.4 |
none |
64 |
108 |
109 |
117 |
122 |
390 |
63 |
800
Language model |
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
|||
n-gram |
1 |
21 |
20.4 |
21 |
31.3 |
32 |
0.999 |
n-gram |
64 |
140 |
130 |
170 |
220 |
400 |
63.1 |
n-gram |
128 |
210 |
200 |
230 |
390 |
740 |
124.8 |
n-gram |
256 |
374 |
360 |
420 |
770 |
2100 |
243 |
none |
1 |
18.2 |
17.6 |
18 |
25 |
26 |
1 |
none |
64 |
120 |
110 |
150 |
230 |
400 |
63.1 |
none |
128 |
190 |
190 |
210 |
378 |
720 |
124.9 |
none |
256 |
336 |
310 |
390 |
740 |
2060 |
244 |
none |
384 |
550 |
500 |
580 |
1490 |
4640 |
357 |
Speaker Diarization |
Language model |
# of streams |
Throughput (RTFX) |
---|---|---|---|
False |
n-gram |
32 |
1020 |
False |
none |
32 |
1000 |
True |
n-gram |
32 |
100 |
True |
none |
32 |
100 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
19.13 |
19.14 |
19.7 |
19.9 |
22 |
0.999 |
8 |
40 |
40 |
40 |
50 |
53 |
7.97 |
16 |
54 |
53 |
67 |
70 |
72 |
15.93 |
32 |
66 |
73.4 |
80 |
80 |
83 |
31.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
20 |
20 |
21 |
22 |
22 |
0.998 |
64 |
120 |
120 |
140 |
150 |
160 |
63.3 |
128 |
200 |
210 |
230 |
240 |
250 |
125.5 |
256 |
340 |
370 |
410 |
420 |
500 |
248 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1000 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
18 |
17.8 |
18.3 |
18.5 |
27 |
1 |
8 |
37 |
35.8 |
37.3 |
45 |
80 |
7.99 |
16 |
50 |
50 |
64 |
66 |
109 |
15.97 |
32 |
64 |
68 |
71 |
80 |
180 |
31.9 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
19.5 |
19 |
20 |
28.7 |
29 |
1 |
64 |
130 |
120 |
166 |
240 |
400 |
63.7 |
128 |
200 |
195 |
220 |
400 |
730 |
127 |
256 |
434 |
350 |
850 |
1500 |
2200 |
247.6 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
640 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
19.1 |
19.1 |
19.6 |
19.7 |
21.4 |
1 |
8 |
40 |
40 |
40 |
40 |
50 |
7.99 |
16 |
53 |
50 |
64 |
67 |
70 |
15.96 |
32 |
72 |
78 |
82 |
83 |
90 |
31.9 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
19.5 |
19.7 |
21 |
22 |
22 |
1 |
64 |
134 |
129 |
174 |
200 |
200 |
63.6 |
128 |
215 |
210 |
300 |
350 |
430 |
126.3 |
256 |
367 |
372 |
500 |
680 |
700 |
249 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1300 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
18.3 |
18.3 |
18.6 |
18.7 |
22 |
1 |
8 |
38 |
37.3 |
40 |
50 |
53 |
7.99 |
16 |
51 |
50 |
65 |
66 |
68 |
15.96 |
32 |
60 |
68.4 |
70.7 |
71.4 |
80 |
31.9 |
48 |
88 |
94 |
98 |
99 |
113 |
47.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
19 |
19.1 |
20 |
20.3 |
20 |
1 |
64 |
120 |
120 |
150 |
200 |
200 |
63.6 |
128 |
177 |
170 |
220 |
300 |
330 |
126.6 |
256 |
360 |
374 |
500 |
740 |
800 |
249.6 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1770 |
160
none
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
21.1 |
20.9 |
21.2 |
21.7 |
30 |
0.998 |
8 |
49 |
49 |
60 |
61 |
85 |
7.95 |
16 |
60 |
60 |
76 |
80 |
120 |
15.85 |
32 |
78 |
82 |
90 |
100 |
170 |
31.55 |
48 |
114 |
118 |
130 |
140 |
300 |
47 |
800
none
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
24 |
23.1 |
28.6 |
29 |
30 |
0.998 |
64 |
160 |
147 |
220 |
320 |
400 |
62.2 |
128 |
270 |
250 |
400 |
600 |
780 |
121.3 |
256 |
600 |
480 |
1000 |
1600 |
2300 |
229 |
none
# of streams |
Throughput (RTFX) |
---|---|
32 |
560 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
18.2 |
18.2 |
18.7 |
18.9 |
20.7 |
1 |
8 |
35.6 |
35.4 |
37 |
37 |
50 |
7.99 |
16 |
51 |
50 |
64.3 |
65.7 |
70 |
15.96 |
32 |
63 |
67.6 |
70 |
72 |
94 |
31.9 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
19.2 |
19.3 |
20 |
21 |
23 |
1 |
64 |
120 |
120 |
150 |
160 |
170 |
63.6 |
128 |
174 |
180 |
213 |
220 |
250 |
127 |
256 |
295 |
333 |
395 |
406 |
450 |
252 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1400 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
18.4 |
17.8 |
18.3 |
27 |
30.5 |
0.999 |
8 |
40 |
37 |
50 |
50 |
100 |
7.97 |
16 |
48 |
46 |
62.6 |
66 |
147 |
15.9 |
32 |
66.4 |
68.4 |
72 |
100 |
253 |
31.7 |
48 |
97.4 |
92 |
102 |
156 |
440 |
47.4 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
21 |
19 |
28.3 |
28.7 |
30 |
1 |
64 |
160 |
123 |
320 |
390 |
460 |
63.1 |
128 |
250 |
200 |
550 |
700 |
800 |
124.6 |
256 |
605 |
380 |
1480 |
2060 |
2320 |
242.6 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
660 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.84 |
13.54 |
13.9 |
14 |
26.3 |
1 |
8 |
23 |
23 |
24 |
30 |
57 |
7.98 |
16 |
40.5 |
41 |
42.7 |
43.5 |
102 |
15.9 |
32 |
60 |
60 |
65 |
67 |
170 |
31.7 |
48 |
64 |
60 |
68 |
70 |
210 |
47.5 |
64 |
80 |
80 |
88 |
90 |
280 |
63.1 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15 |
14 |
15 |
22.7 |
24 |
1 |
64 |
106 |
96 |
110 |
207 |
380 |
63.2 |
128 |
150 |
135 |
160 |
300 |
650 |
125.1 |
256 |
274 |
250 |
280 |
610 |
1470 |
244.5 |
384 |
420 |
370 |
400 |
1060 |
2850 |
359 |
512 |
620 |
500 |
600 |
1550 |
4700 |
467 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
2500 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.4 |
13.4 |
13.7 |
14 |
15.2 |
1 |
8 |
24 |
23 |
30 |
30 |
31 |
8 |
16 |
39 |
40 |
42 |
42 |
46 |
15.98 |
32 |
60 |
60 |
65 |
67 |
70 |
31.93 |
48 |
61.3 |
61.5 |
68 |
70 |
75 |
47.9 |
64 |
80 |
80 |
87 |
89 |
97 |
63.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.3 |
14.4 |
15 |
15.6 |
17 |
1 |
64 |
90 |
100 |
104 |
110 |
110 |
63.8 |
128 |
120 |
130 |
150 |
160 |
170 |
127.3 |
256 |
220 |
246 |
270 |
280 |
290 |
253.5 |
384 |
320 |
360 |
420 |
440 |
470 |
378.6 |
512 |
420 |
480 |
550 |
600 |
700 |
501 |
768 |
1500 |
1600 |
2000 |
2400 |
3000 |
731 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1000 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.8 |
13.8 |
14.1 |
14.1 |
16 |
1 |
8 |
26 |
25 |
30 |
30 |
35 |
7.99 |
16 |
42 |
43 |
45 |
45 |
48 |
15.97 |
32 |
58 |
58 |
65 |
67 |
72 |
31.9 |
48 |
67 |
70 |
78 |
80 |
83 |
47.85 |
64 |
80 |
87 |
100 |
100 |
100 |
63.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.7 |
14.7 |
15.5 |
16 |
17 |
1 |
64 |
90 |
90 |
130 |
130 |
160 |
63.7 |
128 |
150 |
157 |
210 |
220 |
280 |
127 |
256 |
272 |
283 |
420 |
480 |
510 |
251 |
384 |
385 |
405 |
600 |
780 |
900 |
372.6 |
512 |
550 |
540 |
1000 |
1130 |
1500 |
491 |
768 |
5400 |
6000 |
9000 |
10000 |
10400 |
636 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
900 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15.2 |
15.1 |
15.4 |
15.5 |
18 |
1 |
8 |
28.7 |
28 |
29.3 |
30 |
40 |
7.99 |
16 |
50 |
53 |
56 |
56.4 |
60 |
15.97 |
32 |
64 |
63 |
70 |
70 |
75 |
31.9 |
48 |
85 |
85 |
90 |
91 |
97 |
47.8 |
64 |
104 |
104 |
113 |
116 |
140 |
63.7 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
16.1 |
16 |
16.5 |
17 |
20 |
1 |
64 |
100 |
100 |
116 |
128 |
175 |
63.7 |
128 |
170 |
176 |
240 |
290 |
300 |
126.7 |
256 |
325 |
340 |
420 |
620 |
680 |
250.3 |
384 |
450 |
475 |
600 |
1000 |
1300 |
371 |
512 |
660 |
637 |
1260 |
1500 |
2000 |
490 |
768 |
5300 |
5000 |
8300 |
8650 |
8900 |
598 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
900 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
17.8 |
16.6 |
21.3 |
32 |
39 |
1 |
8 |
39 |
38.3 |
46 |
57 |
80 |
7.99 |
16 |
56 |
55 |
67 |
77 |
100 |
15.96 |
32 |
80 |
80 |
92 |
110 |
130 |
31.9 |
48 |
96 |
94 |
114 |
125 |
170 |
47.8 |
64 |
122 |
121 |
147 |
160 |
190 |
63.7 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
24.5 |
21.6 |
39 |
39.3 |
40 |
1 |
64 |
150 |
150 |
200 |
200 |
200 |
63.6 |
128 |
230 |
230 |
300 |
320 |
340 |
126.8 |
256 |
420 |
420 |
600 |
640 |
800 |
250 |
384 |
640 |
620 |
1000 |
1200 |
1300 |
367 |
512 |
2000 |
2000 |
3000 |
4000 |
5000 |
480 |
768 |
7000 |
7000 |
10000 |
10000 |
12000 |
600 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
600 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15.4 |
15.3 |
15.8 |
16 |
17 |
1 |
8 |
30 |
30 |
40 |
40 |
43 |
7.99 |
16 |
55 |
56 |
61.4 |
62 |
70 |
15.96 |
32 |
64.3 |
63.6 |
70.4 |
73 |
80 |
31.9 |
48 |
80 |
84 |
91 |
93 |
130 |
47.8 |
64 |
106 |
105.4 |
113 |
118 |
170 |
63.6 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
16 |
16 |
16.5 |
16.6 |
18 |
1 |
64 |
105 |
104 |
140 |
150 |
158 |
63.7 |
128 |
164 |
177 |
199 |
205 |
220 |
127 |
256 |
300 |
330 |
370 |
400 |
400 |
252 |
384 |
460 |
500 |
600 |
700 |
900 |
373.4 |
512 |
650 |
670 |
1100 |
1400 |
1600 |
491 |
768 |
8000 |
8000 |
11000 |
11000 |
12000 |
590 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
800 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
19 |
18.66 |
19.6 |
20.6 |
30 |
0.999 |
8 |
25 |
23 |
30 |
30 |
58 |
7.98 |
16 |
34 |
30 |
40 |
42 |
93 |
15.93 |
32 |
46.2 |
44 |
52 |
55 |
160 |
31.8 |
48 |
58 |
53 |
60 |
64 |
260 |
47.6 |
64 |
74.5 |
64 |
72 |
80 |
440 |
63.3 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
23.6 |
22.5 |
29 |
33.7 |
34 |
1 |
64 |
120 |
103 |
216 |
330 |
420 |
63.1 |
128 |
180 |
140 |
329 |
540 |
700 |
125.1 |
256 |
320 |
230 |
680 |
1190 |
1560 |
245 |
384 |
600 |
340 |
1600 |
2200 |
3200 |
357 |
512 |
1060 |
450 |
3350 |
4200 |
6500 |
456 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1800 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10 |
9.4 |
10.4 |
11.3 |
20.8 |
1 |
8 |
15 |
14 |
18 |
20 |
58 |
7.98 |
16 |
20 |
18 |
23.3 |
26 |
97 |
15.93 |
32 |
29.6 |
25 |
33 |
37 |
180 |
31.8 |
48 |
39 |
32 |
43 |
50 |
270 |
47.5 |
64 |
47 |
36 |
50 |
90 |
350 |
63.2 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15 |
14 |
22 |
22.7 |
24 |
1 |
64 |
80 |
57 |
177 |
300 |
400 |
63.2 |
128 |
124 |
78 |
300 |
530 |
710 |
125 |
256 |
220 |
120 |
570 |
1000 |
1370 |
245.5 |
384 |
320 |
160 |
1030 |
1480 |
2110 |
361 |
512 |
500 |
202 |
1600 |
2300 |
3200 |
468 |
768 |
1150 |
300 |
3900 |
5000 |
7400 |
640 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
2000 |
160
Language model |
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
|||
n-gram |
1 |
14 |
13.74 |
14.1 |
14.2 |
27 |
1 |
n-gram |
8 |
27 |
26 |
30 |
32 |
58 |
7.98 |
n-gram |
16 |
39 |
40 |
46 |
46.6 |
100 |
15.93 |
n-gram |
32 |
70 |
72 |
77 |
78 |
200 |
31.74 |
n-gram |
48 |
102 |
106 |
110 |
118 |
300 |
47.4 |
none |
1 |
12 |
12 |
12 |
12 |
19 |
1 |
none |
8 |
30 |
30 |
30 |
30 |
50 |
7.98 |
none |
16 |
36 |
34 |
43 |
44 |
80 |
15.94 |
none |
32 |
65 |
70 |
74 |
75 |
175 |
31.75 |
none |
48 |
96 |
102 |
106.3 |
109 |
292 |
47.5 |
none |
64 |
132 |
133 |
141 |
160 |
420 |
63 |
800
Language model |
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
|||
n-gram |
1 |
15.7 |
15.2 |
16 |
23.5 |
25 |
1 |
n-gram |
64 |
140 |
137 |
156 |
210 |
340 |
63.3 |
n-gram |
128 |
240 |
240 |
300 |
403 |
685 |
125 |
n-gram |
256 |
440 |
440 |
500 |
780 |
2180 |
244.7 |
none |
1 |
13.15 |
12.8 |
13.2 |
17.2 |
18 |
1 |
none |
64 |
132 |
133 |
140 |
200 |
340 |
63.3 |
none |
128 |
224 |
220 |
260 |
380 |
660 |
125.2 |
none |
256 |
430 |
430 |
496 |
770 |
2176 |
245 |
none |
384 |
900 |
687 |
840 |
2400 |
10370 |
354 |
Speaker Diarization |
Language model |
# of streams |
Throughput (RTFX) |
---|---|---|---|
False |
n-gram |
32 |
920 |
False |
none |
32 |
940 |
True |
n-gram |
32 |
100 |
True |
none |
32 |
100 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.86 |
12.8 |
13.4 |
13.6 |
16 |
1 |
8 |
30 |
30 |
35 |
35 |
40 |
7.98 |
16 |
40 |
40 |
50 |
50 |
50 |
15.94 |
32 |
68 |
75 |
80 |
80 |
85 |
31.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14 |
14.3 |
15 |
16 |
16 |
0.999 |
64 |
130 |
140 |
150 |
153 |
155 |
63.3 |
128 |
240 |
247 |
270 |
270 |
280 |
125.4 |
256 |
430 |
480 |
530 |
550 |
560 |
246 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1100 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.2 |
13 |
13.6 |
13.7 |
19.4 |
1 |
8 |
25 |
24 |
30 |
30 |
50 |
7.99 |
16 |
36 |
35 |
43 |
44 |
78 |
15.98 |
32 |
70 |
77 |
82 |
86 |
156 |
31.9 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.4 |
14 |
17 |
20.3 |
20.5 |
1 |
64 |
130 |
130 |
140 |
210 |
330 |
63.7 |
128 |
222 |
230 |
255 |
403 |
650 |
127 |
256 |
513 |
434 |
1025 |
1780 |
2560 |
247 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
620 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.8 |
13.86 |
14.26 |
14.34 |
16.2 |
1 |
8 |
27 |
27 |
30 |
30 |
35 |
7.99 |
16 |
38 |
36 |
45 |
50 |
50 |
15.97 |
32 |
72.4 |
80.5 |
89 |
90 |
94 |
31.9 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.5 |
14.64 |
16 |
16.6 |
18 |
1 |
64 |
140 |
140 |
180 |
200 |
200 |
63.6 |
128 |
280 |
270 |
400 |
440 |
500 |
126 |
256 |
500 |
480 |
900 |
1000 |
1300 |
247 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1220 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.53 |
13.5 |
13.8 |
13.9 |
16.16 |
1 |
8 |
25 |
24.3 |
30 |
30 |
30 |
7.99 |
16 |
34 |
33 |
40 |
42 |
44 |
15.98 |
32 |
60 |
70 |
72 |
73 |
77 |
31.9 |
48 |
91 |
102 |
105 |
106.4 |
120 |
47.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.8 |
13.7 |
14.4 |
15 |
16 |
1 |
64 |
140 |
140 |
160 |
200 |
240 |
63.5 |
128 |
230 |
230 |
300 |
440 |
500 |
126.2 |
256 |
460 |
470 |
700 |
1000 |
1000 |
248 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1370 |
160
none
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15.4 |
15.3 |
15.5 |
16 |
20 |
0.999 |
8 |
36 |
40 |
40 |
41 |
50 |
7.97 |
16 |
58 |
60 |
68 |
73 |
90 |
15.9 |
32 |
94 |
100 |
110 |
120 |
160 |
31.6 |
48 |
140 |
145 |
160 |
170 |
300 |
47.1 |
800
none
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
18 |
17.7 |
20.2 |
21 |
21 |
0.999 |
64 |
200 |
200 |
200 |
300 |
360 |
62.4 |
128 |
280 |
273 |
400 |
520 |
650 |
122.5 |
256 |
680 |
535 |
1270 |
1700 |
2500 |
231 |
none
# of streams |
Throughput (RTFX) |
---|---|
32 |
630 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.4 |
13.5 |
14 |
14.3 |
16.3 |
1 |
8 |
26 |
25 |
30 |
30 |
30 |
7.99 |
16 |
35 |
33.6 |
43 |
44 |
46 |
15.97 |
32 |
63 |
70 |
75 |
76 |
100 |
31.9 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.6 |
15 |
16 |
16.3 |
17 |
1 |
64 |
124 |
134 |
145 |
149 |
154 |
63.6 |
128 |
220 |
244 |
260 |
266 |
270 |
126.7 |
256 |
400 |
430 |
500 |
580 |
700 |
250.3 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1250 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13 |
12.74 |
13.3 |
15 |
19.6 |
1 |
8 |
26 |
24.4 |
27 |
34 |
56 |
7.98 |
16 |
38.3 |
40 |
45 |
48 |
93 |
15.94 |
32 |
70 |
70 |
80 |
100 |
230 |
31.74 |
48 |
110 |
107 |
120 |
200 |
450 |
47.4 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15 |
13.9 |
20 |
20.4 |
20.7 |
1 |
64 |
150 |
135 |
264 |
316 |
357 |
63.2 |
128 |
270 |
240 |
520 |
620 |
700 |
125 |
256 |
670 |
480 |
1510 |
2160 |
2350 |
244.3 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
670 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.5 |
11.4 |
11.6 |
11.8 |
19 |
1 |
8 |
18 |
17 |
19 |
21 |
38 |
7.99 |
16 |
30 |
29 |
32 |
33 |
67 |
15.95 |
32 |
50 |
48 |
50 |
54 |
123 |
31.8 |
48 |
68 |
65 |
70 |
72 |
200 |
47.6 |
64 |
80 |
78 |
86 |
90 |
265 |
63.2 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.2 |
11.8 |
12.2 |
17 |
18 |
1 |
64 |
83 |
76 |
91 |
143 |
280 |
63.4 |
128 |
150 |
150 |
170 |
277 |
540 |
125.6 |
256 |
292 |
280 |
320 |
550 |
1300 |
246.5 |
384 |
433 |
404 |
460 |
840 |
2300 |
363 |
512 |
600 |
520 |
630 |
1470 |
4700 |
476 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
2400 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10.5 |
10.6 |
11 |
11 |
12.4 |
1 |
8 |
17 |
17 |
19 |
20 |
22 |
8 |
16 |
30 |
30 |
33 |
35 |
36 |
15.98 |
32 |
43 |
42 |
48 |
50 |
54 |
31.95 |
48 |
54 |
54 |
58 |
60 |
64 |
47.9 |
64 |
66 |
66 |
72 |
73 |
76.7 |
63.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.5 |
11.7 |
12.4 |
13 |
14 |
1 |
64 |
80 |
80 |
90 |
93 |
100 |
63.8 |
128 |
140 |
147 |
170 |
170 |
190 |
127.3 |
256 |
256 |
280 |
306 |
310 |
330 |
253.3 |
384 |
380 |
410 |
440 |
450 |
500 |
378 |
512 |
500 |
540 |
640 |
700 |
1000 |
500 |
768 |
3300 |
3200 |
6000 |
7000 |
7000 |
700 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1300 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10.05 |
10.07 |
10.3 |
10.36 |
11.5 |
1 |
8 |
20 |
20 |
23 |
24 |
25 |
8 |
16 |
30 |
30 |
35 |
35 |
37 |
15.98 |
32 |
53 |
55 |
60 |
60 |
70 |
31.9 |
48 |
70 |
78 |
84 |
85 |
90 |
47.8 |
64 |
92 |
102 |
110 |
114 |
130 |
63.7 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.5 |
12.48 |
13.1 |
13.5 |
15 |
1 |
64 |
100 |
110 |
120 |
140 |
200 |
63.7 |
128 |
203 |
210 |
300 |
320 |
386 |
126.4 |
256 |
360 |
370 |
600 |
640 |
660 |
250 |
384 |
530 |
530 |
800 |
1000 |
1400 |
369.5 |
512 |
900 |
730 |
1800 |
2000 |
2700 |
486 |
768 |
7000 |
7000 |
12000 |
12400 |
13000 |
530 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
900 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.1 |
12 |
12.2 |
12.3 |
14 |
1 |
8 |
25 |
26 |
28 |
29 |
30 |
7.99 |
16 |
35 |
30 |
40 |
44 |
46 |
15.98 |
32 |
67 |
66 |
73 |
75 |
90 |
31.9 |
48 |
100 |
98 |
103 |
106 |
126 |
47.8 |
64 |
120 |
124 |
130 |
140 |
200 |
63.6 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12 |
12 |
12.54 |
13 |
14 |
1 |
64 |
130 |
130 |
169 |
190 |
250 |
63.6 |
128 |
230 |
235 |
360 |
400 |
440 |
126.3 |
256 |
420 |
440 |
500 |
860 |
1000 |
249 |
384 |
730 |
673 |
1400 |
1700 |
2000 |
367.4 |
512 |
4500 |
5000 |
6600 |
7000 |
8000 |
423 |
768 |
13500 |
12500 |
24200 |
25000 |
26000 |
427.5 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
800 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.1 |
13 |
18.3 |
28.6 |
36.1 |
1 |
8 |
34 |
33 |
40 |
50 |
75 |
7.99 |
16 |
50 |
50 |
57 |
70 |
90 |
15.97 |
32 |
96 |
96 |
110 |
123 |
140 |
31.9 |
48 |
123 |
124 |
143 |
153 |
184 |
47.8 |
64 |
170 |
160 |
230 |
270 |
300 |
63.5 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
21 |
18.3 |
35.6 |
36 |
37 |
1 |
64 |
146 |
144 |
190 |
200 |
210 |
63.7 |
128 |
260 |
260 |
330 |
340 |
400 |
126.7 |
256 |
460 |
460 |
700 |
750 |
930 |
249.5 |
384 |
800 |
700 |
1500 |
1700 |
2000 |
367 |
512 |
3000 |
3000 |
5000 |
5000 |
6000 |
470 |
768 |
10000 |
10700 |
16000 |
17000 |
17000 |
500 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
700 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11 |
11 |
11.3 |
11.4 |
12.2 |
1 |
8 |
23.5 |
23 |
26 |
27 |
30 |
7.99 |
16 |
38 |
37 |
45 |
46 |
47 |
15.97 |
32 |
67.7 |
67 |
74 |
76 |
100 |
31.9 |
48 |
97 |
96 |
100 |
100 |
158 |
47.7 |
64 |
126 |
124 |
130 |
135 |
220 |
63.5 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12 |
12.1 |
12.7 |
12.8 |
14 |
1 |
64 |
120 |
130 |
142 |
145 |
150 |
63.6 |
128 |
220 |
240 |
258 |
264 |
270 |
126.7 |
256 |
440 |
470 |
550 |
600 |
700 |
250 |
384 |
730 |
710 |
1170 |
1500 |
1700 |
368 |
512 |
6000 |
7000 |
8000 |
9000 |
9000 |
403 |
768 |
13600 |
13000 |
23000 |
24000 |
25000 |
408 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
800 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.8 |
12.6 |
13.6 |
14.5 |
21 |
1 |
8 |
20 |
20 |
23 |
25 |
40 |
7.99 |
16 |
30 |
29 |
33 |
36 |
60 |
15.95 |
32 |
52 |
49.6 |
55 |
58 |
150 |
31.8 |
48 |
73 |
67 |
74 |
78 |
300 |
47.6 |
64 |
122 |
87 |
99 |
302 |
995 |
63.3 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
17 |
16.2 |
21 |
24 |
24.5 |
1 |
64 |
110 |
94.5 |
160 |
240 |
310 |
63.4 |
128 |
200 |
180 |
300 |
467 |
600 |
125.5 |
256 |
390 |
330 |
670 |
1200 |
1520 |
246 |
384 |
830 |
500 |
2180 |
3000 |
4200 |
358 |
512 |
3000 |
670 |
10000 |
13000 |
15600 |
418 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1500 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
7.6 |
7.2 |
8.4 |
9.2 |
16 |
1 |
8 |
11.4 |
10.5 |
13 |
15 |
35 |
7.99 |
16 |
15.3 |
14.3 |
17.5 |
20 |
56 |
15.96 |
32 |
23 |
21 |
26 |
28 |
95 |
31.86 |
48 |
29 |
26 |
32.2 |
35 |
153 |
47.7 |
64 |
38 |
33 |
40 |
46 |
210 |
63.5 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13 |
13 |
18 |
18 |
20 |
1 |
64 |
63 |
49 |
107 |
190 |
257 |
63.5 |
128 |
104 |
76 |
200 |
367 |
502 |
126 |
256 |
190 |
130 |
390 |
729 |
1095 |
248 |
384 |
290 |
182 |
713 |
1230 |
1800 |
366 |
512 |
420 |
244 |
1220 |
1850 |
2400 |
480 |
768 |
945 |
370 |
3080 |
4000 |
6040 |
675 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
2000 |
160
Language model |
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
|||
n-gram |
1 |
13 |
12.7 |
13 |
13.5 |
23 |
1 |
n-gram |
8 |
23 |
22 |
25 |
25 |
47 |
7.98 |
n-gram |
16 |
26 |
24.9 |
27 |
34 |
60 |
15.96 |
n-gram |
32 |
42 |
40 |
50 |
50 |
110 |
31.83 |
n-gram |
48 |
54 |
57 |
61.5 |
64 |
132 |
47.7 |
none |
1 |
12.2 |
12.05 |
12.6 |
12.8 |
17 |
1 |
none |
8 |
21.5 |
21 |
23 |
24 |
38 |
7.99 |
none |
16 |
26 |
24 |
33 |
35 |
50 |
15.96 |
none |
32 |
38 |
36 |
46.7 |
47.6 |
90 |
31.85 |
none |
48 |
48.3 |
50 |
58.6 |
59.7 |
116 |
47.7 |
none |
64 |
60 |
65 |
68.6 |
70 |
149 |
63.5 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.6 |
14.2 |
15 |
21 |
23 |
1 |
64 |
71 |
71 |
76.4 |
120 |
214 |
63.5 |
128 |
110 |
104 |
119 |
210 |
400 |
126.2 |
256 |
184 |
170 |
210 |
405 |
784 |
249 |
n-gram
Speaker Diarization |
Language model |
# of streams |
Throughput (RTFX) |
---|---|---|---|
False |
n-gram |
32 |
1830 |
False |
none |
32 |
1900 |
True |
n-gram |
32 |
140 |
True |
none |
32 |
140 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.3 |
12.2 |
12.8 |
13 |
15 |
1 |
8 |
22.6 |
22.4 |
24 |
26 |
30 |
7.99 |
16 |
30 |
30 |
37 |
38 |
39 |
15.96 |
32 |
43 |
43 |
50 |
50 |
55 |
31.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.3 |
13.2 |
14 |
14.2 |
19 |
1 |
8 |
23 |
22.3 |
24 |
29 |
35 |
8 |
16 |
30 |
30 |
37 |
38 |
50 |
15.98 |
32 |
43 |
41 |
53 |
54 |
80 |
31.95 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12 |
12 |
13 |
14 |
16 |
0.998 |
8 |
22 |
21 |
25 |
27 |
30 |
7.98 |
16 |
27 |
25 |
33 |
35 |
40 |
15.94 |
32 |
40 |
40 |
46 |
47 |
53 |
31.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13 |
13 |
13.7 |
14 |
16 |
1 |
8 |
24 |
22.8 |
30 |
30 |
31 |
7.99 |
16 |
29 |
27.4 |
36.3 |
37.3 |
38.5 |
15.98 |
32 |
43 |
43 |
52 |
53 |
55 |
31.94 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.6 |
12.6 |
13 |
13 |
15 |
1 |
8 |
23 |
21.6 |
27 |
29 |
30 |
8 |
16 |
27 |
25.3 |
34 |
35 |
36 |
15.98 |
32 |
40 |
40 |
47 |
48 |
50 |
31.94 |
48 |
54 |
56.5 |
59 |
59 |
65 |
47.9 |
160
none
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.8 |
14.6 |
15.2 |
15.5 |
19 |
0.999 |
8 |
28 |
27 |
30 |
34 |
40 |
7.97 |
16 |
39 |
40 |
47 |
48 |
67 |
15.9 |
32 |
60 |
60.4 |
70 |
77 |
120 |
31.7 |
48 |
80 |
80 |
100 |
100 |
150 |
47.4 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.5 |
12.4 |
13 |
13.3 |
15 |
1 |
8 |
22 |
21 |
27 |
28 |
30 |
7.99 |
16 |
27 |
25 |
34 |
35 |
36 |
15.98 |
32 |
40 |
40 |
46 |
46.7 |
50 |
31.94 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
25 |
22.4 |
32.4 |
40.6 |
50 |
0.997 |
8 |
42 |
37 |
56 |
64 |
90 |
7.96 |
16 |
55 |
50 |
68 |
86 |
110 |
15.9 |
32 |
90 |
90 |
110 |
120 |
160 |
31.64 |
48 |
120 |
120 |
143 |
160 |
208 |
47.4 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
36.3 |
39.6 |
66 |
66 |
66 |
0.997 |
64 |
190 |
170 |
300 |
300 |
300 |
63 |
128 |
320 |
300 |
440 |
480 |
500 |
124.6 |
256 |
600 |
570 |
800 |
900 |
1000 |
240 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
900 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.3 |
12.3 |
12.7 |
12.9 |
15 |
1 |
8 |
22 |
21.3 |
25 |
28 |
29 |
7.99 |
16 |
27 |
25 |
34 |
34 |
35 |
15.97 |
32 |
38 |
37 |
46 |
47 |
50 |
31.93 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.7 |
12.6 |
13 |
15 |
16 |
1 |
64 |
62 |
68 |
72 |
74 |
82 |
63.8 |
128 |
94 |
96 |
115 |
117 |
200 |
127.4 |
256 |
150 |
160 |
200 |
205 |
220 |
254 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
2390 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14 |
13.3 |
16 |
19 |
19.4 |
0.998 |
8 |
25 |
24 |
30 |
32.7 |
40 |
7.96 |
16 |
32 |
29.4 |
41 |
50 |
50 |
15.9 |
32 |
50 |
50 |
70 |
83 |
90 |
31.6 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15 |
14 |
17 |
17 |
17 |
0.997 |
64 |
84 |
88 |
100 |
100 |
110 |
63 |
128 |
130 |
138 |
150 |
156 |
160 |
125 |
256 |
230 |
250 |
280 |
300 |
300 |
245 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1600 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14 |
13.3 |
16 |
16 |
20 |
0.996 |
8 |
27 |
26 |
30 |
30 |
30 |
7.93 |
16 |
35 |
33 |
43 |
44 |
48 |
15.82 |
32 |
60 |
60 |
70 |
80 |
90 |
31.4 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.7 |
12.7 |
13.3 |
14 |
16 |
0.998 |
8 |
23 |
21.8 |
27 |
30 |
30 |
7.98 |
16 |
26 |
25 |
30 |
35 |
38 |
15.94 |
32 |
40 |
40 |
48.6 |
49.3 |
54 |
31.84 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12 |
11.7 |
12.3 |
14 |
17.3 |
1 |
8 |
22 |
21 |
26 |
30 |
43 |
7.99 |
16 |
26 |
23.7 |
30 |
35 |
63 |
15.96 |
32 |
40 |
37 |
46.6 |
50 |
111 |
31.85 |
48 |
50 |
50 |
59 |
70 |
160 |
47.7 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
29.4 |
28.6 |
29.3 |
33 |
53 |
0.998 |
4 |
40 |
37.7 |
50 |
50 |
75 |
3.99 |
8 |
70 |
67.6 |
70 |
90 |
138 |
7.96 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
31.4 |
31 |
32 |
38 |
39 |
0.999 |
4 |
40 |
38.9 |
50 |
60 |
60 |
3.99 |
8 |
70 |
69 |
71 |
90 |
97 |
7.98 |
n-gram
Speaker Diarization |
# of streams |
Throughput (RTFX) |
---|---|---|
False |
1 |
170 |
False |
4 |
290 |
False |
8 |
320 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9.4 |
9.3 |
9.7 |
9.8 |
13 |
1 |
8 |
13 |
12.8 |
14 |
15 |
24.3 |
7.99 |
16 |
17.6 |
17 |
19.5 |
20.5 |
39 |
15.97 |
32 |
25 |
24 |
27 |
28 |
65 |
31.9 |
48 |
32 |
31 |
34 |
35 |
90 |
47.7 |
64 |
40 |
38 |
41 |
41.7 |
120 |
63.5 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9.5 |
9.5 |
10 |
10 |
12 |
1 |
8 |
13 |
13 |
16 |
16.4 |
17.5 |
8 |
16 |
18 |
18 |
20 |
21 |
22 |
16 |
32 |
25 |
25 |
27 |
28 |
29.4 |
32 |
48 |
30 |
30 |
34 |
34 |
36 |
47.9 |
64 |
37 |
37 |
40 |
41 |
43 |
63.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10.3 |
10.2 |
11 |
11 |
12 |
0.999 |
8 |
18.6 |
18 |
22 |
23 |
24 |
7.98 |
16 |
22 |
20 |
28 |
29 |
30 |
15.95 |
32 |
38 |
38.5 |
41 |
42 |
45 |
31.87 |
48 |
49 |
50 |
53 |
54 |
57 |
47.7 |
64 |
60 |
62 |
68 |
70 |
71 |
63.6 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9.7 |
9.6 |
10.1 |
10.4 |
12 |
1 |
8 |
15.3 |
15 |
17 |
18 |
19 |
8 |
16 |
20 |
20 |
23.3 |
23.7 |
24.6 |
16 |
32 |
32 |
32 |
35 |
36 |
38 |
31.95 |
48 |
40 |
41 |
45 |
46 |
48 |
47.9 |
64 |
48 |
49 |
54 |
55 |
56.4 |
63.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11 |
11 |
12 |
12 |
13 |
1 |
8 |
18 |
17.7 |
20 |
20 |
23 |
8 |
16 |
23 |
20 |
28.2 |
28.7 |
30 |
15.98 |
32 |
37 |
38 |
40.1 |
41 |
44 |
31.95 |
48 |
47 |
49 |
54 |
55 |
58 |
47.9 |
64 |
60 |
61 |
68 |
70 |
72 |
63.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.6 |
12.3 |
17 |
27.6 |
38 |
1 |
8 |
21.6 |
19.7 |
25.3 |
38 |
60 |
8 |
16 |
30 |
28 |
35 |
55 |
71 |
15.98 |
32 |
47 |
45 |
54 |
66 |
100 |
31.94 |
48 |
64 |
63 |
73 |
90 |
130 |
47.9 |
64 |
78.6 |
77 |
91 |
114 |
150 |
63.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11 |
11 |
11.5 |
11.7 |
12.2 |
1 |
8 |
18 |
18 |
20 |
20 |
24 |
8 |
16 |
23 |
21 |
29 |
29.4 |
30.3 |
15.98 |
32 |
37.4 |
39 |
41.4 |
42 |
43.3 |
31.95 |
48 |
48.6 |
50 |
53 |
54 |
58 |
47.9 |
64 |
61 |
61.2 |
66.5 |
69 |
80 |
63.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10.6 |
10.4 |
11 |
12 |
13 |
0.997 |
8 |
18.27 |
18 |
20 |
20 |
20 |
7.96 |
16 |
24 |
22 |
30 |
30 |
30 |
15.9 |
32 |
41 |
42 |
48 |
50 |
54 |
31.6 |
48 |
55 |
53 |
73 |
76 |
90 |
47.1 |
64 |
71 |
67 |
100 |
106 |
110 |
62.4 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.25 |
11.2 |
12 |
13 |
14 |
0.999 |
8 |
18 |
17.5 |
20 |
22 |
30 |
7.98 |
16 |
26 |
25 |
30 |
32 |
36 |
15.94 |
32 |
38 |
38.6 |
42 |
44 |
47 |
31.86 |
48 |
49.3 |
50.4 |
54 |
55 |
60 |
47.7 |
64 |
60 |
60 |
70 |
70 |
74 |
63.5 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10.5 |
10 |
11.3 |
14 |
15.5 |
1 |
8 |
19 |
18 |
20 |
25 |
38 |
7.99 |
16 |
23 |
20 |
28 |
34 |
60 |
15.96 |
32 |
40 |
38 |
41.7 |
60 |
113 |
31.85 |
48 |
53 |
49 |
54 |
85 |
162 |
47.7 |
64 |
65 |
60 |
66 |
105 |
250 |
63.4 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9.7 |
9.4 |
10.2 |
11.3 |
16.4 |
1 |
8 |
12 |
11 |
14 |
14.4 |
26 |
7.99 |
16 |
14 |
13 |
16 |
17 |
38 |
15.97 |
32 |
20 |
19 |
22 |
23.3 |
62 |
31.9 |
48 |
24 |
22.65 |
26 |
28 |
83 |
47.8 |
64 |
28 |
26 |
30 |
32 |
120 |
63.6 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
7.8 |
7.5 |
8.4 |
9.4 |
15 |
1 |
8 |
9 |
9 |
10.5 |
12 |
28 |
7.99 |
16 |
12 |
11 |
14 |
15 |
44.4 |
15.97 |
32 |
16 |
14.5 |
17.4 |
19 |
71 |
31.9 |
48 |
18 |
16 |
20 |
22 |
100 |
47.8 |
64 |
23 |
20 |
24 |
26 |
139 |
63.6 |
160
Language model |
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
|||
n-gram |
1 |
19 |
19 |
19.4 |
19.65 |
38 |
0.999 |
n-gram |
8 |
40 |
40 |
50 |
50 |
92 |
7.97 |
n-gram |
16 |
50 |
46.6 |
60 |
66 |
110 |
15.92 |
n-gram |
32 |
80 |
80 |
94 |
95 |
250 |
31.7 |
n-gram |
48 |
112 |
120 |
126 |
130 |
320 |
47.4 |
none |
1 |
17 |
17 |
17 |
17 |
30 |
1 |
none |
8 |
38 |
37 |
37.6 |
37.9 |
77 |
7.98 |
none |
16 |
50 |
50 |
60 |
60 |
100 |
15.93 |
none |
32 |
74 |
80 |
90 |
90 |
210 |
31.7 |
none |
48 |
105 |
116 |
119.5 |
122 |
310 |
47.4 |
none |
64 |
138 |
140 |
150 |
157 |
430 |
63 |
800
Language model |
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
|||
n-gram |
1 |
20 |
20 |
20.7 |
29 |
29.8 |
1 |
n-gram |
64 |
146 |
148 |
162 |
220 |
370 |
63.2 |
n-gram |
128 |
240 |
250 |
270 |
410 |
710 |
125 |
n-gram |
256 |
415 |
400 |
470 |
770 |
2130 |
244.4 |
none |
1 |
17 |
17 |
18 |
22 |
23 |
1 |
none |
64 |
133 |
138 |
150 |
220 |
370 |
63.2 |
none |
128 |
220 |
210 |
262 |
400 |
690 |
125 |
none |
256 |
380 |
360 |
450 |
750 |
2060 |
244.7 |
none |
384 |
620 |
570 |
660 |
1570 |
5000 |
358 |
Speaker Diarization |
Language model |
# of streams |
Throughput (RTFX) |
---|---|---|---|
False |
n-gram |
32 |
900 |
False |
none |
32 |
900 |
True |
n-gram |
32 |
80 |
True |
none |
32 |
76 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
18 |
18 |
19 |
19 |
21 |
0.999 |
8 |
41 |
41 |
42.4 |
44 |
60 |
7.98 |
16 |
53 |
51 |
70 |
72 |
75 |
15.93 |
32 |
80 |
90 |
98 |
100 |
104 |
31.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
19 |
19 |
20 |
20 |
20 |
0.998 |
64 |
148 |
160 |
172 |
180 |
180 |
63.2 |
128 |
250 |
270 |
300 |
310 |
320 |
125.3 |
256 |
410 |
450 |
495 |
510 |
540 |
246.6 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
900 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
17 |
17 |
18 |
18 |
24 |
1 |
8 |
40 |
39 |
40 |
40 |
68 |
7.99 |
16 |
50 |
47 |
60 |
65 |
100 |
15.97 |
32 |
76 |
80 |
92 |
94 |
220 |
31.9 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
18 |
18 |
20 |
25 |
26 |
1 |
64 |
140 |
143 |
155 |
230 |
370 |
63.7 |
128 |
237 |
240 |
260 |
420 |
696 |
127 |
256 |
480 |
390 |
970 |
1700 |
2400 |
247 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
570 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
18.77 |
18.83 |
19.3 |
19.4 |
21.8 |
1 |
8 |
40.7 |
40.2 |
42.4 |
43 |
54 |
7.99 |
16 |
54 |
50 |
70 |
70 |
74 |
15.97 |
32 |
77 |
87 |
95 |
96 |
100 |
31.9 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
20.2 |
20.4 |
22 |
22.6 |
23 |
1 |
64 |
153 |
160 |
195 |
200 |
250 |
63.5 |
128 |
268 |
267 |
400 |
420 |
468 |
126 |
256 |
480 |
460 |
800 |
900 |
1000 |
247 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1100 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
18 |
18 |
18 |
18 |
20 |
1 |
8 |
39.5 |
39.1 |
40 |
42 |
55 |
7.99 |
16 |
50 |
50 |
70 |
70 |
70 |
15.96 |
32 |
75 |
87 |
93 |
94 |
100 |
31.9 |
48 |
105 |
119 |
122.9 |
124.4 |
150 |
47.7 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
19 |
19 |
19 |
20.6 |
22 |
1 |
64 |
140 |
147 |
178.5 |
194 |
234 |
63.6 |
128 |
235 |
240 |
310 |
400 |
430 |
126.3 |
256 |
380 |
400 |
500 |
830 |
967 |
249 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1530 |
160
none
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
19 |
19 |
19 |
20 |
25 |
0.998 |
8 |
60 |
60 |
63 |
65 |
80 |
7.95 |
16 |
63 |
62 |
79 |
82 |
100 |
15.87 |
32 |
107 |
111 |
122 |
130 |
200 |
31.55 |
48 |
155 |
162 |
180 |
190 |
318 |
47 |
800
none
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
22 |
22 |
24 |
25 |
26 |
0.998 |
64 |
200 |
200 |
300 |
400 |
420 |
62 |
128 |
400 |
380 |
600 |
800 |
1000 |
121 |
256 |
1600 |
2000 |
2000 |
3000 |
4000 |
215 |
none
# of streams |
Throughput (RTFX) |
---|---|
32 |
590 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
18 |
18 |
18.4 |
18.6 |
21 |
1 |
8 |
38.2 |
38 |
38.8 |
40 |
48.3 |
7.99 |
16 |
50 |
50 |
66 |
68 |
72 |
15.95 |
32 |
73 |
80 |
89 |
90 |
130 |
31.85 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
18 |
19 |
19.5 |
20.7 |
21 |
1 |
64 |
128 |
140 |
150 |
155 |
160 |
63.6 |
128 |
210 |
230 |
250 |
250 |
260 |
126.7 |
256 |
330 |
360 |
440 |
480 |
520 |
251 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1230 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
18 |
17 |
18 |
24 |
28 |
1 |
8 |
40 |
37 |
40 |
50 |
80 |
7.97 |
16 |
50 |
45 |
60 |
65 |
120 |
15.92 |
32 |
80 |
86 |
90 |
110 |
270 |
31.7 |
48 |
118 |
116.6 |
127.6 |
220 |
477 |
47.4 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
20 |
18 |
25 |
25 |
25 |
1 |
64 |
158 |
140 |
285 |
340 |
383 |
63.2 |
128 |
280 |
240 |
533 |
644 |
730 |
124.9 |
256 |
610 |
427 |
1420 |
1900 |
2240 |
244.4 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
653 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13 |
12.6 |
13 |
13 |
22 |
1 |
8 |
20.7 |
20 |
21.2 |
21.6 |
46.6 |
7.98 |
16 |
30 |
28 |
33 |
35 |
70 |
15.94 |
32 |
48 |
46 |
50 |
50 |
130 |
31.8 |
48 |
65 |
62 |
70 |
70 |
200 |
47.5 |
64 |
77 |
73 |
80 |
80 |
270 |
63.2 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.7 |
13 |
13.6 |
19 |
20 |
1 |
64 |
90 |
80 |
90 |
156 |
300 |
63.3 |
128 |
155 |
140 |
150 |
290 |
580 |
125.4 |
256 |
280 |
250 |
290 |
560 |
1300 |
246 |
384 |
416 |
370 |
423 |
900 |
2310 |
362 |
512 |
570 |
490 |
590 |
1450 |
4300 |
473 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
2000 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.4 |
12.5 |
13 |
13 |
14.3 |
1 |
8 |
20 |
19.7 |
21 |
21.5 |
25 |
8 |
16 |
29 |
28.4 |
34 |
34.5 |
37 |
15.98 |
32 |
48.7 |
49 |
50.7 |
51.4 |
56 |
31.94 |
48 |
66 |
66 |
70 |
72 |
76 |
47.9 |
64 |
71 |
72 |
77 |
78 |
83 |
63.8 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13 |
13 |
13.8 |
14.3 |
16 |
1 |
64 |
81 |
81 |
88 |
90 |
96 |
63.8 |
128 |
134 |
135 |
149 |
153 |
160 |
127.4 |
256 |
240 |
243 |
270 |
280 |
300 |
253.7 |
384 |
330 |
350 |
400 |
410 |
440 |
379 |
512 |
450 |
480 |
550 |
600 |
800 |
500 |
768 |
1800 |
2000 |
3000 |
3000 |
3000 |
725 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
900 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.2 |
13.23 |
13.5 |
13.6 |
14.7 |
1 |
8 |
24.5 |
24.3 |
25.6 |
26 |
32 |
8 |
16 |
33 |
30 |
44 |
45 |
50 |
15.98 |
32 |
64 |
64 |
70 |
74 |
80 |
31.9 |
48 |
80 |
82 |
92 |
93 |
96 |
47.8 |
64 |
94 |
100 |
112 |
113 |
120 |
63.7 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.4 |
13.4 |
14 |
14.6 |
16 |
1 |
64 |
100 |
105 |
118 |
124 |
180 |
63.7 |
128 |
182 |
183 |
256 |
280 |
330 |
126.7 |
256 |
320 |
330 |
500 |
570 |
600 |
250.3 |
384 |
470 |
470 |
750 |
960 |
1330 |
371 |
512 |
780 |
640 |
1600 |
2000 |
2400 |
488 |
768 |
10000 |
10000 |
15300 |
16000 |
16000 |
565 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
700 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.4 |
14 |
15 |
15 |
17 |
1 |
8 |
30.4 |
30.1 |
31.2 |
31.7 |
43 |
7.99 |
16 |
50 |
50 |
58 |
58 |
60 |
15.97 |
32 |
80 |
80 |
80 |
83 |
90 |
31.9 |
48 |
98 |
98.5 |
102 |
104 |
110 |
47.8 |
64 |
123 |
122 |
127 |
130 |
166 |
63.7 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14 |
14 |
14.36 |
15 |
16 |
1 |
64 |
128 |
125 |
140 |
158 |
190 |
63.7 |
128 |
200 |
203 |
240 |
330 |
350 |
126.5 |
256 |
347 |
340 |
400 |
730 |
800 |
250 |
384 |
550 |
550 |
1000 |
1200 |
1660 |
369 |
512 |
1300 |
900 |
2700 |
3000 |
3000 |
486 |
768 |
16000 |
17000 |
23000 |
24000 |
25000 |
500 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
700 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
16.6 |
15 |
20.4 |
32 |
41 |
1 |
8 |
33.3 |
31 |
39 |
50 |
70 |
7.99 |
16 |
50 |
45 |
60 |
80 |
113 |
15.97 |
32 |
90 |
90 |
100 |
120 |
160 |
31.9 |
48 |
122 |
120 |
137 |
160 |
210 |
47.8 |
64 |
150 |
150 |
180 |
200 |
280 |
63.6 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
23.4 |
21 |
36 |
38 |
38.5 |
1 |
64 |
170 |
160 |
230 |
240 |
260 |
63.6 |
128 |
315 |
310 |
420 |
450 |
550 |
126.4 |
256 |
640 |
600 |
1010 |
1200 |
1300 |
246.5 |
384 |
2300 |
2300 |
3500 |
4000 |
5000 |
357 |
512 |
7000 |
7000 |
10000 |
10000 |
12000 |
400 |
768 |
13800 |
15000 |
20000 |
22000 |
23000 |
470 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
500 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14 |
14 |
15 |
15 |
16 |
1 |
8 |
29.7 |
29.4 |
31 |
31.2 |
40 |
7.99 |
16 |
40 |
36 |
56 |
57 |
60 |
15.97 |
32 |
74 |
73 |
80 |
81 |
100 |
31.9 |
48 |
97 |
97 |
102 |
104 |
145 |
47.7 |
64 |
120 |
119 |
125 |
127 |
200 |
63.5 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15 |
15 |
15.6 |
16 |
18 |
1 |
64 |
125 |
125 |
132 |
135 |
150 |
63.7 |
128 |
185 |
200 |
220 |
220 |
230 |
127 |
256 |
310 |
330 |
396 |
410 |
460 |
251.7 |
384 |
470 |
540 |
700 |
800 |
870 |
372.5 |
512 |
800 |
747 |
1400 |
2000 |
2000 |
489 |
768 |
14000 |
15000 |
20000 |
20000 |
21000 |
510 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
600 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
22 |
22 |
23 |
24 |
31.2 |
1 |
8 |
26 |
25 |
26.6 |
29 |
50 |
7.98 |
16 |
36 |
36 |
42.7 |
45 |
70 |
15.95 |
32 |
59 |
57 |
63 |
67 |
171 |
31.8 |
48 |
83 |
74 |
82 |
91 |
400 |
47.6 |
64 |
170 |
96 |
200 |
700 |
1500 |
63.3 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
27 |
26 |
33 |
35 |
50 |
1 |
64 |
120 |
104 |
180 |
270 |
340 |
63.3 |
128 |
220 |
187 |
326 |
508 |
650 |
125.3 |
256 |
440 |
356 |
720 |
1260 |
1800 |
245.6 |
384 |
920 |
520 |
2500 |
3500 |
4700 |
355 |
512 |
3500 |
710 |
13000 |
15000 |
17000 |
404 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1360 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9 |
8.6 |
9.8 |
10.5 |
18 |
1 |
8 |
12 |
11 |
13 |
15 |
43 |
7.98 |
16 |
16 |
15 |
19 |
21 |
67 |
15.95 |
32 |
22 |
20 |
25 |
28 |
114 |
31.83 |
48 |
29.5 |
27 |
32.5 |
37 |
167 |
47.7 |
64 |
38.5 |
34 |
42 |
50 |
240 |
63.4 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.5 |
13.7 |
20 |
20.4 |
22 |
1 |
64 |
66 |
50 |
120 |
210 |
280 |
63.4 |
128 |
108 |
76 |
214 |
393 |
537 |
125.8 |
256 |
190 |
120 |
397 |
750 |
1100 |
247.4 |
384 |
280 |
160 |
710 |
1210 |
1750 |
365 |
512 |
400 |
210 |
1180 |
1780 |
2340 |
479 |
768 |
820 |
310 |
2680 |
3560 |
5140 |
675 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1500 |
160
Language model |
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
|||
n-gram |
1 |
34.4 |
33.5 |
34 |
34 |
79 |
0.998 |
n-gram |
8 |
56 |
56.5 |
62 |
65 |
150 |
7.95 |
n-gram |
16 |
63 |
60 |
70 |
70 |
170 |
15.88 |
n-gram |
32 |
146 |
132.2 |
170 |
220 |
470 |
31.5 |
none |
1 |
32.8 |
32.2 |
32.4 |
32.4 |
70 |
0.998 |
none |
8 |
54 |
55 |
60.7 |
63 |
135 |
7.96 |
none |
16 |
60 |
57 |
61 |
70 |
150 |
15.9 |
none |
32 |
140 |
126 |
160 |
200 |
460 |
31.5 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
29.5 |
28.4 |
29 |
48 |
49 |
0.998 |
64 |
240 |
250 |
282 |
420 |
700 |
62.5 |
128 |
520 |
500 |
568 |
900 |
2360 |
122 |
n-gram
Speaker Diarization |
Language model |
# of streams |
Throughput (RTFX) |
---|---|---|---|
False |
n-gram |
32 |
430 |
False |
none |
32 |
430 |
True |
n-gram |
32 |
90 |
True |
none |
32 |
90 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
33.6 |
33.6 |
34.2 |
34.4 |
36 |
0.998 |
8 |
58 |
57 |
64 |
65 |
70 |
7.96 |
16 |
63 |
63 |
66 |
70 |
80 |
15.93 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
36 |
35.6 |
36.3 |
36.6 |
54 |
0.999 |
8 |
56 |
56 |
63 |
63 |
95 |
7.99 |
16 |
76 |
75 |
77 |
78 |
126 |
15.96 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
33 |
32.8 |
33.5 |
34 |
36.5 |
0.996 |
8 |
57 |
56 |
62 |
63 |
66 |
7.95 |
16 |
59 |
58 |
62 |
65 |
73 |
15.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
34.7 |
34.7 |
35.3 |
35.6 |
37.5 |
0.999 |
8 |
58.7 |
58.7 |
64 |
65 |
67 |
7.98 |
16 |
62.1 |
62 |
65.1 |
66 |
80 |
15.96 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
33 |
33 |
33.2 |
33.5 |
36.6 |
0.999 |
8 |
57.4 |
58 |
62.5 |
63.4 |
66 |
7.98 |
16 |
60 |
59.3 |
63 |
64 |
74 |
15.96 |
32 |
125 |
130 |
143 |
145 |
220 |
31.8 |
160
none
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
33.1 |
33 |
33.5 |
33.8 |
36 |
0.998 |
8 |
65 |
65 |
70 |
70 |
80 |
7.96 |
16 |
72 |
73 |
83 |
86 |
100 |
15.92 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
33 |
33.1 |
33.5 |
34 |
35.7 |
0.999 |
8 |
57 |
57 |
63 |
65 |
74 |
7.98 |
16 |
59 |
58 |
62 |
70 |
80 |
15.95 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
55 |
52 |
63 |
70 |
82 |
0.995 |
8 |
84 |
79.6 |
100 |
110 |
155 |
7.93 |
16 |
154 |
147 |
170 |
200 |
300 |
15.78 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
56.2 |
58.7 |
87.2 |
90 |
90 |
0.996 |
64 |
700 |
650 |
1000 |
1000 |
1400 |
60 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
200 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
26.7 |
26.65 |
26.9 |
27 |
30 |
0.999 |
8 |
57 |
57 |
61.5 |
62.4 |
65 |
7.98 |
16 |
59 |
58 |
61.4 |
63 |
80 |
15.96 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
32.6 |
32.5 |
33 |
35 |
35.2 |
0.999 |
64 |
220 |
251 |
270 |
277 |
300 |
63.3 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
560 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
40.5 |
36.8 |
62.2 |
65 |
65.6 |
0.99 |
8 |
60 |
52 |
80 |
95 |
110 |
7.88 |
16 |
82 |
75 |
110 |
140 |
150 |
15.64 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
37.4 |
36.3 |
40 |
40 |
40 |
0.994 |
64 |
320 |
350 |
370 |
400 |
400 |
60.5 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
400 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
28.6 |
28.2 |
31.6 |
32 |
32 |
0.993 |
8 |
60 |
55 |
80 |
80 |
90 |
7.84 |
16 |
74 |
70 |
100 |
110 |
110 |
15.6 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
32.8 |
32.7 |
33.3 |
34 |
36 |
0.997 |
8 |
57 |
60 |
60 |
60 |
70 |
7.95 |
16 |
60 |
59 |
64 |
70 |
70 |
15.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
32.5 |
31.8 |
32.4 |
33.8 |
57.8 |
0.998 |
8 |
56 |
56 |
61.5 |
63 |
110 |
7.97 |
16 |
59 |
57.4 |
61 |
64 |
120 |
15.94 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
61.7 |
59.4 |
61 |
70 |
129 |
0.996 |
4 |
3400 |
3300 |
5500 |
6000 |
7000 |
3.43 |
8 |
22500 |
22000 |
41000 |
42300 |
43500 |
3.43 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
100 |
100 |
120 |
120 |
130 |
0.996 |
4 |
175 |
172 |
176 |
200 |
230 |
3.97 |
8 |
352 |
350 |
352.6 |
390 |
418 |
7.9 |
n-gram
Speaker Diarization |
# of streams |
Throughput (RTFX) |
---|---|---|
False |
1 |
46 |
False |
4 |
50 |
False |
8 |
50.4 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
20.7 |
20 |
20.3 |
20.4 |
48 |
0.998 |
8 |
42 |
41.8 |
42.9 |
44 |
120 |
7.95 |
16 |
56 |
55 |
62 |
64 |
160 |
15.87 |
32 |
78 |
73 |
77 |
84 |
280 |
31.6 |
48 |
124 |
112 |
120 |
130 |
476 |
47.1 |
64 |
300 |
150 |
600 |
700 |
800 |
62.5 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
21.2 |
21.2 |
21.8 |
22 |
23.3 |
1 |
8 |
42.2 |
42.2 |
42.8 |
43 |
45.5 |
7.99 |
16 |
55 |
55 |
60 |
62 |
65 |
15.97 |
32 |
70.7 |
71 |
74 |
75.6 |
80 |
31.9 |
48 |
109 |
110 |
115 |
116 |
118 |
47.8 |
64 |
144 |
144 |
150 |
154 |
260 |
63.6 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
24.75 |
24.7 |
25 |
25 |
27 |
0.998 |
8 |
55 |
55 |
59 |
59.4 |
70 |
7.96 |
16 |
56 |
56 |
60 |
60 |
70 |
15.9 |
32 |
120 |
119 |
124 |
160 |
200 |
31.5 |
48 |
800 |
800 |
1500 |
1700 |
2000 |
45 |
64 |
4300 |
4500 |
7700 |
8000 |
8000 |
45.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
20.7 |
20.8 |
21 |
21.1 |
22.5 |
1 |
8 |
46 |
46 |
46.6 |
47 |
50 |
7.99 |
16 |
56 |
56 |
62 |
65 |
70 |
15.97 |
32 |
90 |
90 |
95 |
96 |
100 |
31.9 |
48 |
137 |
138 |
144 |
147 |
200 |
47.7 |
64 |
5000 |
4000 |
10000 |
12000 |
12000 |
59 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
25.15 |
25.1 |
25.3 |
25.4 |
27.4 |
1 |
8 |
55 |
55.6 |
58.4 |
58.7 |
60 |
7.98 |
16 |
58 |
57 |
62 |
64 |
70 |
15.96 |
32 |
121 |
120 |
123 |
124 |
170 |
31.8 |
48 |
1000 |
1000 |
2000 |
2000 |
3000 |
46.7 |
64 |
15000 |
15000 |
25000 |
30000 |
30000 |
46 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
23.7 |
22.3 |
27.6 |
38.2 |
46 |
1 |
8 |
55 |
53 |
60 |
75 |
103 |
7.98 |
16 |
70 |
70 |
80 |
100 |
128 |
15.96 |
32 |
120 |
120 |
130 |
150 |
184 |
31.85 |
48 |
210 |
190 |
300 |
318 |
340 |
47.6 |
64 |
8000 |
8000 |
14000 |
15000 |
16000 |
52 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
25 |
25 |
25.3 |
25.4 |
26.4 |
0.999 |
8 |
56 |
55.5 |
59 |
59.7 |
100 |
7.98 |
16 |
59 |
57.5 |
63 |
70 |
104 |
15.95 |
32 |
123 |
121 |
123 |
125 |
220 |
31.76 |
48 |
840 |
890 |
1200 |
1300 |
1400 |
46.5 |
64 |
11000 |
12000 |
20000 |
20000 |
20000 |
46.5 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
25.6 |
25.5 |
26.5 |
26.7 |
27 |
0.994 |
8 |
60 |
50 |
80 |
90 |
100 |
7.86 |
16 |
70 |
60 |
100 |
108 |
110 |
15.67 |
32 |
146 |
125 |
210 |
300 |
300 |
30.47 |
48 |
700 |
700 |
1200 |
1300 |
1400 |
41 |
64 |
2300 |
2500 |
3700 |
3800 |
4000 |
41.5 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
26.2 |
26 |
27 |
28 |
29.3 |
0.997 |
8 |
56 |
57 |
60 |
60 |
70 |
7.95 |
16 |
60 |
57.3 |
70 |
70 |
80 |
15.9 |
32 |
119 |
120 |
122 |
124 |
130 |
31.6 |
48 |
560 |
550 |
800 |
900 |
1000 |
46.2 |
64 |
4500 |
4700 |
8000 |
8200 |
8500 |
46.5 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
26.1 |
24.7 |
25.2 |
41.2 |
51.6 |
0.998 |
8 |
46 |
39 |
55 |
86 |
146 |
7.95 |
16 |
62 |
55.2 |
66.5 |
105 |
219 |
15.88 |
32 |
166 |
121 |
290 |
470 |
790 |
31.5 |
48 |
7800 |
7100 |
16100 |
17000 |
18300 |
41.6 |
64 |
18650 |
18500 |
29000 |
32000 |
35000 |
41.8 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
43.7 |
44.5 |
46 |
47 |
61 |
0.998 |
8 |
41.7 |
43.5 |
47.5 |
49 |
90 |
7.97 |
16 |
50 |
47 |
50 |
60 |
160 |
15.9 |
32 |
260 |
100 |
500 |
1200 |
2100 |
31.64 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.9 |
14.3 |
15.5 |
16.18 |
33 |
0.999 |
8 |
20 |
18.3 |
20.4 |
23.4 |
116 |
7.96 |
16 |
25.7 |
23.4 |
27 |
37 |
200 |
15.88 |
32 |
42 |
32.3 |
48 |
60 |
290 |
31.7 |
48 |
56 |
40 |
54 |
150 |
420 |
47.4 |
160
Language model |
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
|||
n-gram |
1 |
13.5 |
13.25 |
13.63 |
13.8 |
26.5 |
1 |
n-gram |
8 |
24 |
23 |
24 |
25 |
55 |
7.98 |
n-gram |
16 |
33 |
30 |
40 |
41 |
90 |
15.94 |
n-gram |
32 |
56 |
57 |
64 |
65 |
180 |
31.74 |
n-gram |
48 |
90 |
100 |
100 |
110 |
300 |
47.4 |
none |
1 |
12.3 |
12.14 |
12.3 |
12.4 |
19.5 |
1 |
none |
8 |
22 |
20.8 |
23 |
24 |
44 |
7.98 |
none |
16 |
27.1 |
26.16 |
27.2 |
30 |
63 |
15.95 |
none |
32 |
51 |
48 |
60 |
61 |
160 |
31.76 |
none |
48 |
86 |
90 |
96 |
100 |
287 |
47.45 |
none |
64 |
118 |
119 |
123 |
140 |
420 |
63 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15 |
14.7 |
15.4 |
22.6 |
25 |
1 |
64 |
140 |
140 |
150 |
220 |
380 |
63.2 |
128 |
228 |
220 |
256 |
400 |
720 |
125 |
256 |
455 |
440 |
510 |
810 |
2280 |
243.7 |
n-gram
Speaker Diarization |
Language model |
# of streams |
Throughput (RTFX) |
---|---|---|---|
False |
n-gram |
32 |
920 |
False |
none |
32 |
920 |
True |
n-gram |
32 |
90 |
True |
none |
32 |
90 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.5 |
13.4 |
14 |
14.5 |
17 |
0.999 |
8 |
26 |
25 |
30 |
30 |
33 |
7.98 |
16 |
40 |
40 |
50 |
50 |
50 |
15.95 |
32 |
68 |
73 |
80 |
81 |
83 |
31.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14 |
14 |
14.6 |
14.7 |
21.7 |
1 |
8 |
27 |
26 |
28 |
29 |
43 |
8 |
16 |
37 |
40 |
40 |
46 |
65 |
15.98 |
32 |
73 |
80 |
86 |
87 |
139 |
31.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.9 |
12.7 |
13.4 |
14 |
17 |
0.998 |
8 |
22 |
22 |
24 |
25 |
30 |
7.98 |
16 |
33 |
31 |
40 |
41 |
45 |
15.93 |
32 |
54 |
60 |
64 |
66 |
75 |
31.77 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.7 |
13.7 |
14.2 |
14.4 |
16 |
1 |
8 |
28 |
28 |
30.1 |
31 |
33 |
7.99 |
16 |
38 |
38.3 |
45 |
47 |
50 |
15.97 |
32 |
70 |
80 |
85 |
87 |
90 |
31.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13 |
12.9 |
13.2 |
13.3 |
17 |
1 |
8 |
23 |
23 |
24 |
27 |
30 |
7.99 |
16 |
31.5 |
30 |
40 |
40 |
43 |
15.97 |
32 |
60 |
64 |
70 |
70 |
74 |
31.9 |
48 |
95 |
103 |
108 |
110 |
140 |
47.8 |
160
none
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15.6 |
15.4 |
15.8 |
16 |
20 |
0.999 |
8 |
38 |
40 |
44 |
48 |
59 |
7.96 |
16 |
53 |
54 |
64 |
66 |
90 |
15.9 |
32 |
89 |
96 |
104 |
110 |
170 |
31.6 |
48 |
131 |
140 |
150 |
160 |
300 |
47 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.77 |
12.8 |
13.3 |
13.5 |
15.6 |
1 |
8 |
23 |
22.5 |
24 |
26 |
30 |
7.99 |
16 |
33.6 |
31.8 |
41 |
42 |
45 |
15.97 |
32 |
58 |
64 |
67 |
68 |
90 |
31.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
28 |
25 |
36 |
44 |
56 |
0.997 |
8 |
54 |
52 |
67 |
80 |
106 |
7.95 |
16 |
80 |
76 |
100 |
110 |
140 |
15.86 |
32 |
177 |
172 |
220 |
250 |
300 |
31.5 |
48 |
6300 |
6400 |
10000 |
10200 |
11000 |
34.3 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
39.5 |
42.4 |
72 |
74 |
74 |
0.997 |
64 |
330 |
344 |
410 |
440 |
500 |
62.3 |
128 |
700 |
680 |
1000 |
1200 |
1300 |
120 |
256 |
9000 |
9000 |
12000 |
13000 |
13000 |
150 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
500 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.9 |
12.8 |
13.3 |
13.4 |
16 |
1 |
8 |
23 |
22 |
30 |
30 |
30 |
7.99 |
16 |
31 |
29 |
39 |
39.6 |
41.6 |
15.97 |
32 |
56 |
63 |
66.3 |
67 |
70 |
31.9 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.7 |
13.6 |
15 |
16 |
16 |
1 |
64 |
120 |
130 |
140 |
146 |
154 |
63.7 |
128 |
210 |
220 |
250 |
260 |
300 |
126.6 |
256 |
364 |
413 |
467 |
477 |
500 |
251.4 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1300 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
15.7 |
14.8 |
18.5 |
22 |
22.2 |
0.997 |
8 |
29 |
27 |
35 |
39 |
40 |
7.95 |
16 |
43 |
40 |
50 |
70 |
80 |
15.84 |
32 |
89 |
86 |
150 |
180 |
200 |
31.2 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
17 |
16 |
20 |
20 |
20 |
0.997 |
64 |
180 |
190 |
205 |
210 |
210 |
62 |
128 |
340 |
350 |
400 |
400 |
400 |
121.3 |
256 |
580 |
630 |
720 |
740 |
800 |
231 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
790 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.3 |
14 |
17 |
17 |
18 |
0.996 |
8 |
30 |
30 |
30 |
40 |
40 |
7.93 |
16 |
50.5 |
51.5 |
60 |
70 |
74 |
15.75 |
32 |
87 |
90 |
130 |
140 |
150 |
31 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13 |
12.9 |
13.6 |
15 |
16 |
0.998 |
8 |
23.6 |
23 |
26 |
28 |
31.4 |
7.97 |
16 |
34 |
34 |
40.7 |
41.4 |
45 |
15.93 |
32 |
57 |
62.2 |
65 |
67 |
74 |
31.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.9 |
12.63 |
13.16 |
15 |
19.6 |
1 |
8 |
23 |
22 |
26 |
30 |
56 |
7.98 |
16 |
34 |
35 |
39.4 |
43 |
96 |
15.94 |
32 |
60.5 |
62.6 |
66.4 |
87 |
233 |
31.74 |
48 |
102 |
98 |
116 |
170 |
447 |
47.4 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
35 |
34 |
34.6 |
40 |
70 |
0.998 |
4 |
76 |
73 |
74 |
90 |
170 |
3.98 |
8 |
1300 |
1300 |
2000 |
2500 |
3000 |
7.5 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
36 |
35.8 |
36.7 |
44 |
46 |
0.999 |
4 |
75 |
74 |
80 |
90 |
100 |
3.99 |
8 |
156.4 |
154.5 |
158 |
180 |
200 |
7.95 |
n-gram
Speaker Diarization |
# of streams |
Throughput (RTFX) |
---|---|---|
False |
1 |
94 |
False |
4 |
115 |
False |
8 |
117 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.25 |
11.07 |
11.3 |
11.46 |
19.3 |
1 |
8 |
16 |
16 |
17 |
20 |
36 |
7.99 |
16 |
28 |
28 |
30 |
30 |
65 |
15.95 |
32 |
46 |
44 |
50 |
50 |
124 |
31.8 |
48 |
60 |
60 |
67 |
70 |
200 |
47.5 |
64 |
84 |
79 |
88 |
90 |
290 |
63.1 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10.8 |
10.8 |
11.3 |
11.5 |
12.9 |
1 |
8 |
16.3 |
16 |
19 |
20 |
20.85 |
8 |
16 |
26 |
26 |
28 |
29 |
34 |
16 |
32 |
43 |
42 |
50 |
50 |
55 |
31.95 |
48 |
60 |
60 |
66 |
70 |
70 |
47.9 |
64 |
70 |
70 |
80 |
84 |
90 |
63.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.9 |
11.87 |
12.3 |
12.6 |
14 |
0.999 |
8 |
20 |
20 |
22 |
23 |
27 |
7.98 |
16 |
38 |
40 |
45 |
46 |
46 |
15.94 |
32 |
67 |
68 |
72 |
73 |
76 |
31.76 |
48 |
112 |
110 |
126 |
140 |
170 |
47.3 |
64 |
200 |
170 |
300 |
300 |
370 |
62.5 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.5 |
11.5 |
11.8 |
11.9 |
13 |
1 |
8 |
18 |
17.5 |
22 |
23 |
24 |
8 |
16 |
26 |
25 |
32 |
32 |
34 |
15.98 |
32 |
50 |
50 |
53 |
53.6 |
56 |
31.94 |
48 |
78.6 |
79 |
84 |
86 |
90 |
47.9 |
64 |
100 |
100 |
110 |
114 |
130 |
63.7 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.1 |
12.04 |
12.3 |
12.5 |
14 |
1 |
8 |
20 |
20 |
30 |
30 |
30 |
7.99 |
16 |
30 |
27.4 |
38.4 |
39 |
40 |
15.98 |
32 |
70 |
70 |
70 |
70 |
80 |
31.9 |
48 |
118 |
120 |
126 |
130 |
150 |
47.75 |
64 |
180 |
172 |
220 |
300 |
340 |
63.5 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
14.6 |
13.2 |
19 |
30 |
37 |
1 |
8 |
33 |
32 |
38 |
53 |
76 |
7.99 |
16 |
50 |
50 |
56.3 |
70 |
93 |
15.98 |
32 |
84 |
80 |
100 |
110 |
130 |
31.9 |
48 |
120 |
120 |
137 |
150 |
180 |
47.8 |
64 |
153 |
150 |
190 |
220 |
270 |
63.5 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12 |
12 |
12.3 |
12.5 |
14 |
1 |
8 |
25 |
20 |
30 |
30 |
32 |
7.99 |
16 |
40 |
42 |
45 |
45 |
47 |
15.97 |
32 |
72 |
72 |
77 |
79.6 |
100 |
31.86 |
48 |
110 |
110 |
125 |
130 |
200 |
47.7 |
64 |
134 |
127 |
170 |
250 |
400 |
63.4 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.4 |
12.3 |
13.2 |
13.5 |
15 |
0.997 |
8 |
21 |
20.5 |
23 |
24 |
28 |
7.96 |
16 |
30 |
28 |
40 |
39 |
40 |
15.87 |
32 |
71 |
67 |
90 |
102 |
110 |
31.24 |
48 |
130 |
114 |
180 |
200 |
240 |
46 |
64 |
230 |
180 |
360 |
400 |
500 |
60.1 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.1 |
12 |
12.45 |
13 |
15 |
0.999 |
8 |
21 |
20 |
23 |
25 |
28 |
7.98 |
16 |
40 |
40 |
44 |
44 |
49 |
15.93 |
32 |
65 |
65 |
70 |
73 |
80 |
31.8 |
48 |
108 |
108 |
117 |
120 |
130 |
47.4 |
64 |
142 |
147 |
162 |
170 |
200 |
63 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.1 |
11.8 |
12.2 |
17.6 |
18.2 |
1 |
8 |
23 |
21 |
25 |
32 |
51 |
7.98 |
16 |
40 |
37.5 |
40 |
60 |
99 |
15.93 |
32 |
72 |
65 |
76.5 |
100 |
245 |
31.74 |
48 |
125 |
105 |
140 |
290 |
480 |
47.4 |
64 |
350 |
160 |
900 |
1280 |
1900 |
62.8 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
16.6 |
16.3 |
17.4 |
18.4 |
25 |
1 |
8 |
21.4 |
20.6 |
23 |
24.5 |
44 |
7.98 |
16 |
34 |
30 |
38 |
40 |
67 |
15.95 |
32 |
59 |
55.7 |
66 |
70 |
180 |
31.8 |
48 |
96.4 |
80 |
92 |
110 |
600 |
47.6 |
64 |
430 |
250 |
940 |
1700 |
2800 |
63 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
7 |
6.7 |
7.84 |
8.7 |
15.1 |
1 |
8 |
11 |
10 |
12 |
12.5 |
37 |
7.99 |
16 |
14 |
13 |
16 |
18 |
60 |
15.96 |
32 |
21.7 |
19.7 |
23.8 |
26.5 |
110 |
31.84 |
48 |
27 |
24 |
30 |
33 |
166 |
47.7 |
64 |
34 |
29 |
36 |
42 |
244 |
63.4 |
160
Language model |
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
|||
n-gram |
1 |
11 |
10.9 |
11.3 |
11.5 |
20 |
1 |
n-gram |
8 |
21 |
20.5 |
22 |
23 |
43 |
7.99 |
n-gram |
16 |
28 |
27 |
33 |
34 |
64 |
15.96 |
n-gram |
32 |
40 |
43 |
46 |
48 |
100 |
31.85 |
n-gram |
48 |
53 |
55 |
60 |
60 |
128 |
47.7 |
none |
1 |
10.4 |
10.3 |
10.4 |
10.5 |
15 |
1 |
none |
8 |
19 |
18.8 |
19.4 |
19.6 |
32 |
7.99 |
none |
16 |
22 |
22 |
23 |
23 |
41 |
15.97 |
none |
32 |
34.7 |
33 |
41 |
42 |
81 |
31.9 |
none |
48 |
47 |
50 |
54 |
54.7 |
103 |
47.7 |
none |
64 |
58 |
61 |
63 |
65 |
135 |
63.5 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.7 |
12.4 |
12.9 |
19.2 |
20 |
1 |
64 |
75 |
72 |
82 |
130 |
200 |
63.5 |
128 |
113 |
105 |
140 |
200 |
360 |
126.4 |
256 |
180 |
180 |
220 |
370 |
685 |
250 |
n-gram
Speaker Diarization |
Language model |
# of streams |
Throughput (RTFX) |
---|---|---|---|
False |
n-gram |
32 |
2100 |
False |
none |
32 |
2100 |
True |
n-gram |
32 |
120 |
True |
none |
32 |
110 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.2 |
11.1 |
11.8 |
11.96 |
14 |
1 |
8 |
22 |
22 |
24 |
26 |
30 |
7.99 |
16 |
29 |
29 |
34 |
35 |
37 |
15.96 |
32 |
40 |
42 |
50 |
50 |
56 |
31.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.3 |
11.2 |
11.7 |
12 |
17 |
1 |
8 |
21 |
20.5 |
23 |
24 |
34 |
8 |
16 |
28 |
30 |
33.7 |
34.4 |
45 |
15.98 |
32 |
43 |
47 |
49 |
50 |
70.4 |
31.95 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10.7 |
10.5 |
11 |
12 |
14 |
0.999 |
8 |
19.14 |
19 |
20 |
21 |
24 |
7.98 |
16 |
25 |
30 |
30 |
30 |
33 |
15.94 |
32 |
35 |
36 |
42 |
43 |
47 |
31.85 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.3 |
11.3 |
11.9 |
12 |
13.9 |
1 |
8 |
22 |
22 |
24 |
25 |
30 |
8 |
16 |
28.4 |
28.5 |
35 |
36 |
38 |
15.98 |
32 |
44 |
47 |
54 |
55 |
60 |
31.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11 |
10.9 |
11.3 |
11.4 |
13.7 |
1 |
8 |
18.2 |
17.8 |
20 |
20 |
24 |
8 |
16 |
23.5 |
23 |
28 |
29 |
31.7 |
15.98 |
32 |
35 |
39 |
41.15 |
41.9 |
48 |
31.95 |
48 |
48 |
51 |
54 |
55 |
64 |
47.9 |
160
none
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.2 |
13 |
13.4 |
14 |
17 |
0.999 |
8 |
26 |
26.3 |
28 |
30 |
37 |
7.98 |
16 |
37 |
37 |
45 |
50 |
60 |
15.92 |
32 |
56 |
60 |
70 |
72 |
100 |
31.75 |
48 |
80 |
85 |
96 |
100 |
150 |
47.4 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10.4 |
10.3 |
10.8 |
11 |
13 |
1 |
8 |
19.3 |
19.2 |
20 |
20 |
23 |
8 |
16 |
24 |
23 |
30 |
30.6 |
33 |
15.98 |
32 |
36 |
40 |
43 |
44 |
47 |
31.94 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
16.95 |
14.3 |
24.4 |
33 |
42 |
0.997 |
8 |
33 |
29.2 |
46 |
64.4 |
82 |
7.96 |
16 |
50 |
50 |
60 |
80 |
100 |
15.9 |
32 |
80 |
80 |
100 |
110 |
140 |
31.7 |
48 |
120 |
127 |
160 |
180 |
200 |
47.2 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
29 |
33 |
59 |
59 |
59 |
0.997 |
64 |
180 |
175 |
260 |
300 |
400 |
63 |
128 |
320 |
300 |
450 |
460 |
500 |
124.6 |
256 |
660 |
630 |
900 |
1000 |
1300 |
240 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
932 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10.46 |
10.4 |
10.7 |
10.9 |
13.6 |
1 |
8 |
18 |
17.4 |
20 |
20 |
23.6 |
7.99 |
16 |
23 |
22 |
28 |
29 |
32 |
15.98 |
32 |
35 |
38 |
40.8 |
42 |
50 |
31.94 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.6 |
11.4 |
13 |
14 |
15 |
1 |
64 |
64 |
70 |
76 |
77 |
80 |
63.8 |
128 |
90 |
93 |
110 |
120 |
130 |
127.4 |
256 |
160 |
175 |
213 |
220 |
240 |
253.7 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
2600 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.6 |
12 |
16 |
16.4 |
16.7 |
0.998 |
8 |
22.5 |
21.8 |
27 |
28.6 |
30 |
7.96 |
16 |
30.6 |
30 |
37 |
39 |
40 |
15.9 |
32 |
52.4 |
52 |
83 |
88.5 |
93 |
31.6 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
13.5 |
13 |
16 |
16 |
16 |
0.998 |
64 |
87 |
90 |
104 |
107 |
110 |
63 |
128 |
160 |
160 |
190 |
200 |
200 |
124 |
256 |
260 |
280 |
320 |
400 |
400 |
243 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1700 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12 |
11.8 |
14.7 |
15 |
16 |
0.996 |
8 |
22 |
22 |
25 |
27 |
30 |
7.94 |
16 |
31 |
30 |
37 |
40 |
43 |
15.87 |
32 |
50 |
50 |
75 |
85 |
90 |
31.44 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10.6 |
10.6 |
11.1 |
12 |
13.4 |
0.999 |
8 |
19.6 |
19.4 |
21 |
23 |
24 |
7.98 |
16 |
26.5 |
25 |
31.7 |
32.6 |
40 |
15.94 |
32 |
38 |
40 |
46 |
48 |
60 |
31.84 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
10.6 |
10.3 |
11 |
13 |
17 |
1 |
8 |
20 |
19.5 |
21 |
25 |
41 |
7.99 |
16 |
27 |
27 |
31.5 |
34 |
62 |
15.96 |
32 |
40 |
40 |
43 |
50 |
104 |
31.86 |
48 |
50 |
50 |
55 |
70 |
145 |
47.7 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
18.7 |
18 |
19.3 |
23 |
33 |
0.999 |
4 |
31.4 |
30 |
31.5 |
40 |
64 |
3.99 |
8 |
59 |
57 |
59 |
71 |
130 |
7.96 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
21 |
20.5 |
22 |
27 |
30 |
1 |
4 |
32.6 |
32 |
33.4 |
43 |
50 |
3.99 |
8 |
60 |
58 |
61 |
77 |
88 |
7.98 |
n-gram
Speaker Diarization |
# of streams |
Throughput (RTFX) |
---|---|---|
False |
1 |
194 |
False |
4 |
270 |
False |
8 |
280 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
8.7 |
8.6 |
8.8 |
9 |
13.6 |
1 |
8 |
15 |
15 |
16 |
16.3 |
28 |
7.99 |
16 |
20 |
20 |
23 |
24 |
46 |
15.96 |
32 |
32 |
32 |
33.6 |
35 |
80 |
31.9 |
48 |
40 |
40 |
44 |
46 |
100 |
47.7 |
64 |
53 |
51 |
57 |
60 |
140 |
63.5 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
8.34 |
8.4 |
8.7 |
8.9 |
10.1 |
1 |
8 |
17 |
17 |
18.3 |
19 |
23 |
8 |
16 |
23 |
23.5 |
26 |
27 |
30 |
16 |
32 |
35 |
36 |
38 |
38 |
40 |
31.96 |
48 |
46 |
46 |
52 |
53.1 |
56 |
47.9 |
64 |
55 |
56 |
62 |
64 |
70 |
63.9 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9.4 |
9.4 |
9.7 |
10 |
10 |
0.999 |
8 |
16.5 |
16.4 |
18 |
18 |
20 |
7.99 |
16 |
20 |
20 |
23 |
26 |
28 |
15.96 |
32 |
37 |
37.6 |
40 |
40 |
42 |
31.9 |
48 |
49 |
49 |
53 |
55 |
60 |
47.7 |
64 |
59.7 |
60.4 |
65 |
66 |
70 |
63.6 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9.1 |
9.1 |
9.5 |
9.6 |
10.7 |
1 |
8 |
15.8 |
15.8 |
17 |
18 |
20 |
8 |
16 |
24 |
25 |
26 |
27 |
30 |
16 |
32 |
34 |
34 |
37 |
39 |
42 |
31.95 |
48 |
46 |
46 |
50 |
52 |
60 |
47.9 |
64 |
54 |
55 |
60 |
60 |
70 |
63.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9.54 |
9.5 |
9.66 |
9.7 |
12 |
1 |
8 |
17 |
16.4 |
18 |
20 |
20 |
8 |
16 |
22 |
22 |
25 |
27 |
28 |
16 |
32 |
37 |
38 |
40 |
41 |
47 |
31.95 |
48 |
48 |
49 |
52 |
54 |
60 |
47.9 |
64 |
59.5 |
60 |
64.6 |
66 |
70 |
63.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
12.4 |
11 |
16.6 |
27.5 |
34.6 |
1 |
8 |
20 |
18 |
24.7 |
42 |
67 |
8 |
16 |
28 |
26 |
33 |
53 |
76 |
15.98 |
32 |
47 |
44 |
60 |
70 |
100 |
31.94 |
48 |
61 |
60 |
70 |
95 |
114 |
47.9 |
64 |
75 |
73 |
86 |
100 |
142 |
63.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9.2 |
9.2 |
9.6 |
9.8 |
10.6 |
1 |
8 |
17 |
16.4 |
18 |
19 |
20 |
8 |
16 |
23 |
23 |
26.7 |
27.4 |
30 |
15.98 |
32 |
36.6 |
37 |
39.5 |
40.5 |
43 |
31.95 |
48 |
48 |
49 |
52 |
53 |
56 |
47.9 |
64 |
60 |
60.5 |
65 |
67 |
82 |
63.8 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9.7 |
9.6 |
10 |
10 |
10 |
0.997 |
8 |
17.4 |
17 |
19 |
20 |
23 |
7.96 |
16 |
22.3 |
22.3 |
23.7 |
27 |
30 |
15.9 |
32 |
43 |
45 |
50 |
60 |
60 |
31.6 |
48 |
60 |
60 |
80 |
90 |
100 |
47 |
64 |
80 |
76 |
110 |
120 |
120 |
62.3 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9.6 |
9.5 |
10 |
10 |
12 |
0.999 |
8 |
18 |
17 |
20 |
20.7 |
24 |
7.98 |
16 |
28 |
28 |
30.4 |
31 |
35 |
15.95 |
32 |
38 |
39 |
42 |
45 |
50 |
31.85 |
48 |
49.4 |
50.5 |
56 |
58 |
65 |
47.7 |
64 |
60 |
62 |
69.4 |
71.6 |
81 |
63.5 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
9.3 |
9.1 |
9.5 |
12.8 |
13.2 |
1 |
8 |
17.4 |
16.5 |
20 |
22.7 |
34 |
7.99 |
16 |
27.3 |
26.6 |
28.6 |
36 |
60 |
15.96 |
32 |
40 |
38 |
42 |
57 |
100 |
31.87 |
48 |
52 |
49 |
54 |
81 |
141.5 |
47.7 |
64 |
64 |
60 |
66 |
100 |
220 |
63.5 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
11.1 |
10.8 |
12 |
13 |
18 |
1 |
8 |
16.7 |
16 |
19 |
20 |
31 |
7.99 |
16 |
23.7 |
23 |
27 |
28.3 |
46 |
15.97 |
32 |
37 |
36 |
41 |
42.5 |
75 |
31.9 |
48 |
46 |
45 |
51 |
53.6 |
99 |
47.8 |
64 |
57 |
55 |
61 |
65 |
170 |
63.6 |
100
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
6.9 |
6.6 |
7.6 |
8.4 |
14 |
1 |
8 |
9.2 |
8.7 |
10 |
11 |
27 |
7.99 |
16 |
13 |
12.4 |
15 |
17 |
41 |
15.97 |
32 |
17.5 |
17 |
20 |
23 |
68 |
31.9 |
48 |
23 |
22 |
28 |
30 |
94 |
47.8 |
64 |
27 |
25 |
30 |
36 |
120 |
63.7 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
27.6 |
29.14 |
29.95 |
53.7 |
28.16 |
0.995 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
26.64 |
28.2 |
31.86 |
37.6 |
26.93 |
0.994 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
25.63 |
26.66 |
27.36 |
33.75 |
25.53 |
0.997 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
25.43 |
27 |
34.16 |
35.2 |
26.43 |
0.99 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
27.4 |
28.7 |
29.4 |
40.6 |
27.4 |
0.997 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
25.6 |
26.3 |
26.5 |
33.9 |
25.86 |
0.997 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
24.4 |
24.84 |
24.93 |
47.6 |
24.9 |
0.997 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
26.2 |
27 |
29.7 |
40.2 |
26.27 |
0.995 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
41.6 |
42.24 |
45.1 |
61.2 |
42.4 |
0.99 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
23.93 |
24.5 |
24.6 |
33.05 |
24.15 |
0.997 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
29.1 |
40.8 |
44.6 |
44.6 |
31.3 |
0.98 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
26.5 |
41.05 |
41.05 |
41.05 |
29 |
0.974 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
26.4 |
28.1 |
33.7 |
33.7 |
26.94 |
0.989 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
25.2 |
26.3 |
29.36 |
39.4 |
25.55 |
0.996 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
18.37 |
19.05 |
19.36 |
29.44 |
18.73 |
0.997 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
18.07 |
18.86 |
18.9 |
27.8 |
18.1 |
0.998 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
22.7 |
23.38 |
25.56 |
32.5 |
23.2 |
0.993 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
18.47 |
19.66 |
21.44 |
29.3 |
18.66 |
0.998 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
22.75 |
23.17 |
23.66 |
32.16 |
22.98 |
0.997 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
19.14 |
34 |
52.7 |
67.6 |
22.96 |
0.998 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
23.97 |
25 |
27.27 |
34.8 |
24.03 |
0.996 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
23.57 |
33.93 |
33.93 |
33.93 |
24.86 |
0.981 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
23.73 |
24.7 |
30.45 |
32.3 |
24.06 |
0.991 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
22.4 |
23.24 |
24.5 |
32.9 |
22.6 |
0.997 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
40.9 |
44.4 |
48.1 |
184 |
44 |
0.992 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
41.6 |
100 |
184.6 |
189.6 |
52.8 |
0.992 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
37.8 |
40.55 |
44.4 |
50.7 |
37.7 |
0.989 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
64.4 |
592 |
713 |
754 |
192.5 |
0.98 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
38.5 |
39.8 |
40.5 |
58.4 |
38.9 |
0.994 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
36.1 |
37.9 |
39.3 |
73.6 |
37.06 |
0.995 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
201.8 |
888 |
1032 |
1130 |
290 |
0.994 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
73.3 |
76 |
79.7 |
136.6 |
76 |
0.952 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
34.64 |
35.4 |
36.25 |
47.6 |
34.96 |
0.996 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
45.5 |
72.4 |
77.7 |
77.7 |
50.4 |
0.967 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
2573 |
2806 |
2806 |
2806 |
2360 |
0.3496 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
41.4 |
50.4 |
52.1 |
54.7 |
42.2 |
0.984 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
37.8 |
169.7 |
325 |
570 |
70.4 |
0.986 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
29.77 |
31.9 |
32.5 |
78.1 |
30.94 |
0.995 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
31.44 |
192.5 |
227 |
306 |
69.3 |
0.996 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
40.24 |
188.7 |
189.4 |
254.6 |
90.7 |
0.965 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
35.6 |
37.6 |
41.3 |
52.8 |
36.3 |
0.995 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
1790 |
3250 |
3410 |
3600 |
1598 |
0.769 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
718 |
1560 |
1630 |
1843 |
742 |
0.978 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
1007 |
1260 |
1260 |
1260 |
847 |
0.711 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
37.7 |
48.9 |
55.7 |
176.7 |
44.9 |
0.986 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
34.6 |
54.8 |
183 |
432 |
51.8 |
0.994 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
14.2 |
14.87 |
14.96 |
99 |
15.45 |
0.998 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
14.4 |
15.97 |
17 |
99.8 |
17.36 |
0.997 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
13.48 |
14.38 |
14.8 |
102.8 |
14.8 |
0.999 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
13.65 |
15.34 |
17.65 |
99.3 |
17.64 |
0.995 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
14.16 |
14.8 |
15 |
93.2 |
15.52 |
0.998 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
14.1 |
14.55 |
14.83 |
93.7 |
15.23 |
0.998 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
15.3 |
16.17 |
16.66 |
96.4 |
16.78 |
0.998 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
13.97 |
15.1 |
17.28 |
98.8 |
15.5 |
0.998 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
24.3 |
25.75 |
62 |
219 |
32.7 |
0.994 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
13.26 |
14.38 |
14.97 |
192.7 |
16.73 |
0.998 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
16.1 |
20.7 |
98 |
98 |
21.95 |
0.99 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
14.43 |
97 |
97 |
97 |
23.05 |
0.987 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
13.23 |
14.88 |
17.96 |
155.8 |
19.73 |
0.995 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
13.46 |
14.33 |
16.06 |
102 |
14.92 |
0.998 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
11.15 |
12.1 |
12.56 |
187 |
14.06 |
0.999 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
11.15 |
11.96 |
12.31 |
103 |
12.6 |
0.999 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
13.2 |
14.22 |
14.44 |
94 |
16.82 |
0.995 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
11.35 |
12.02 |
12.66 |
91.2 |
12.73 |
0.999 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
13.26 |
14.07 |
14.28 |
107.1 |
15.36 |
0.998 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
13.16 |
20.66 |
36.24 |
96.5 |
16.22 |
0.998 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
12.77 |
13.44 |
14.02 |
100.8 |
14.3 |
0.998 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
13.03 |
93.8 |
93.8 |
93.8 |
20.98 |
0.99 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
13.63 |
14.98 |
17.28 |
93.8 |
17.24 |
0.995 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
p50 |
p90 |
p95 |
p99 |
avg |
||
1 |
12.98 |
13.47 |
13.72 |
98 |
14.2 |
0.999 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
50 |
49.6 |
50.1 |
50.4 |
100 |
0.997 |
8 |
70 |
70 |
80 |
80 |
130 |
7.96 |
16 |
100 |
100 |
111 |
114 |
230 |
15.87 |
32 |
300 |
280 |
450 |
500 |
660 |
31.4 |
48 |
12100 |
13000 |
23200 |
24000 |
24700 |
34.2 |
64 |
22000 |
24500 |
38400 |
39100 |
39700 |
34 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
52.6 |
52 |
53 |
64.3 |
67 |
0.998 |
64 |
300 |
320 |
340 |
550 |
700 |
62.7 |
128 |
580 |
570 |
680 |
900 |
2000 |
123.2 |
256 |
17000 |
19000 |
30000 |
30000 |
38000 |
154 |
384 |
31000 |
32600 |
50000 |
53000 |
62600 |
155.5 |
512 |
45500 |
47000 |
71000 |
73000 |
86800 |
159 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
423 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
22.3 |
21.76 |
22 |
22 |
42.6 |
0.999 |
8 |
50 |
48 |
50 |
60 |
100 |
7.97 |
16 |
57 |
56 |
60 |
63 |
125 |
15.9 |
32 |
100 |
95 |
100 |
100 |
280 |
31.7 |
48 |
170 |
145 |
200 |
330 |
500 |
47.3 |
64 |
10000 |
10000 |
19000 |
20000 |
20400 |
52.5 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
24 |
23.4 |
23.8 |
32 |
34 |
0.999 |
64 |
185 |
180 |
190 |
300 |
370 |
63.2 |
128 |
360 |
370 |
386 |
510 |
740 |
124.8 |
256 |
1300 |
800 |
1800 |
2200 |
12000 |
240 |
384 |
17000 |
18000 |
27400 |
27800 |
34700 |
246 |
512 |
28000 |
32000 |
40500 |
41000 |
50000 |
250 |
768 |
45000 |
50000 |
68000 |
70000 |
80000 |
256 |
n-gram
# of streams |
Throughput (RTFX) |
---|---|
32 |
1100 |
160
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
49 |
48.6 |
49 |
49 |
100 |
0.997 |
8 |
70 |
80 |
85 |
85 |
120 |
7.97 |
16 |
90 |
100 |
100 |
102 |
200 |
15.9 |
32 |
300 |
286 |
400 |
485 |
780 |
31.5 |
48 |
10000 |
11000 |
21000 |
22000 |
23000 |
36.6 |
64 |
20300 |
23600 |
35400 |
36400 |
37000 |
37 |
800
n-gram
# of streams |
Latency (ms) |
Throughput (RTFX) |
||||
---|---|---|---|---|---|---|
avg |
p50 |
p90 |
p95 |
p99 |
||
1 |
52 |
51.2 |
52 |
63.3 |
65 |
0.998 |
64 |
300 |
300 |
310 |
400 |
500 |
63 |
128 |
520 |
550 |
600 |
790 |
1500 |
124 |
256 |
14000 |
15000 |
25000 |
26000 |
35000 |
172 |
384 |
27000 |
27000 |
44000 |
48000 |
58000 |
174 |
512 |
39000 |
38000 |
6 |