Nemotron Transcribe API#
Nemotron Transcribe exposes a JSON Chat Completions route and multipart audio routes for offline inference.
All inference routes use one transcription pipeline.
/v1/chat/completions returns JSON with the structured result in the top-level transcription field.
The multipart routes return the OpenAI audio formats selected by response_format: a json or verbose_json body, or plain text, SRT, or VTT; refer to Multipart Requests.
For deployment and executable examples, refer to Nemotron Transcribe.
This page is the API reference for Nemotron Transcribe.
The ASR HTTP REST API reference describes the other ASR NIMs and does not apply to Nemotron Transcribe, although some routes and the port are the same.
Method and Endpoint |
Purpose |
|---|---|
|
Discover the model ID and the declared source languages. |
|
Submit audio using a Chat Completions message. |
|
Submit an audio file as a multipart form for transcription. |
|
Submit an audio file as a multipart form for translation. |
|
Check whether the NIM is ready for inference. |
|
Check NIM liveness. |
|
Prometheus metrics for monitoring, in the Prometheus text format. |
|
The NIM release and API version, for example |
|
NIM metadata: the license, the model assets, and the selected model profile. |
|
The NIM license text. |
|
The NIM model manifest: the available model profiles and their tags. |
The health, metrics, version, metadata, license, and manifest routes are the standard NIM management endpoints and take no request body.
Note
Every inference route processes a complete recording.
stream: true streams the generated text after the whole recording is uploaded; it does not accept live or incremental audio.
This NIM does not implement the Riva gRPC API or the realtime WebSocket API.
Chat Completions Request#
Replace <model-id> with the ID returned by /v1/models and <base64-audio> with the base64-encoded audio file.
Send one user message containing one audio part and, optionally, one text instruction of up to the text limit (refer to Limits).
ASR options such as language and word_timestamps are top-level request fields.
{
"model": "<model-id>",
"messages": [{
"role": "user",
"content": [
{"type": "text", "text": "Produce a verbatim transcript of the audio. Preserve named entities, abbreviations, numbers, dates, measurements, acronyms, and technical terms as clearly as possible. Keep the same language as spoken and do not translate."},
{"type": "input_audio", "input_audio": {
"data": "<base64-audio>", "format": "wav"
}}
]
}],
"language": "en",
"word_timestamps": true
}
Read the transcript from choices[0].message.content and the structured result from the top-level transcription field.
With the OpenAI Python SDK, pass these options in extra_body; with the JavaScript SDK, add them to the request object.
The structured result is the top-level transcription field of the completion, next to choices; refer to OpenAI SDK Examples.
For plain transcription, omit the text part; the NIM then uses its default transcription prompt.
To send your own prompt and still get a transcript, use Transcribe the audio.
Note
A text part replaces the default transcription prompt; it is not added to it.
The model follows the text as an instruction, and transcription is not enforced: an instruction that asks for something other than a transcript can make the model answer or act on the speech instead of transcribing it.
Do not combine transcription with another task in one instruction; send separate requests or use Multi-turn Mode.
When you send a text part, language does not add a language hint to the prompt; refer to Language.
This applies to Chat Completions requests; the multipart routes do not accept a prompt and always use the NIM’s prompts.
Multipart Requests#
The /v1/audio/transcriptions and /v1/audio/translations routes take a multipart form with one file field and an optional model field.
curl http://localhost:9000/v1/audio/transcriptions \
-F file=@audio.wav -F response_format=verbose_json \
-F 'timestamp_granularities[]=word'
Form Field |
Values and Behavior |
|---|---|
|
Required. One audio file in a supported format. |
|
Language hint for the default prompt; not added with |
|
Transcriptions: |
|
Translations only; BCP 47 tag, default English. Refer to Translation for the supported pairs. A language outside the |
|
Transcriptions only. |
|
Boolean. On transcriptions, requests word alignment ( |
|
Boolean. On transcriptions, |
|
Positive integer; requires |
|
Same meaning as on |
|
Boolean; streams the transcript as server-sent events in the multipart event format. Refer to Streaming Output. |
The translation route translates into English by default; set target_language to translate English audio into another language.
The multipart prompt and speech_contexts fields are rejected when nonempty.
Use /v1/chat/completions to send a transcription instruction.
profanity_filter=true returns 400.
automatic_punctuation and verbatim are accepted and ignored: the transcript keeps the model’s punctuation, and no inverse text normalization is applied.
Boolean form fields take exactly true or false; any other value, such as 1 or True, returns 400.
With diarization=true, the NIM uses its own speaker prompt, which returns lowercase text without punctuation and numbers as spoken words.
The speaker-attributed Chat Completions prompt in the deployment guide returns punctuated text with numbers in standard written form, so the two routes give different output styles for the same recording.
OpenAI API Compatibility#
The inference routes follow the OpenAI Chat Completions and Audio APIs, with the differences below.
Unknown fields return 400 on every route.
Route |
Supported OpenAI Fields |
Rejected With |
Nemotron Transcribe Extensions |
|---|---|---|---|
|
|
|
|
|
|
Nonempty |
|
|
|
Nonempty |
|
task: "translate" on /v1/chat/completions returns 400; request translation with an instruction or use /v1/audio/translations.
The OpenAI SDKs send none of the rejected fields unless you set them.
Do not pass max_tokens, max_completion_tokens, top_p, stop, tools, logprobs, the penalty fields, or a temperature above 1; each returns 400.
OpenAI SDK Examples#
The following examples transcribe audio.wav with the OpenAI SDKs, first through Chat Completions with word timestamps and then through the multipart transcription route.
The NIM does not check the API key, but the SDKs require a value.
import base64
from openai import OpenAI
client = OpenAI(base_url="http://localhost:9000/v1", api_key="not-used")
model = client.models.list().data[0].id
with open("audio.wav", "rb") as f:
audio_b64 = base64.b64encode(f.read()).decode()
completion = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": [
{"type": "input_audio", "input_audio": {"data": audio_b64, "format": "wav"}},
]}],
extra_body={"language": "en", "word_timestamps": True},
)
print(completion.choices[0].message.content)
print(completion.transcription["words"][:3]) # Nemotron Transcribe extension field
with open("audio.wav", "rb") as f:
result = client.audio.transcriptions.create(
model=model,
file=f,
language="en",
response_format="verbose_json",
timestamp_granularities=["word"],
)
print(result.text, result.words[:3])
import fs from "node:fs";
import OpenAI from "openai";
const client = new OpenAI({ baseURL: "http://localhost:9000/v1", apiKey: "not-used" });
const model = (await client.models.list()).data[0].id;
const audio = fs.readFileSync("audio.wav").toString("base64");
const completion = await client.chat.completions.create({
model,
messages: [{ role: "user", content: [
{ type: "input_audio", input_audio: { data: audio, format: "wav" } },
] }],
language: "en", // Nemotron Transcribe extension fields go in the request object
word_timestamps: true,
});
console.log(completion.choices[0].message.content);
console.log(completion.transcription.words.slice(0, 3));
const result = await client.audio.transcriptions.create({
model,
file: fs.createReadStream("audio.wav"),
language: "en",
response_format: "verbose_json",
timestamp_granularities: ["word"],
});
console.log(result.text, result.words.slice(0, 3));
In TypeScript, the extension fields and completion.transcription need a type cast, because the SDK types do not declare them.
The transcription object and the multipart verbose_json body are described in Structured Result.
The examples keep stream off: a streamed response also carries Nemotron Transcribe events, such as transcription.transcript.replace, that the SDKs might not surface; refer to Streaming Output.
Audio Input#
Nemotron Transcribe supports WAV, FLAC, and Ogg Opus audio. Use mono, 16 kHz PCM WAV for the best results; the server converts other WAV sample rates, bit depths, and channel layouts to mono 16 kHz before inference. Multichannel audio is averaged into one channel. Channels in opposite phase cancel: a stereo file whose right channel is the inverted left channel becomes silence and returns an empty transcript. For such recordings, send one channel.
Other audio and video formats are not supported.
An upload in another format, or one that is not audio, returns 422 with the message unsupported audio format: send WAV, FLAC or Ogg Opus.
A supported container that reaches the backend but cannot be decoded returns 422 with the message invalid or unsupported audio file.
Malformed WAV structure can instead be rejected by the frontend with 400 before backend decoding.
Convert unsupported formats to WAV before sending them.
On /v1/chat/completions, input_audio.format is required and must contain 1–16 lowercase ASCII letters or digits, such as wav.
A missing, null, empty, or malformed value such as WAV2 returns 400.
The server identifies the audio container from the file contents, not from this field.
A format value that does not match the contents is not rejected; the file is accepted or refused by its contents.
For example, valid WAV bytes declared with another format name can succeed; the declared name does not add support for that format.
Invalid base64 returns 400; valid base64 containing random bytes or text is non-audio content and returns the unsupported-format 422 described above.
Limits#
Limit |
Value |
|---|---|
Recording duration |
3,600 seconds per request |
Request body |
256 MiB, including base64 encoding on |
Response body |
64 MiB |
History messages |
62 with Multi-turn Mode on; not applicable off ( |
Instruction and conversation text |
120,000 characters with Multi-turn Mode on; 32,000 off ( |
A request over the body or duration limit returns 413.
On /v1/chat/completions the audio is base64-encoded, so the largest audio file is about 190 MiB.
A one-hour mono, 16 kHz, 16-bit WAV file (about 115 MB) fits; a one-hour 44.1 kHz stereo WAV file does not, so downmix and resample it to mono 16 kHz before sending.
When the server is at capacity, it returns 429 (rate_limit_error, code overloaded): retry with backoff and lower client concurrency.
A request that waits in the admission queue past the timeout returns 503 with code admission_timeout.
Request Options#
The following fields apply to /v1/chat/completions.
Field |
Values and Behavior |
|---|---|
|
Model ID from |
|
Base64-encoded audio file, not a file path or URL. |
|
Required string of 1–16 lowercase ASCII letters or digits, such as |
|
Finite number from |
|
|
|
Optional language hint. Refer to Language. |
|
|
|
Boolean, default |
|
List of |
|
|
|
Nonnegative integer. |
|
|
null for an optional field is treated as if the field were omitted.
This does not apply to required fields such as input_audio.data and input_audio.format.
For example, "word_timestamps": null requests no timestamps, and "temperature": null uses the default temperature.
Values of the wrong type, such as "word_timestamps": "yes" or "word_timestamps": 1, return 400.
Unknown fields and duplicate top-level fields return 400.
stream_options without stream: true returns 400.
The Chat Completions route also rejects tools, tool_choice other than none, audio output, modalities other than ["text"], n other than 1, response_format other than {"type": "text"}, stream_options other than include_usage, and token-limit overrides (max_tokens and max_completion_tokens).
It accepts and ignores user and metadata.
Text in any message must not contain the model’s chat-template control tokens or other special tokens, such as </s>, [INST], or <think>; such a request returns 400 naming the token.
Speaker and event tags such as <spk:0> are allowed.
A request is one user message.
Conversation history and system messages are available only when the deployment enables Multi-turn Mode.
Language#
language is a hint, not a language selector.
Omit it, or set it to auto, to let the model identify the spoken language.
Request |
Effect of |
|---|---|
No instruction (default prompt) |
The server adds a sentence naming the language to the default prompt. |
A client instruction |
No effect on the prompt. State the language in your instruction instead, for example |
|
No effect on the prompt. The server uses its speaker prompt, which names no language, and does not add the language sentence. |
|
Names the source language in the NIM’s translation prompt. |
language must be a language from the supported_languages list returned by /v1/models: the transcribe list on every route except /v1/audio/translations, which uses the translate list.
The server compares the primary subtag, so en, en-US, and zh-CN are all accepted when en and zh are listed.
Any other value, such as xx or english, returns 400 with param set to language and the accepted codes in the message.
The response’s transcription.language field echoes the request value, or und when it is omitted; it is not a detected language.
For non-English audio, the default prompt with language set gives better language fidelity than a custom instruction without a language.
The model can still answer in the wrong language or translate, especially for short recordings and low-resource languages; review output on representative recordings.
Translation#
Request translation with an instruction, for example Translate the audio into English. in the message’s text part.
The response keeps transcription.task as transcribe.
The multipart /v1/audio/translations route uses the NIM’s translation prompt; when you set language, the prompt names that source language, and when you set target_language, it names that target.
Supported translation pairs, on both /v1/audio/translations and /v1/chat/completions:
Any language in the
translatelist ofsupported_languagesinto English.English into any language in the
translatelist.
No other translation pairs are supported.
The server does not refuse them: /v1/audio/translations checks only that language and target_language are each in the translate list, so a request such as German into French is accepted but is not supported.
On /v1/chat/completions, the language pair in a translation instruction is not checked.
On /v1/audio/translations, the language field of a verbose_json response is the requested target language (English when target_language is omitted); it is not a detected language.
Word timestamps and subtitles apply to the source speech, not to translated text, so they should not be requested with translation.
/v1/audio/translations returns 400 for word_timestamps=true; word_timestamps=false is accepted.
/v1/chat/completions does not refuse word_timestamps: true with a translation instruction: it returns 200 with word timestamps, but they are not an alignment of the translated text.
Reasoning#
Reasoning is off by default.
Set reasoning_effort to low, medium, or high to turn it on; none, null, or omitting the field keeps it off, and any other value returns 400.
The multipart routes do not support reasoning and return 400 for reasoning fields.
Effort |
Maximum Reasoning Tokens |
|---|---|
|
1,024 |
|
4,096 |
|
16,384 |
Keep reasoning off for transcription, timestamps, translation, speaker tags, and multi-turn follow-ups.
With reasoning on, the model tends to answer or follow requests spoken in the audio instead of transcribing it.
In NVIDIA testing at medium effort, transcription requests were answered rather than transcribed, latency rose about 15 times, and speaker-tagged transcription usually returned an empty transcript.
Do not enable reasoning for audio from untrusted speakers.
When enabled, the reasoning text is returned in choices[0].message.reasoning, never in the transcript, and its token count is in usage.completion_tokens_details.reasoning_tokens.
If the reasoning does not finish, the transcript is empty, finish_reason is length, and transcription.truncated_reason is reasoning; no timestamps are produced for that request.
Structured Result#
The transcription object includes schema_version, task, language, duration, text, segments, and transcript_origin.
transcript_origin distinguishes llm, llm_fallback, and ctc_fallback output.
Inspect it when your application needs to detect degraded transcription.
truncated_reason is present only when the returned text is not the whole output.
Value |
When |
|
|---|---|---|
|
The transcript kept repeating itself; the server stopped it and kept one copy of the repeated text, so the transcript covers the audio only up to the stop. |
|
|
Reasoning was on and never finished; the text is empty and no timestamps are produced. |
|
|
The output reached its token limit. |
|
|
Multi-turn Mode only: an answer that kept repeating itself was trimmed, or the server’s repetition guard ended it. |
|
Conversation turns report truncation in transcription.conversation instead; refer to Multi-turn Mode.
JSON responses with uploaded audio include usage: usage.type is duration and usage.seconds is the decoded audio duration in seconds, including silence.
This applies to /v1/chat/completions and to the multipart json and verbose_json formats.
When a decode ran, usage also has prompt_tokens, completion_tokens, and total_tokens, summed over any retries, and prompt_tokens_details.audio_tokens, the audio’s token count.
With reasoning on, completion_tokens_details.reasoning_tokens reports the reasoning tokens.
On /v1/audio/transcriptions, requesting word timestamps with response_format json (the default) or text returns verbose_json instead and adds a warnings list, for example [{"code": "response_format_overridden", "requested": "json", "returned": "verbose_json"}].
Request response_format=verbose_json explicitly to avoid the warning.
Timestamp requests add the following fields.
Field |
Meaning |
|---|---|
|
Word records with |
|
|
|
Words grouped by |
|
Diarization timing information, with frame duration given by |
|
Caption timing cues. |
|
Requested subtitle |
Check alignment status before reading word timings.
A successful transcript can have unavailable alignment, for example because the transcript is empty or CTC evidence is unavailable.
Requesting subtitles without usable alignment returns 422.
Speaker labels are local to the recording and do not identify a person.
Streaming Output#
Nemotron Transcribe streams its output, not its input.
Set stream: true on /v1/chat/completions, or stream=true on the multipart routes, to receive the transcript as server-sent events while it is generated.
The request still carries the complete recording: the NIM starts generating only after the whole upload has arrived, and it does not accept live, chunked, or incremental audio.
For real-time transcription of live audio, use a streaming ASR model such as Nemotron ASR Streaming or Parakeet CTC.
Event |
|
|
|---|---|---|
Append text |
|
|
Replace text |
|
|
Success |
Final chunk with |
|
Failure after headers |
|
|
On the multipart routes, the stream ends after transcript.text.done or error; do not wait for [DONE].
While the NIM queues, encodes, or aligns, the stream carries keepalive comments with no text.
Streams cannot be resumed: if the connection closes before the final event, treat the request as failed and resend it.
Closing the connection cancels the request’s work on the server.
With reasoning on, Chat Completions streams the reasoning in choices[0].delta.reasoning before the answer deltas.
Streamed deltas are provisional.
When the server corrects the transcript after the first decode, it sends transcription.transcript.replace with the complete corrected text; replace the displayed transcript instead of appending.
The final transcription.text is authoritative.
Word timestamps arrive only in the final transcription payload, and subtitle_format with stream: true returns 400.
On /v1/chat/completions, stream_options: {"include_usage": true} adds a usage chunk before [DONE].
Multi-turn Mode#
Multi-turn Mode lets a client ask follow-up questions about one recording without re-encoding the audio on every turn. It is off by default and must be enabled when the NIM is deployed. Refer to Multi-turn conversations for deployment and restrictions.
With Multi-turn Mode on, /v1/chat/completions accepts conversation history as alternating user and assistant turns, one optional leading system message on instruction turns (a system message with a transcription turn returns 400), and the following top-level fields.
Field |
Turn |
Meaning |
|---|---|---|
|
First turn |
Opens a conversation. The response carries |
|
Later turns |
The |
|
Last turn |
Ends the conversation; the server releases what it cached for it after the response. |
Only the first user turn carries audio; input_audio in any later message returns 400.
The first user message’s content is a list of parts: one input_audio part and at most one text part.
Text-only messages take content as a string or as a list of text parts.
On every later turn, resend the whole conversation: the first user message with the same audio bytes and text, each assistant answer as returned with any chat-template markers removed, and the new user text.
For a streamed answer, resend the final text, which is the last transcription.transcript.replace text when one was sent, not the joined deltas.
An answer can occasionally contain chat-template markers, which the server refuses in history; remove them before resending, as described in Troubleshooting.
The server keeps no conversation state between requests; the cache only makes later turns faster.
A later turn whose audio differs from the recording named by audio_id returns 400 (audio_mismatch).
A later turn that resends the audio without audio_id, or with an audio_id that this NIM cannot verify, is still answered: the server rebuilds the first turn from this request, might encode the audio again, and returns a new audio_id.
A malformed audio_id, or an audio_id without the resent audio, returns 400.
The cache is held in the NIM’s memory, so send all turns of a conversation to the same NIM; a turn sent to another replica is answered but encodes the audio again.
With a fixed ASR_CONVERSATION_KEY, earlier audio_id values stay valid after a restart, but the cache is empty, so the next turn encodes the audio again.
With the default generated key, a restart invalidates earlier audio_id values, and the next turn falls back as described above.
audio_id is a bearer token: anyone who presents it with the same audio can continue the conversation on that NIM.
The NIM has no notion of client identity; if clients must be isolated from each other, enforce it in front of the NIM.
The first turn’s kind follows its content.
No text, task: "transcribe", a standard transcription prompt, or a speaker-tag transcription prompt returns a transcript.
Other text is answered as an instruction over the audio.
Under Multi-turn Mode, task also accepts respond, which answers a request spoken in the audio.
Each conversation turn’s transcription.conversation object reports turn, cached_tokens, audio_id, truncated, truncated_reason, empty_answer, and closed.
truncated, empty_answer, and closed are always present and are false when they do not apply.
truncated_reason is always present: null when the output is complete, otherwise the reason it was cut, for example length when an answer reached its token limit.
empty_answer describes an answer to a question; an empty transcript of silent audio reports false.
budget_exhausted is present and true only when the turn’s decode stopped on its token budget.
audio_store reports where a later turn’s audio came from: resend (the decoder cache), hit (the host audio store), miss (re-encoded), or none (the first turn, or a turn without audio).
An answer is limited to 8,192 tokens, or less when the conversation leaves less room in the model’s context; a longer answer is cut with truncated_reason: "length".
usage.prompt_tokens_details.cached_tokens shows how many prompt tokens were reused.
On a server without Multi-turn Mode, follow_up, audio_id, close, a system message, and an assistant message each return 400.
On a Multi-turn Mode server, word_timestamps returns 400 on every turn.
The multipart routes do not accept conversation fields in either mode.
Errors#
Errors use a JSON error object with message, type, param, and code fields.
error.param names the request field when the server can attribute the error to one, for example language, task, or the path of a field with the wrong type.
It is null for body-level errors, such as an over-size body, a wrong Content-Type, invalid input_audio.format or base64 data, and errors reported after the audio is decoded; the message then describes the problem.
HTTP Status |
Meaning |
|---|---|
|
Invalid request or unsupported option, including unknown fields, a |
|
|
|
|
|
Request body or audio duration exceeds the configured limit. |
|
Wrong |
|
The audio is in an unsupported format ( |
|
Ingress or queue capacity is exhausted. Retry with backoff and reduce client concurrency. |
|
Admission or backend timeout. Check readiness, logs, and resource usage. |