What did Nvidia announce on October 28, 2026?
Nvidia released Nemotron 3.5 Transcribe, an open-weight speech model, on October 28, 2026. Weights went up on Hugging Face and ModelScope, alongside a hosted endpoint list that includes Baseten, DeepInfra and Simplismart. A companion dataset, Granary v2, shipped the same day.
The pitch is consolidation. Most speech pipelines chain a voice activity detector, a transcription model, a separate diarization model and, often, a translation or summarisation step. Nvidia says one checkpoint now covers transcription, speaker attribution, translation and audio question answering. SiliconANGLE described the release as an all-in-one system for multilingual audio.
The headline figures are specific. The model transcribes 40 languages, translates 31 of them to or from English, and labels speakers in 13. It handles recordings up to one hour long and labels up to eight speakers. The context window is 65,536 tokens, and outputs can include word-level timestamps.
The model card marks it ready for commercial or non-commercial use under the OpenMDW-1.1 licence. That matters for hospitals, banks and public agencies that cannot send recordings to a third-party API. It also puts pressure on vendors that charge per minute for the same pipeline.
Nvidia is not selling a single service. There is a free demo on Hugging Face Spaces, downloadable weights, and three hosted providers for teams that would rather rent capacity. The company also published the data pipeline and fine-tuning recipes, which is closer to a toolkit than a product launch.
Speech is one of the few AI workloads where the bill often arrives per minute. Transcription vendors typically charge by audio duration, so long meetings and call archives get expensive quickly. An open model changes the maths for teams with their own GPUs, because the marginal cost becomes compute and engineering time rather than a metered fee. It also changes where the audio lives, which is often the harder constraint.
Agora, whose voice platform powers live apps, used the model's word-level timestamps to show a class transcript beside a replay. That is a small example, but it points at the near-term market: recorded meetings, lectures, interviews and consultations.
How does one model handle transcription, translation and speaker labels?
The checkpoint is larger than its name suggests. It is branded 30B because the language backbone, Nemotron 3.5 Lightning, is a 30B mixture-of-experts model with about 3.5B active parameters. Add the audio encoder and the total reaches about 34B, with roughly 4.3B parameters active per token.
The audio side has two parts. A 600M-parameter Transformer encoder handles speech recognition. A speaker encoder of about 99M parameters, derived from Nemotron 3 Diarization, supplies information about who is talking. A separate alignment head produces timestamps, so the model can place each word in time without a second tool.
Instructions arrive as plain text. A developer can ask for a verbatim transcript, a cleaned one, speaker-tagged output or a translation. Domain terms can be passed as hints; Nvidia's medical example tells the model to use clinical vocabulary only when the audio supports it. The card recommends temperature zero for transcription and a reasoning mode for audio questions.
Style control is the part that differs from older systems. Nvidia demonstrated the model on a scene from the 1940 film His Girl Friday, which is famous for fast, overlapping dialogue. The same recording can yield a raw transcript with filler words or a written version with numbers in digits.
Coverage is not uniform, and Nvidia publishes a matrix rather than one claim. Transcription spans 40 languages, translation 31 and speaker attribution 13, and the card lists support for each language by task. Audio question answering was trained in English, so a French meeting can be transcribed and labelled but not interrogated in the same way.
Weights ship in BF16, with FP8 and NVFP4 quantised builds for smaller GPUs. Nvidia also published a DFlash2 draft model to speed up speculative decoding. The language stack is a hybrid Mamba-2 and Transformer mixture of experts, not a conventional Transformer, and the model reuses a repeated layer to propose several tokens at once.
How accurate is Nemotron 3.5 Transcribe?
The numbers are strong, but they carry footnotes. On the Hugging Face Open ASR Leaderboard, Nvidia reports 5.33% average word error rate in English. That ranks first among open-weight models once API models are excluded. It also reports 4.62% mean error across seven languages and 10.98% on long-form audio.
The model card gives a slightly different English figure. Its snapshot from October 12, 2026 puts the nine-dataset mean at 4.71%, ahead of Qwen3-ASR-1.7B at 4.94%. The two numbers describe different leaderboard snapshots and configurations. Readers should treat vendor results as directional until independent runs settle.
That snapshot shows where errors cluster. LibriSpeech test-clean sits at 1.09% and VoxPopuli at 1.84%, while the accented, spontaneous URGENT24 set reaches 11.00% and AMI meetings 7.81%. Structured content is harder still: 3.55% on numbers, and 8.38% on codes such as email addresses and file paths.
Artificial Analysis, which tests models itself, scores Transcribe at 2.8% AA-WER. Its public ranking lists Mistral's Voxtral Small and Nemotron 3.5 Transcribe at 2.8% each, with Voxtral ordered first. Nvidia calls that a tie for first among open-weight models and notes the rival it ties activates about 24B parameters.
The context behind that ranking matters. Artificial Analysis says it has evaluated 62 speech models, 15 of them open-weight. Leading that subset is a narrower claim than leading the whole market, and Nvidia says the model ranks ahead of several proprietary APIs rather than all of them. On the voice-agent subset that makes up half the index, it scores 1.8% error against 2.3% for the next-best open model.
The exclusion of API models also shapes the Hugging Face position. The strongest commercial systems are not part of that comparison, so a first place among open weights describes a category, not the state of the art. Nvidia frames it that way in the blog, but the distinction is easy to lose in a headline.
Speaker attribution shows the widest margin. On David AI's 21-language diarization benchmark, Transcribe posts an 18.26% diarization error rate. That is second overall and first among open-weight systems, narrowly ahead of Nvidia's dedicated diarization model at 18.38%. On VoiceArena's 14-system English test it leads at 32.4% cpWER and drops 8.32% of reference words, against a median near 31%.
Nvidia also reports a multi-speaker score of 15.80 mean cpWER across four meeting corpora, against 16.83 for a paired DiCoW and DiariZen setup. On its own style-control test, instructed transcription scores 7.56% error by default and 6.16% when told the target format. All of these are vendor-run evaluations.
What is Granary v2, and why does the dataset matter?
Open weights are only half of reproducibility. Nvidia also released Granary v2, the pipeline that produces the transcript formats the model can be prompted for. The release covers 463,062 hours across 48 languages and 16 source corpora, up from 25 languages in Granary v1.
The manifests are labels, not audio. Developers fetch source recordings separately under each corpus's own licence, which keeps the licence terms of broadcast, meeting and read-speech collections intact. About 165,300 hours pass the pipeline's quality gate. The release also includes 92,496 hours of speech translated into English and 39,375 hours translated out of English.
Each recording gets three transcript styles: verbatim with normalised text, inverse-normalised with disfluencies, and inverse-normalised without them. The pipeline runs in four stages, from audio preparation through transcription and label generation to a final fidelity filter. It routes each language to primary and backup recognisers, then rejects output that fails a hallucination check.
The filter has a visible cost. On the VoxPopuli corpus, it reduced 354,000 hours to 111,000 across 22 European languages, discarding noisy, clipped and multi-speaker segments. Nvidia says teams can run the same pipeline on their own recordings using Ray and its Dynamo inference stack.
The licence plumbing is worth reading before any training run. Manifests carry entity hints, instruction pairs, provenance and a per-record quality flag, but the audio stays with its original distributor. Broadcast, meeting and read-speech corpora each carry their own terms, and a team that ignores them inherits the risk.
Nvidia is careful about what those figures mean. The announcement states that 463,062 hours describe the dataset release, not the model's full training mixture or sampling weights. The model card says training used about 1.72 million unique indexed audio hours and 5.6 billion unique text tokens, with English the largest single share.
What does the medical fine-tuning study show?
The most concrete evidence is a case study on Eka Care's open benchmark of Indian medical speech, which contains 3,619 clips. With a basic prompt on the full test set, the model reached 30.8% word error rate. A better prompt that named English and the Latin alphabet cut that to 23.8%.
Fine-tuning went further. Nvidia combined about 153 hours of synthetic speech, 29 hours of accented English from Common Voice and 63 hours of medical recordings from MultiMed. The full-set error rate then fell to 11.8%, which Nvidia describes as about 62% fewer errors than the untuned model.
The synthetic data is built from written targets rather than read-outs. The written form is a drug name such as HbA1c, while the synthesised audio speaks the letters aloud. Multiple synthetic voices widen the acoustic range. Training text was filtered to remove exact matches and shared eight-word sequences with benchmark references.
The cleaner result uses 2,764 held-out clips from speakers the model had not heard. There, fine-tuning cut error from 22.44% to 12.17%, a 45.8% relative reduction. Keyword error fell from 9.2% to 6.5% and semantic error from 10.0% to 7.6%. The adapted model beat an open medical specialist at 13.79% and a widely used general recogniser at 14.92%.
The trade-off is visible too. General English accuracy slipped on LibriSpeech test-clean, from 1.54% to 1.69%. The specialist model also stayed ahead on exact brand-name drugs. Nvidia recommends keeping at least 100 real, labelled clips for checkpoint selection, and the whole adaptation took about 14 GPU-hours on one GB300.
The result matters beyond one benchmark. It shows that a general speech model can be pushed toward a specialist domain with modest labelled audio and a large amount of synthetic speech. That pattern suits languages and specialties where big labelled corpora do not exist, which is most of them.
What are the main limitations?
The biggest is that it does not stream. Nvidia's blog states that Transcribe is built for recorded audio, not live streaming. Teams building voice agents, live captions or call-centre assistance still need a different system for the real-time path. The model's throughput is high, but it is built for batches of complete recordings.
Language support varies by task. Translation covers 31 languages, speaker labels cover 13, and audio question answering was trained only in English. The card also warns that text-only performance is weaker than the underlying language model, because text reasoning was not a primary training goal.
Accuracy claims rest on evaluations run by the vendor, and the company says so. The card lists substitutions, deletions, repetitions, hallucinated speech, wrong language choices and speaker errors as expected failure modes. It asks users to spot-check names, speaker changes and answers against the recording.
Bias assessment is thin. The card states that no ranked demographic assessment exists. It also says coverage across age, gender, accent and recording environment is not established proportionally. For medical, legal or hiring use, that is a real gap rather than a formality.
The card is also explicit about what has not been tested. It recommends targeted checks for long recordings, overlapping speakers, background noise, proper names, code-switching and misleading prompts. None of those are exotic conditions in a call centre or a clinic.
Privacy carries familiar caveats. Nvidia says training data included personal information, handled under applicable data protection law, with filters meant to stop the model reciting it. Any deployment that processes calls or consultations still needs its own consent and retention rules.
Who can run it, and what should you watch next?
Requirements are modest for a 34B model. One H100 with 80GB runs it in BF16. The card also lists A100, B200, GB200, GB300, DGX Spark, Jetson Thor and RTX 5090 support, with the consumer card limited to NVFP4. Open serving uses vLLM 0.32.0 or SGLang-Omni 0.2.0, on Linux, WSL2 or native Windows through GGML.
Throughput figures need the same caution as the accuracy claims. Nvidia reports 100 hours of audio in under three minutes on one RTX PRO 6000 Blackwell Server Edition. It also reports 2,238 seconds of audio processed per second with NVFP4 and speculative decoding, and 0.13 seconds to return a 30-second transcript.
Those gains come from the NIM implementation, which is not yet in the open release. The open vLLM path is expected to be about twice as slow. That gap matters for anyone planning a production service on the downloaded weights rather than on a hosted endpoint.
Pricing is unsettled. Nvidia published no per-minute rate, and hosted providers set their own. The practical comparison is against Mistral's Voxtral family, Alibaba's Qwen3-ASR and Nvidia's own Parakeet line, not against a single list price.
Self-hosting is the other selling point. A hospital or law firm can keep recordings inside its own network, which is often the difference between a pilot and a signed contract. The trade-off is operational: someone has to run inference, watch for drift and re-test after each fine-tune.
The next checks are independent ones. Watch whether Artificial Analysis and the Open ASR Leaderboard hold these rankings as more systems are submitted. Watch whether the NVFP4 and speculative-decoding gains reach open source. For buyers, the useful test is a sample of your own audio, in your own accents, scored against your own error metric.