Best local speech-to-text for voice agents and assistants
A voice agent needs words quickly and predictably, not just accurately. 7 of the 26 reviewed models stream natively: Nemotron 3.5 ASR Streaming 0.6B, Moonshine Tiny Streaming (English), Moonshine Medium Streaming (English), VibeVoice-ASR Streaming 1.5B, Voxtral Mini 4B Realtime 2602, VibeVoice-ASR-Streaming-7B and Whistle. For English, the matcher's first pick for a voice agent is Nemotron 3.5 ASR Streaming 0.6B; for Spanish it is Nemotron 3.5 ASR Streaming 0.6B and for Japanese Nemotron 3.5 ASR Streaming 0.6B.
On this page
What the vendors publish about latency
Only these native streaming models document latency-related behaviour. Figures are vendor-reported and measured on different hardware, so do not rank models across rows.
| Model | What the official source documents | Languages |
|---|---|---|
| Moonshine Tiny Streaming (English) | Published latency 34 ms on a MacBook Pro, 69 ms on Linux x86 and 237 ms on a Raspberry Pi 5. Source ↗ | English |
| Moonshine Medium Streaming (English) | Published latency 107 ms on a MacBook Pro, 269 ms on Linux x86 and 802 ms on a Raspberry Pi 5. Source ↗ | English |
| Nemotron 3.5 ASR Streaming 0.6B | Streaming chunk sizes of 80, 160, 320, 560 or 1120 ms are documented; smaller chunks lower latency. Source ↗ | 40 language-locales in tiers |
| Voxtral Mini 4B Realtime 2602 | Transcription delay configurable in 80 ms steps; Mistral recommends 480 ms and serves it over a websocket realtime API in vLLM. Source ↗ | 13 listed languages |
| VibeVoice-ASR Streaming 1.5B | Emits text once per audio chunk with streaming speaker attribution; chunk and look-ahead come from the checkpoint configuration. Source ↗ | 10 listed languages |
Matcher rankings for a voice agent
Voice-agent use case, speed first, native streaming preferred, mid-range NVIDIA GPU. Re-run with your own hardware in the matcher.
| # | Model | Fit score | Streaming |
|---|---|---|---|
| 1 | Nemotron 3.5 ASR Streaming 0.6B | 91/100 · Excellent fit | Native |
| 2 | Moonshine Tiny Streaming (English) | 86/100 · Excellent fit | Native |
| 3 | Moonshine Medium Streaming (English) | 86/100 · Excellent fit | Native |
| 4 | Voxtral Mini 4B Realtime 2602 | 86/100 · Excellent fit | Native |
| 5 | VibeVoice-ASR Streaming 1.5B | 83/100 · Strong fit | Native |
| # | Model | Fit score | Streaming |
|---|---|---|---|
| 1 | Nemotron 3.5 ASR Streaming 0.6B | 91/100 · Excellent fit | Native |
| 2 | Voxtral Mini 4B Realtime 2602 | 86/100 · Excellent fit | Native |
| 3 | VibeVoice-ASR Streaming 1.5B | 83/100 · Strong fit | Native |
| 4 | Qwen3-ASR 0.6B | 77/100 · Strong fit | Via runtime |
| 5 | VibeVoice-ASR-Streaming-7B | 77/100 · Strong fit | Native |
| # | Model | Fit score | Streaming |
|---|---|---|---|
| 1 | Nemotron 3.5 ASR Streaming 0.6B | 91/100 · Excellent fit | Native |
| 2 | Voxtral Mini 4B Realtime 2602 | 86/100 · Excellent fit | Native |
| 3 | VibeVoice-ASR Streaming 1.5B | 83/100 · Strong fit | Native |
| 4 | Qwen3-ASR 0.6B | 77/100 · Strong fit | Via runtime |
| 5 | VibeVoice-ASR-Streaming-7B | 77/100 · Strong fit | Native |
What else the pipeline needs
- Voice activity detection and endpointing decide when the user has finished speaking. They often add more delay than the recognizer itself. See VAD.
- Partial versus final results. Streaming models emit text continuously; your agent should act on stable text and tolerate revisions.
- Domain words. VibeVoice-ASR Streaming documents customized hotwords and Fun-ASR documents hotword controls; for other models, test product names and jargon explicitly.
- Turn-taking and barge-in. If users interrupt the agent, you need continuous recognition while audio plays back, which favours native streaming.
- Hardware. Moonshine targets laptops, phones and Raspberry Pi; Voxtral Realtime needs a GPU with at least 16 GB. See memory requirements and the edge guide.
Common questions
What is the fastest local speech-to-text model for a voice assistant?
Among published figures, Moonshine Tiny Streaming reports the lowest latency (34 ms on a MacBook Pro), but it is English-only and the vendors measure differently, so the numbers are not a cross-vendor ranking. Measure end-to-end latency, including voice activity detection and endpointing, in your own pipeline.
Can Whisper be used for a voice agent?
Yes, through chunked inference in runtimes such as faster-whisper or whisper.cpp, but Whisper is not a streaming architecture: each chunk adds delay and words can split at chunk borders. For turn-by-turn assistants that is often acceptable; for live captions or barge-in it is a weaker fit than a native streaming model.
Which native streaming model covers more than English?
Nemotron 3.5 ASR Streaming 0.6B (40 language-locales in tiers), VibeVoice-ASR Streaming 1.5B (10 listed languages), Voxtral Mini 4B Realtime 2602 (13 listed languages), VibeVoice-ASR-Streaming-7B (10 listed languages) and Whistle (7 listed languages).
Latency facts checked against the linked official pages on November 2, 2026. Related: streaming guide, Nemotron vs Voxtral.