Best local STT models for an 8 GB GPU
Eight gigabytes of VRAM is a useful middle ground: enough for strong local ASR, but not enough to ignore runtime overhead, precision, batching and the rest of your application.
What “fits in 8 GB” really means
A model checkpoint size is not the same thing as runtime VRAM. Activations, attention caches, audio length, batch size, precision, CUDA kernels and the host application all consume memory. Leave headroom instead of choosing the largest model that barely loads.
For interactive desktop use, a model that stays comfortably below the limit is usually a better choice than one that forces aggressive swapping or leaves no memory for the rest of the pipeline.
Whisper turbo: the obvious baseline
OpenAI's repository lists about 6 GB of required VRAM for turbo and describes it as an optimized large-v3 derivative with much faster inference and minimal accuracy degradation. That makes turbo a natural first benchmark on an 8 GB NVIDIA card.
Whisper also gives you smaller fallback sizes if your application needs more headroom. That flexibility is valuable when the ASR model shares the GPU with an LLM, video processing or a UI.
Parakeet TDT 0.6B v3: strong batch candidate
Parakeet is a 600M multilingual model with an explicit 25-language set. NVIDIA now distributes NeMo, Safetensors/Transformers and an official Q8 GGUF for local inference. That makes it much easier to test across different local stacks than earlier NeMo-only releases.
Parakeet is especially attractive for recordings and subtitle-style workloads where throughput and timestamps matter. Treat streaming as a separate requirement: it is not the same streaming-first design as Nemotron 3.5.
Qwen3-ASR 0.6B: modern but backend-sensitive
Qwen3-ASR 0.6B supports 30 languages plus 22 Chinese dialects and can run in both offline and streaming modes. It is worth testing on an 8 GB class card when your language is supported, but practical memory use depends heavily on backend and precision. Qwen recommends modern GPU-oriented serving paths for its strongest throughput and streaming features.
Do not assume the 1.7B model is automatically the right upgrade on 8 GB. A larger model that forces an awkward quantization or leaves no runtime headroom can be worse operationally than a well-supported 0.6–0.8B model.
Nemotron 3.5 when streaming changes the answer
Nemotron 3.5 is also around the 600M class, but the reason to test it is native streaming rather than size alone. If your application is live dictation or a voice agent, include it even if Whisper or Parakeet look better for batch work. Verify your exact locale tier before deployment because NVIDIA distinguishes ready-to-use and adaptation-ready support.
Do not optimize only for VRAM
- Measure throughput: minutes of audio processed per minute matters for batch jobs.
- Measure latency: live apps care about first partial and finalization delay.
- Test long audio: a 30-second clip does not reveal hour-long memory behavior.
- Leave application headroom: browsers, LLMs, video decode and CUDA libraries all compete for memory.
- Check the exact runtime: NeMo, Transformers, GGUF and specialized Whisper runtimes do not have identical memory profiles.
A practical 8 GB benchmark order
- Whisper turbo as the mature general-purpose baseline.
- Parakeet v3 if your language is listed and batch transcription matters.
- Qwen3-ASR 0.6B for a newer offline/streaming multilingual family.
- Nemotron 3.5 if native streaming is a product requirement.
Run the same representative audio through all candidates and record error rate, peak VRAM, real-time factor, timestamp quality and setup complexity. The operational winner often matters more than the model with the best headline benchmark.
Primary sources
Reviewed against primary sources on September 13, 2026. Model behavior, runtime support, language coverage and licenses can change; re-check the linked primary source before production use.