How much memory do local speech-to-text models need?
From about 1 GB of VRAM for Whisper tiny and base to at least 16 GB for Voxtral Realtime 4B: vendors publish an actual memory figure for 10 of the 26 reviewed models. This page lists every official figure next to our 1–5 memory and hardware ratings, and says plainly where no figure exists, so you can rule models in or out before downloading gigabytes of weights.
On this page
Memory guidance and hardware fit for every reviewed model
Sorted from the heaviest to the lightest by our memory rating (5 = lightest). The guidance column quotes the vendor's model card or repository; ratings are Speech Model Match's conservative editorial scores (methodology).
| Model | Official memory or hardware guidance | Memory rating | CPU | Apple | NVIDIA |
|---|---|---|---|---|---|
| Moonshine Tiny Streaming (English) Moonshine AI · 34M | 34M parameters, 8-bit weights. Published latency for Tiny Streaming: 34 ms on a MacBook Pro, 69 ms on Linux x86, 237 ms on a Raspberry Pi 5. Source ↗ | 5/5 | 5/5 | 5/5 | 4/5 |
| Parakeet Redux Moondream · 1.58-bit Parakeet 0.6B (178 MB) | The model fits in 178 MB. Moondream reports 113× real time on eight x86 CPU cores (AMD EPYC 9575F) and about 38× on the CPU of an M2 MacBook Air. Source ↗ | 5/5 | 5/5 | 4/5 | 3/5 |
| VibeVoice-ASR-BitNet Microsoft · 0.3B quantized | 1.58 GB of model files in total and about 2 GB of disk; no GPU required, with a real-time factor below 1 on three or more CPU threads in Microsoft's benchmarks. On Windows, MSVC is not supported (use GCC or Clang). Source ↗ | 5/5 | 5/5 | 4/5 | 3/5 |
| Whisper base OpenAI · 74M | OpenAI lists about 1 GB of VRAM for base (74M parameters); whisper.cpp lists about 388 MB of memory. Source ↗ | 5/5 | 5/5 | 5/5 | 4/5 |
| Whisper tiny OpenAI · 39M | OpenAI lists about 1 GB of VRAM for tiny (39M parameters); whisper.cpp lists about 273 MB of memory. Source ↗ | 5/5 | 5/5 | 5/5 | 4/5 |
| Whistle Cactus Compute · 16.9 MB file (parameters not stated) | A single 16.9 MB file that runs on CPU with no GPU and no dependencies. Source ↗ | 5/5 | 5/5 | 4/5 | 2/5 |
| Omnilingual ASR CTC 300M Meta AI · 325M | About 2 GiB of inference VRAM and a 1.3 GiB FP32 download, measured by Meta at batch size 1 on 30-second audio in BF16 on an A100. Source ↗ | 4/5 | 3/5 | 4/5 | 5/5 |
| Granite Speech 5.0 470M TurboCTC IBM · 470M | No memory figure is published; IBM describes the model as suited to laptops, smartphones and other edge devices. Source ↗ | 4/5 | 4/5 | 4/5 | 4/5 |
| Moonshine Medium Streaming (English) Moonshine AI · 245M | 245M parameters, 8-bit weights. Published latency: 107 ms on a MacBook Pro, 269 ms on Linux x86, 802 ms on a Raspberry Pi 5. Source ↗ | 4/5 | 4/5 | 5/5 | 4/5 |
| Whisper small OpenAI · 244M | OpenAI lists about 2 GB of VRAM for small (244M parameters); whisper.cpp lists about 852 MB of memory. Source ↗ | 4/5 | 4/5 | 5/5 | 5/5 |
| Distil-Whisper large-v3 Hugging Face / Distil-Whisper · large-v3 distilled | Not published by the vendor | 3/5 | 2/5 | 4/5 | 5/5 |
| Fun-ASR-MLT-Nano-2512 FunAudioLLM · 800M | The model card's example sets the device to a CUDA GPU or the CPU; no memory figure is published. Source ↗ | 3/5 | 2/5 | 2/5 | 5/5 |
| Granite Speech 5.0 2B IBM · 2B | No memory figure is published. Source ↗ | 3/5 | 2/5 | 4/5 | 5/5 |
| Nemotron 3.5 ASR Streaming 0.6B NVIDIA · 600M | Listed hardware: NVIDIA Ampere, Blackwell, Hopper, Jetson, Lovelace, Turing and Volta on Linux. No CPU-only or Apple Silicon guidance is published. Source ↗ | 3/5 | 2/5 | 3/5 | 5/5 |
| Parakeet TDT 0.6B v3 NVIDIA · 600M | At least 2 GB of RAM to load the model; more memory allows longer audio. Listed hardware: NVIDIA Ampere, Blackwell, Hopper and Volta GPUs, preferably on Linux. Source ↗ | 3/5 | 2/5 | 3/5 | 5/5 |
| Qwen3-ASR 0.6B Qwen · 0.6B | Not published by the vendor | 3/5 | 2/5 | 3/5 | 5/5 |
| Orukeet Oruk AI · 627M | No RAM or VRAM figure is published. Download sizes: Q8 GGUF about 714 MB, F16 GGUF about 1.3 GB. Source ↗ | 3/5 | 3/5 | 4/5 | 5/5 |
| Jais-ASR IFM · 5.7B total | Plan on roughly 12 GB of GPU memory in BF16; FP8 (7.68 GB) and 4-bit AWQ (5.85 GB) downloads are also published. Source ↗ | 2/5 | 1/5 | 2/5 | 4/5 |
| VibeVoice-ASR Streaming 1.5B Microsoft · 3B total | Microsoft's documentation uses an NVIDIA GPU container. Source ↗ | 2/5 | 1/5 | 2/5 | 4/5 |
| Whisper medium OpenAI · 769M | OpenAI lists about 5 GB of VRAM for medium (769M parameters); whisper.cpp lists about 2.1 GB of memory. Source ↗ | 2/5 | 2/5 | 4/5 | 5/5 |
| Whisper turbo OpenAI · 809M | OpenAI lists about 6 GB of VRAM for turbo (809M parameters), an optimized version of large-v3. OpenAI also notes that turbo is not trained for translation tasks. Source ↗ | 2/5 | 2/5 | 4/5 | 5/5 |
| Cohere Transcribe 03-2026 Cohere Labs · 2B | Not published by the vendor | 1/5 | 1/5 | 2/5 | 5/5 |
| Qwen3-ASR 1.7B Qwen · 1.7B | Not published by the vendor | 1/5 | 1/5 | 3/5 | 5/5 |
| VibeVoice-ASR 7B Microsoft · 9B total | Microsoft's setup uses an NVIDIA GPU container. No CPU-only path is documented for this model; the BitNet build is the CPU option. Source ↗ | 1/5 | 1/5 | 2/5 | 4/5 |
| VibeVoice-ASR-Streaming-7B Microsoft · 9B total | No memory figure is published; the card lists 9B parameters. Source ↗ | 1/5 | 1/5 | 1/5 | 4/5 |
| Voxtral Mini 4B Realtime 2602 Mistral AI · 4B | Runs on a single GPU with at least 16 GB of memory (BF16 weights). Source ↗ | 1/5 | 1/5 | 2/5 | 5/5 |
No model reaches that rating on this hardware. Lower the minimum rating.
Whisper sizes compared
Whisper is the only family that publishes a figure for every size, from two sources: OpenAI's PyTorch package and whisper.cpp.
| Size | Parameters | openai-whisper VRAM | whisper.cpp memory |
|---|---|---|---|
| Whisper tiny | 39M | ~1 GB | ~273 MB |
| Whisper base | 74M | ~1 GB | ~388 MB |
| Whisper small | 244M | ~2 GB | ~852 MB |
| Whisper medium | 769M | ~5 GB | ~2.1 GB |
| Whisper turbo | 809M | ~6 GB | Not listed |
Sources: OpenAI Whisper README ↗ and whisper.cpp memory table ↗ (checked November 2, 2026). faster-whisper also documents INT8 inference on CPU.
What changes memory use
- Precision. FP32 weights take twice the memory of FP16/BF16 and four times INT8. Quantized builds such as whisper.cpp, faster-whisper INT8, Moonshine's 8-bit models or VibeVoice-ASR-BitNet exist for exactly this reason. See quantization.
- Audio per pass. Longer context costs memory: Parakeet documents 24 minutes per file with full attention and up to 3 hours with local attention; Voxtral defaults to about 3 hours of context and can be lowered.
- Runtime. The same weights can use very different memory in PyTorch, CTranslate2 or ggml. Compare like with like.
- Batching and concurrency. Serving engines such as vLLM reserve a share of GPU memory up front for throughput; a single-user desktop needs less.
Shortlists by hardware
- CPU only
- Rated 4/5 or better: Whisper tiny, Whisper base, Whisper small, Moonshine Tiny Streaming (English), Moonshine Medium Streaming (English), VibeVoice-ASR-BitNet, Granite Speech 5.0 470M TurboCTC, Parakeet Redux and Whistle. Read the CPU-only guide.
- Small GPU or 8 GB laptop
- Lightest memory ratings (4–5/5): Whisper tiny, Whisper base, Whisper small, Moonshine Tiny Streaming (English), Moonshine Medium Streaming (English), Omnilingual ASR CTC 300M, VibeVoice-ASR-BitNet, Granite Speech 5.0 470M TurboCTC, Parakeet Redux and Whistle. For an 8 GB card see the 8 GB VRAM guide.
- Apple Silicon
- Rated 4/5 or better on Apple: Whisper (tiny, base, small, medium, turbo), Orukeet, Distil-Whisper large-v3, Moonshine Tiny Streaming (English), Moonshine Medium Streaming (English), Omnilingual ASR CTC 300M, VibeVoice-ASR-BitNet, Granite Speech 5.0 2B, Granite Speech 5.0 470M TurboCTC, Parakeet Redux and Whistle. See the Mac guide.
- 16 GB+ GPU
- Needed for Voxtral Mini 4B Realtime 2602 and comfortable for the large Qwen3-ASR 1.7B, Cohere Transcribe 03-2026, VibeVoice-ASR 7B and VibeVoice-ASR-Streaming-7B.
Common questions
How much VRAM does Whisper need?
OpenAI's README lists about 1 GB of VRAM for tiny and base, 2 GB for small, 5 GB for medium and 6 GB for turbo. whisper.cpp lists lower memory use for its own builds (for example about 852 MB for small). Actual use depends on runtime, precision and batch size.
Can I run a speech-to-text model without a GPU?
Yes. Whisper tiny, Whisper base, Whisper small, Moonshine Tiny Streaming (English), Moonshine Medium Streaming (English), VibeVoice-ASR-BitNet, Granite Speech 5.0 470M TurboCTC, Parakeet Redux and Whistle are rated 4/5 or better for CPU-only use. VibeVoice-ASR-BitNet is built for CPUs: Microsoft reports real-time speed on three or more CPU threads with no GPU.
Which reviewed model needs the most memory?
Voxtral Mini 4B Realtime 2602 is the only model here with an official minimum: a single GPU with at least 16 GB. Qwen3-ASR 1.7B, Cohere Transcribe 03-2026, VibeVoice-ASR 7B and VibeVoice-ASR-Streaming-7B carry our lowest memory rating as well, but their vendors do not publish a figure.
Why do the figures differ between runtimes?
Memory depends on weight precision (FP32, FP16/BF16, INT8 or lower), the runtime, batch size and how much audio is processed at once. Parakeet, for example, documents 24 minutes per file with full attention and up to 3 hours with local attention; Voxtral lets you lower the maximum context to save memory.
Official figures copied from the linked vendor pages on November 2, 2026; ratings from the reviewed model database (latest review November 3, 2026). Measure on your own hardware with the benchmark tool.