Speech-to-text glossary
Plain-language definitions of the terms that decide which local speech-to-text model fits: accuracy metrics such as WER and CER, speed metrics such as real-time factor, streaming versus chunking, diarization, timestamps, model architectures and quantization. Each entry says why the term matters for your choice and where to apply it on this site.
Sections
Basics
- ASR / speech-to-text (STT) (automatic speech recognition)
Software that converts spoken audio into text. A local ASR model runs on your own computer, server or device instead of sending audio to a cloud API.
Why it matters: Local models keep audio private and avoid per-minute fees, but you take on hardware, setup and updates yourself.
- Open weights
The trained model parameters are published for download, usually with a license that says how they may be used. Open weights do not automatically mean the training data or training code are public.
Why it matters: Only open-weight models can run fully offline. The license attached to the weights decides commercial use.
- Model card
The official page that documents a model: intended use, supported languages, input format, evaluation results, limitations and license.
Why it matters: It is the primary source for every claim on this site; check it again before production because vendors update cards.
Accuracy
- WER (word error rate)
The share of words a transcript gets wrong compared with a correct reference: (substitutions + deletions + insertions) ÷ number of reference words. Lower is better, and because insertions count, WER can exceed 100%.
Why it matters: WER is the standard way to compare models on your audio. Published WER from other datasets rarely transfers to your accents, microphones and vocabulary.
- CER (character error rate)
The same calculation as WER, but on characters instead of words.
Why it matters: Use CER for languages written without spaces between words, such as Chinese, Japanese or Thai, where word boundaries make WER unreliable.
- Text normalization
Making reference and model output comparable before scoring: lower-casing, removing punctuation, and unifying numbers or spellings ("10" versus "ten").
Why it matters: Two models can differ by several WER points purely because one writes punctuation and digits. Normalize both sides the same way before comparing.
- Hallucination
Text the model outputs that was never spoken, for example repeated phrases or invented sentences during silence, music or noise. It is a known failure mode of encoder-decoder models such as Whisper.
Why it matters: Hallucinations hurt subtitles and legal or medical notes more than a few misheard words. Voice activity detection and checking long silences reduce the risk.
Speed and hardware
- Real-time factor (RTF)
Processing time divided by audio duration. An RTF of 0.1 means one hour of audio takes about six minutes. Some leaderboards publish the inverse (RTFx), where higher is faster.
Why it matters: RTF tells you how much hardware a backlog needs. Always measure it on your own machine with the same runtime and settings you will deploy.
- Latency
The delay between a word being spoken and its text appearing. For live use it includes audio buffering, model computation and any network hops.
Why it matters: Captions and voice agents need low, stable latency; batch transcription of files does not care about it at all.
- VRAM and unified memory
VRAM is the memory on a dedicated GPU that must hold the model weights and working buffers. Apple Silicon Macs use unified memory shared by the CPU and GPU.
Why it matters: If a model does not fit in memory it either fails or falls back to much slower execution. Quantization lowers the footprint.
- Quantization
Storing model weights with fewer bits (for example 8-bit or 4-bit integers instead of 16-bit floats) to cut memory use and often speed up inference. Extremely low-bit variants exist too, such as the BitNet build of VibeVoice-ASR.
Why it matters: Quantized models fit smaller GPUs, CPUs and edge devices, but accuracy can drop. Re-check WER after quantizing.
Live transcription
- Native streaming
A model architecture that emits text continuously from a live audio stream, with a bounded look-ahead, instead of waiting for a complete file.
Why it matters: Native streaming gives predictable low latency for dictation, captions and voice agents. On this site, models are labelled native, via runtime, or batch-first.
- Chunked (pseudo-)streaming
Running a batch model repeatedly on short, often overlapping audio windows to imitate live output.
Why it matters: It works with almost any model but adds latency and can split or repeat words at window borders. Compare it against a native streaming model when latency matters.
- VAD (voice activity detection)
A small detector that marks where speech starts and stops, so silence and noise can be skipped or used to cut audio into segments.
Why it matters: VAD speeds up long recordings, gives cleaner segment boundaries and reduces hallucinations during silence.
Speakers and timing
- Speaker diarization
Working out who spoke when and labelling segments as Speaker 1, Speaker 2 and so on. It does not identify people by name.
Why it matters: Meetings, interviews and podcasts need it. Most ASR models do not diarize natively and are paired with a separate diarization pipeline.
- Timestamps (segment and word level)
Start and end times attached to text. Segment timestamps cover a phrase or sentence; word-level timestamps mark every word.
Why it matters: Subtitles, search-in-audio and editing tools need reliable timing. Support ranges from native output to runtime-dependent or not documented.
- Forced alignment
Matching a known transcript to the audio to compute precise word timings, usually with a separate alignment model.
Why it matters: It upgrades rough segment times to word-accurate subtitles. Some model families, such as Qwen3-ASR, document timestamps through a separate aligner.
Languages
- Language identification
Automatically detecting which language is being spoken before or during transcription.
Why it matters: Useful for mixed archives. When you know the language, setting it explicitly is usually more reliable than auto-detection.
- Code-switching
Switching between two or more languages inside one conversation or sentence.
Why it matters: Common in multilingual workplaces. A model listing both languages does not guarantee it handles switching mid-sentence; test it with real recordings.
- Punctuation and capitalization
Whether the transcript comes out as readable sentences with punctuation and capital letters, or as plain lower-case words.
Why it matters: Readable output saves editing time for notes and subtitles, but affects WER unless you normalize before scoring.
- Fine-tuning and adaptation
Continuing training on your own labelled audio so the model learns new vocabulary, accents or a new language. Some vendors list languages as adaptation-ready: supported only after fine-tuning.
Why it matters: Fine-tuning needs transcribed data and GPU time, and the license of the base model still applies to the result.
Architectures
- Encoder-decoder (attention) model
An encoder turns audio into a representation and a text decoder writes tokens one by one while attending to it. Whisper is the best-known example.
Why it matters: Flexible (translation, prompts, language detection) but decoding is sequential, which costs speed and can cause hallucinations.
- CTC (connectionist temporal classification)
An output layer that predicts a character or token for each audio frame and merges repeats, without a separate autoregressive decoder.
Why it matters: CTC models are fast and do not hallucinate long passages, but without a language model they can be weaker on spelling and context. Omnilingual ASR CTC 300M uses it.
- Transducer (RNN-T, TDT)
An architecture that combines an audio encoder with a small prediction network, emitting tokens as audio arrives. NVIDIA's Token-and-Duration Transducer (TDT) also predicts how many frames to skip.
Why it matters: Transducers suit fast and streaming transcription. Parakeet TDT models use this design.
No term matches. Try a shorter word such as "error", "stream" or "speaker".
Definitions written on November 2, 2026. Model-specific facts referenced from here come from the reviewed model database; see the methodology and sources.