Speech recognition and models
Fine-tuning
Training a finished model further on a narrower material, for instance speech in one language.
Fine-tuning is taking an already trained model and training it further on a smaller, more specific material.
The point is that the model already knows the general case. It knows what speech looks like as a frequency pattern, and how language hangs together. Fine-tuning teaches it something narrow on top of that, without everything having to be learned again.
Why it matters for smaller languages
This is the whole explanation for why transcription in smaller languages became good. Training a speech model from scratch requires enormous amounts of data, and a small language does not have that. Taking an international model that already works and fine-tuning it on speech in one language with its dialects requires far less, and produces a model that has heard those dialects before.
NB-Whisper is exactly this: Whisper fine-tuned on Norwegian.
Other uses
- Jargon. A model fine-tuned on medical dictation does better on medical terminology.
- Audio conditions. Training on meeting-room recordings gives better results in meeting rooms.
- Custom dictionary. A simpler alternative: many tools let you add your own words and names without training the model. That is not fine-tuning, but it often solves the same practical problem.
What it costs
Fine-tuning requires hardware, training data with reference text, and expertise. For most organisations it is not on the table. Using a model somebody else has fine-tuned is.