Abstract
OpenAI's Whisper model provides strong baseline performance for Swedish speech recognition, yet struggles significantly with regional dialects. We present a systematic study of fine-tuning Whisper Large v3 on Skånska (Southern Swedish), achieving a 40% relative reduction in word error rate on dialect-heavy speech. Our approach combines targeted data collection, phoneme-aware augmentation, and curriculum learning to adapt the model without catastrophic forgetting of standard Swedish capabilities.
The Dialect Challenge
Sweden's regional dialects differ substantially from Rikssvenska (Standard Swedish) in pronunciation, vocabulary, and prosody. Skånska, spoken in the Skåne region, is particularly challenging for ASR systems due to:
- Posterior /r/ — Unlike the alveolar trill in standard Swedish, Skånska uses a uvular fricative similar to Danish
- Diphthongisation — Long vowels are often realised as diphthongs (e.g., /iː/ → [iə])
- Vocabulary — Regional words like "pansen" (stomach), "krull" (curly), and "fansen" (the devil) have no standard Swedish equivalents
Dataset Construction
We assembled a 240-hour Skånska speech corpus from three sources:
- Radio Malmöhus archives — 120 hours of broadcast speech from local radio (2010–2023)
- Oral history recordings — 80 hours from the Institute for Language and Folklore
- Crowd-sourced recordings — 40 hours collected via our mobile app from 450 volunteer speakers across Skåne
All audio was transcribed by native Skånska speakers and phonetically aligned using Montreal Forced Aligner with a custom Skånska pronunciation dictionary.
Fine-tuning Strategy
We employed a three-stage curriculum:
- Stage 1: Standard Swedish refresh — 2 epochs on NST Swedish data to stabilise the model
- Stage 2: Dialect exposure — 5 epochs on mixed Standard + Skånska data (70/30 ratio)
- Stage 3: Dialect focus — 3 epochs on Skånska-only data with reduced learning rate
Results
| Model | Standard WER (%) | Skånska WER (%) |
|---|---|---|
| Whisper Large v3 (baseline) | 6.8 | 22.4 |
| + Fine-tuned (naive) | 7.1 | 16.2 |
| + Curriculum learning | 6.9 | 13.5 |
The curriculum approach preserves standard Swedish performance while achieving a 40% relative WER reduction on Skånska speech, validating our hypothesis that structured training schedules are essential for dialect adaptation.
Conclusion
Regional dialect support is not merely a nice-to-have — for many speakers, it is the difference between a usable and unusable product. Our work demonstrates that careful fine-tuning can bridge the dialect gap without sacrificing general-purpose performance, paving the way for truly inclusive Nordic ASR systems.