Tokenizer Transplantation: Mitigating Autoregressive Collapse in Edge-Efficient Bengali ASR
2026-07-10 • Computation and Language
Computation and Language
AI summaryⓘ
The authors show that a popular lightweight speech recognition model struggles with Bengali because its English-based tokenizer breaks Bengali words into many small pieces, causing errors during speech decoding. They fix this by swapping in a Bengali-specific vocabulary and adjusting the model accordingly, which cuts down the number of word pieces and makes the decoding process more stable. Tested on a large Bengali dataset, their approach achieves good recognition accuracy and runs very quickly without needing extensive retraining. This work offers a practical way to adapt speech models to new languages with different scripts more easily.
speech recognitiontokenizerByte-Pair EncodingWordPiece vocabularyautoregressive decodingmorphologically rich languagesBengali languagemodel adaptationword error ratereal-time factor
Authors
Sanjid Hasan, Md. Abdur Rahman
Abstract
Lightweight speech recognition models are critical for edge deployment, yet highly optimized architectures like Moonshine often fail on morphologically rich, non-Latin languages such as Bengali. This study identifies the root cause of this failure as the model's English-centric byte-level tokenizer, which fragments Bengali words into high-fertility byte chains and triggers catastrophic autoregressive collapse during inference. To resolve this, a novel vocabulary transplantation pipeline is proposed to replace the decoder vocabulary with the native-script BanglaBERT WordPiece vocabulary and resize the corresponding token embedding matrix. Experimental results demonstrate a reduction in token fertility from 9.16 to 1.30. By decreasing autoregressive sequence length by 85.8%, decoding instability is entirely mitigated. When evaluated on the 882-hour Lipi-Ghor dataset, the modified architecture achieves a competitive 21.54% Word Error Rate (WER) and a Real-Time Factor (RTF) of 0.0053. Ultimately, this research provides a scalable, reproducible blueprint for cross-script adaptation of compact ASR models without the need for resource-intensive pre-training.