Phoneme- and Word-Level Metrics Using Self-Supervised Speech Representations for Forced Alignment Evaluation
2026-08-28 • Computation and Language
Computation and Language
AI summaryⓘ
The authors developed two new ways to check the quality of forced alignment, which is matching speech to text, without needing hand-made time labels. They created two metrics, PCMI and WACS, that use patterns learned from speech itself to see how well words and sounds line up. They tested these metrics on many languages and found they work well to spot good versus bad alignments and match traditional manual checks. This approach allows quick and large-scale evaluation of alignments without extra human work.
Forced AlignmentSelf-Supervised LearningPhonemeMutual InformationDynamic Time WarpingSpeech RepresentationsMultilingual SpeechEvaluation MetricsLow-resource LanguagesTimestamp Annotation
Authors
V. S. D. S. Mahesh Akavarapu, Michael Daniel, Gerhard Jäger
Abstract
Forced alignment evaluation typically requires manually annotated timestamps, limiting large-scale and multilingual analysis. We introduce two corpus-level metrics based on self-supervised (SSL) speech representations for reference-free forced alignment evaluation: Phoneme-Cluster Mutual Information (PCMI) and Word Acoustic Consistency Score (WACS). PCMI measures agreement between aligned phoneme labels and clusters induced from SSL-speech representations, while WACS measures consistency of repeated word realizations using dynamic time warping similarity between word representation sequences. Using both random and systematic perturbations, we show that PCMI and WACS degrade consistently under alignment perturbations. We further analyze the metrics across multiple alignment systems on 85 languages from FLEURS, validate them against manually annotated alignments from 45 languages in DoReCo, and evaluate them on two phonologically complex low-resource languages. The metrics effectively separate high- and low-quality alignments and correlate strongly with timestamp-based alignment quality measures. Our results demonstrate that SSL-speech representations enable scalable, reference-free forced alignment evaluation. The metrics are available as an open-source Python package at https://github.com/mahesh-ak/forced-aligner-metrics.