Beyond task performance: Decoding bioacoustic embeddings with speech features
2026-06-12 • Machine Learning
Machine LearningSound
AI summaryⓘ
The authors studied how pretrained audio models understand different sound features in animal recordings. They tested which speech-like features these models can recognize well, like loudness or pitch, using a set of standard sound measurements called eGeMAPS. They found that no single model understands all features perfectly, but combining models improves results. Loudness was the easiest feature for the models to capture, while pitch (F0) was the hardest. Their findings help guide which models to use for studying different animal sounds, especially when data is limited.
pretrained audio embeddingsbioacousticseGeMAPSacoustic featuresregression probesloudnessfundamental frequency (F0)feature saliencenonlinear regressionmodel selection
Authors
Ines Nolasco, Jules Cauzinille, Marius Miron, Gagan Narula, Milad Alizadeh, Emmanuel Fernandez, Matthieu Geist, Ellen Gilsenan-McMahon, Olivier Pietquin, Emmanuel Chemla, Sara Keen
Abstract
Pretrained audio embeddings are standard in bioacoustics, yet little is known about which acoustic features these models encode, nor which are useful for a given task. This hinders transparency and limits extension to rare species or data-scarce domains. Here we reveal which speech-like features are encoded in bioacoustic representations. Using the 88~eGeMAPS features across six taxonomic groups, we apply linear and nonlinear regression probes to quantify which acoustic properties each model captures. Results confirm a ``no free lunch'' pattern: no single model captures the full feature space. A concatenated embedding achieves the highest performance, suggesting complementary acoustic space coverage across models. Loudness features are best encoded ($R^2 = 0.76$) while F0 is hardest to recover ($R^2 = 0.33$). By cross-referencing recoverability with per-species feature salience (NMI), we derive data-driven model selection guidance for bioacoustics.