What does the model actually see? Evaluation protocols and input availability in data-driven prediction of room acoustic parameters
2026-07-16 • Sound
Sound
AI summaryⓘ
The authors studied how well machine learning models predict room acoustics using limited measurements. They found that very high accuracy numbers often come from how the tests are set up rather than how good the models really are. When tested more realistically, the models performed with lower accuracy but still better than simple methods for some metrics. The study also showed that certain model inputs might give misleading advantages by identifying measurement positions instead of learning true acoustic properties. Overall, the authors highlight the importance of evaluation methods to understand true model performance.
ISO 3382-1room acousticsmachine learningcoefficient of determinationvalidation protocolimpulse responseconvolutional neural networkrandom forestsound strengthreverberation time
Authors
Akın Oktav
Abstract
Machine-learnt models are increasingly used to predict ISO 3382-1 room acoustic parameters from sparse measurements, with reported coefficients of determination frequently above 0.85. This paper shows that such figures are often determined by the evaluation protocol rather than by the model. Using a multi-condition measurement campaign in a 264-seat conference hall and a 180-seat concert hall, three model families were evaluated under a factorial protocol ablation: validation splits either row-based or grouped by receiver position, and input features either including measured-at-test quantities or restricted to source-receiver geometry and environmental state. Row-based splits with measured-at-test inputs reproduce the high reported accuracies (mean $R^2$ 0.81 for the core parameters); grouping the splits by position and restricting inputs to information available at an unmeasured position reduces these to 0.09-0.57, reordering the apparent difficulty of parameter classes. A hybrid CNN evaluated with the target's own impulse response as input is shown to exploit it as a position fingerprint rather than as transferable acoustic information; training-only signal access yields no gain for any parameter tested, including reverberation time. Under the deployment-consistent protocol, the spread between Random Forest, the hybrid CNN, and inverse-distance weighting is an order of magnitude smaller than the spread between protocols for a fixed model; the learnt models retain a genuine advantage for sound strength and reverberation time, and the high accuracy of the original pipelines re-emerges as condition interpolation at measured positions (band means 0.80-0.88), a distinct and operationally useful task.