Implementing neural network mixed-effects models in Template Model Builder (TMB)
2026-08-31 • Machine Learning
Machine Learning
AI summaryⓘ
The authors explain a new way to build neural network mixed-effects models (NMMs) that avoids complicated manual math by using a tool called Template Model Builder (TMB). This tool automatically handles tricky calculations and finds exact solutions without needing shortcuts. They show that this method works well through practical examples, making it easier to create more accurate and flexible NMMs. The authors also share their code so others can use their approach easily.
neural networkmixed-effects modelTemplate Model Builder (TMB)automatic differentiationLaplace approximationlog-likelihoodrandom effectsgradientregularizationmonotonic models
Authors
Nan Zheng, Hoi Yiu Cheung, Vibhu Sharma, James T. Thorson, Noel G. Cadigan
Abstract
Neural network mixed-effects models (NMMs) have gained traction by combining the strong representation and predictive power of artificial neural networks with the capacity of mixed-effects modeling to capture complex correlation structures. However, existing estimation approaches rely heavily on manual derivations of objective functions and gradients, which inherently forces simplifying approximations and severely constrains the complexity and accuracy of NMMs. In this work, we introduce a general framework for implementing NMMs using Template Model Builder (TMB). By leveraging automatic differentiation and Laplace approximation, TMB requires users to specify only the negative joint log-likelihood and any regularization terms. The framework automatically integrates out random effects and evaluates the marginal objective function alongside its exact gradients, eliminating the need for manual derivations or ad hoc approximations. We demonstrate the efficiency, flexibility, and statistical performance of TMB-based NMMs across two numerical examples, including an application to monotonic NMMs. Reproducible code is provided to facilitate broader adoption.