Weighted universal approximation of differentiable maps on infinite-dimensional manifolds
2026-06-08 • Machine Learning
Machine Learning
AI summaryⓘ
The authors extended a known result about neural networks that take functions as inputs, showing these networks can not only approximate the outputs but also the derivatives of those outputs. They did this by working in a more general mathematical setting called weighted manifolds and Banach spaces. Their work also applies to special types of functionals important in stochastic analysis, like non-anticipative functionals, and shows that linear combinations of signatures can approximate these functionals and their directional changes. This provides a stronger foundation for understanding how neural networks can handle complicated function inputs and their sensitivities.
Functional input neural networksUniversal approximation theoremWeighted manifoldBanach spaceDifferentiable mapsNachbin theoremNon-anticipative functionalsHorizontal and vertical derivativesSignature of pathsDirectional derivatives
Authors
Philipp Schmocker, Josef Teichmann
Abstract
We generalize the universal approximation theorem for functional input neural networks (FNN) to differentiable maps by including the approximation of the derivatives. A FNN maps the input from a possibly infinite-dimensional weighted manifold to the real-valued hidden layer, on which a non-linear scalar activation function is applied, and then returns the output into a Banach space via some linear readouts. By proving a weighted Nachbin theorem, we establish a universal approximation theorem (UAT) for differentiable maps, which goes beyond the usual formulation on compact sets and also includes the approximation of the derivatives. This leads us to approximation results for non-anticipative functionals including the horizontal and vertical derivatives. As a further application, we show that linear functions of the signature are able to approximate path space functionals including their directional derivatives.