Hyper Input Convex Neural Networks for Shape Constrained Learning and Optimal Transport
2026-04-29 • Machine Learning
Machine Learning
AI summaryⓘ
The authors present Hyper Input Convex Neural Networks (HyCNNs), a new type of neural network that always learns convex functions. By combining ideas from Maxout networks and input convex neural networks (ICNNs), HyCNNs use fewer parameters to approximate certain functions and work better than ICNNs and standard neural networks on synthetic tasks. They also show that HyCNNs can learn complex mappings in high-dimensional data, such as those from single-cell RNA sequencing, often outperforming existing methods. This suggests HyCNNs are a more efficient and reliable tool for tasks requiring convex function learning.
Convex functionsInput Convex Neural Networks (ICNNs)Maxout networksNeural network architectureConvex regressionOptimal transportHigh-dimensional dataSingle-cell RNA sequencingFunction approximationPredictive performance
Authors
Shayan Hundrieser, Insung Kong, Johannes Schmidt-Hieber
Abstract
We introduce Hyper Input Convex Neural Networks (HyCNNs), a novel neural network architecture designed for learning convex functions. HyCNNs combine the principles of Maxout networks with input convex neural networks (ICNNs) to create a neural network that is always convex in the input, theoretically capable of leveraging depth, and performs reliable when trained at scale compared to ICNNs. Concretely, we prove that HyCNNs require exponentially fewer parameters than ICNNs to approximate quadratic functions up to a given precision. Throughout a series of synthetic experiments, we demonstrate that HyCNNs outperform existing ICNNs and MLPs in terms of predictive performance for convex regression and interpolation tasks. We further apply HyCNNs to learn high-dimensional optimal transport maps for synthetic examples and for single-cell RNA sequencing data, where they oftentimes outperform ICNN-based neural optimal transport methods and other baselines across a wide range of settings.