Toward a Functional Geometric Algebra for Natural Language Semantics
2026-04-28 • Computation and Language
Computation and LanguageArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors explain that current methods for understanding language in AI mostly use regular math tools like vectors and matrices, which struggle with capturing complex meanings and relationships. They suggest using geometric algebra, a different math system, because it can better represent how concepts combine and relate to each other. Their approach, called Functional Geometric Algebra, aims to improve how machines grasp meaning in a way that fits well with existing AI models. They show examples and explain how this math could make language understanding more organized and interpretable.
distributional semanticslinear algebrageometric algebraClifford algebracompositional semanticsembedding spacetransformer architecturesmultivector algebratyped semanticsneural networks
Authors
James Pustejovsky
Abstract
Distributional and neural approaches to natural language semantics have been built almost exclusively on conventional linear algebra: vectors, matrices, tensors, and the operations that accompany them. These methods have achieved remarkable empirical success, yet they face persistent structural limitations in compositional semantics, type sensitivity, and interpretability. I argue in this paper that geometric algebra (GA) -- specifically, Clifford algebras -- provides a mathematically superior foundation for semantic representation, and that a Functional Geometric Algebra (FGA) framework extends GA toward a typed, compositional semantics capable of supporting inference, transformation, and interpretability while retaining full compatibility with distributional learning and modern neural architectures. I develop the formal foundations, identify three core capabilities that GA provides and linear algebra does not, present a detailed worked example illustrating operator-level semantic contrasts, and show how GA-based operations already implicit in current transformer architectures can be made explicit and extended. The central claim is not merely increased dimensionality but increased structural organization: GA expands an $n$-dimensional embedding space into a $2^n$ multivector algebra where base semantic concepts and their higher-order interactions are represented within a single, principled algebraic framework.