MDTransformer: A Hardware-Software Co-Design of Mode-Division Photonic Transformer Accelerator with Inverse-Designed Coherent Crossbar
2026-07-28 • Hardware Architecture
Hardware ArchitectureArtificial IntelligenceDistributed, Parallel, and Cluster Computing
AI summaryⓘ
The authors introduce MDTransformer, a new photonic transformer accelerator design that uses multiple light modes to perform many calculations at once without needing complex light sources. Instead of relying on costly multi-wavelength components, their design uses spatial-mode interference and special photonic components to carry out matrix multiplications efficiently in the optical domain. This approach reduces size, power, and energy usage while maintaining speed comparable to current advanced systems. Their experiments demonstrate practical improvements for transformer models used in machine learning.
photonic transformer acceleratormode-division multiplexingspatial-mode interferenceMach-Zehnder IQ modulatormatrix multiplicationcoherent detectioncomplex-valued arithmeticcontinuous-wave operationenergy-efficient computing
Authors
Solomon Micheal Serunjogi, Rachmad Vidya Wicaksana Putra, Ayat Taha, Muhammad Shafique, Mahmoud Rasras
Abstract
Recently, photonic transformer accelerators (PTAs) have successfully achieved significant speedup and energy efficiency improvements over electronic accelerators for expediting Transformer inference. However, state-of-the-art rely on expensive multi-wavelength light generation and large dot-product units due to active phase-shifter components, thus making their approach inefficient and impractical. To address this, we propose MDTransformer, a novel hardware-software co-design of PTA based on mode-division optical dataflow and operations. Specifically, MDTransformer performs complex matrix operations using spatial-mode interference, that leverages the inverse-designed multi-mode couplers, crossings, and Mach-Zehnder IQ modulators into a compact mode-division photonic tensor core (MPTC), capable of executing matrix multiplications in the optical domain. Its each guided mode (i.e., TE0-TE3) acts as an independent computational lane, enabling four-fold parallelism-per-waveguide without spectral filtering or free-spectral-range limitations. Moreover, its coherent detection and IQ modulation jointly encode amplitude and phase, realizing complex-valued arithmetic for full-range operations in transformers. MDTransformer offers analog multiplication with sub-4-bit effective precision and inter-modal crosstalk below -30 dB. Its inverse-designed approach also offers scalable and full compatibility with single-laser continuous-wave operation at 1550 nm. Experimental results show that MDTransformer achieves 40.4% area reduction, 63.6% power saving, 40.6% energy saving, and comparable latency over the state-of-the-art PTA across different workloads (i.e., DeiT-Tiny/Small/Base and BERT-Base/Large). These results show that MDTransformer offers a practical solution for high-performance and energy-efficient transformer-based systems.