Defeat the Heap: Zero-Copy Data Movement in AXI4MLIR
2026-06-09 • Hardware Architecture
Hardware ArchitectureProgramming Languages
AI summaryⓘ
The authors study how data is moved between a computer's main memory and custom hardware accelerators used in machine learning. They point out that current methods copy data unnecessarily, which slows things down. By changing how memory is allocated, they eliminate this extra copying step, making data transfer faster and more efficient. Their tests show this improvement reduces memory movement by half and helps the hardware work better.
hardware acceleratorMLIRAXI4MLIRzero-copyDMAdata transfermemory buffermatrix multiplicationcompiler optimization
Authors
Elam Cohavi, Nicolas Bohm Agostini, Jude Haris, Antonino Tumeo, David Kaeli, José Cano
Abstract
As custom hardware accelerators become increasingly central to machine learning workloads, efficient data transfer is critical for maximizing accelerator performance on linear algebra kernels. AXI4MLIR, an extension of the Multi-Level Intermediate Representation (MLIR) compiler framework for automated generation of host-accelerator driver code, incurs significant runtime overhead due to non-zero-copy CPU-accelerator data movement. During transfers from the host to the accelerator, data is copied from heap-allocated memory buffers into contiguous Direct Memory Access (DMA)-mapped buffers. This work identifies this copy as a redundant staging operation and eliminates it through zero-copy data movement. The optimization extends accel, an MLIR dialect introduced by AXI4MLIR, and implements lowering support that allocates buffers directly within DMA-mapped memory, thereby omitting the staging copy. We evaluate the proposed scheme using a configurable matrix-matrix multiplication accelerator and show that the zero-copy optimization reduces main memory data movement by up to 2x, increasing overall accelerator utilization.