Can LLMs Design Video Coding Tools? A Case Study on Planar Mode
2026-09-01 • Multimedia
MultimediaArtificial Intelligence
AI summaryⓘ
The authors studied whether large language models (LLMs) can create new tools for video compression, which is a complex task. They focused on improving the Planar mode, a standard technique used to predict video data. By having the LLM suggest changes and testing these in video encoders, they found the LLM-generated tools could slightly improve compression efficiency with a small extra computational cost. Their tests on different encoder models showed that LLM-designed predictors might help in video coding, though the approach still has challenges to address.
Large Language ModelsVideo CodingPlanar ModeIntra PredictionBitrate SavingsVideo EncoderCompression EfficiencyFraunhofer VVenCEnhanced Compression ModelSyntax Elements
Authors
Yingwen Zhang, Meng Wang, Liqiang He, Shiqi Wang
Abstract
This paper explores whether large language models (LLMs) can design video coding tools, a highly challenging task due to the intricate algorithmic coupling of tool modifications. In particular, we present an empirical case study on the Planar mode, a long-standing intra prediction tool in video coding standards. Our experiments operate within a generation-and-evaluation loop, with the LLM generating new Planar predictors, encoder trials evaluating their coding performance, and the LLM re-generating refined implementations based on the evaluation feedback. We first examine directly replacing the default Planar mode in the Fraunhofer Versatile Video Encoder (VVenC) under its faster preset. Experimental results demonstrate that the LLM-generated mode can outperform the conventional Planar mode on this lightweight toolset, achieving 0.18% bitrate savings with 0.4% complexity overhead on the standard benchmark. We further extend our evaluation to the Enhanced Compression Model (ECM). Leveraging newly introduced directional Planar modes, we investigate two integration strategies: directly replacing them, and introducing the LLM-generated predictor as an additional prediction mode with new syntax elements. The empirical results suggest that both strategies can yield coding gains under a constrained low-resolution setting. Overall, this study offers preliminary evidence and practical insights, highlighting both the potential and open challenges of LLM-based coding tool design.