Automating UI Optimization through Multi-Agentic Reasoning
2026-02-13 • Human-Computer Interaction
Human-Computer Interaction
AI summaryⓘ
The authors present AutoOptimization, a system that changes user interfaces (UIs) based on what users say they want. It takes the user's spoken preferences and uses an automated process to find the best UI layouts that match those preferences. Instead of relying on average settings or manually checking layouts, their system chooses and adjusts objectives automatically and tests the results to ensure they fit the user's instructions. Multiple intelligent agents work together to understand the user's wishes, set up the optimization problem, and pick the best final design.
multi-objective optimizationuser interface (UI)Pareto frontierobjective functionsparameterizationoptimization solveruser preferencesagent-based systemlayout adaptation
Authors
Zhipeng Li, Christoph Gebhardt, Yi-Chi Liao, Christian Holz
Abstract
We present AutoOptimization, a novel multi-objective optimization framework for adapting user interfaces. From a user's verbal preferences for changing a UI, our framework guides a prioritization-based Pareto frontier search over candidate layouts. It selects suitable objective functions for UI placement while simultaneously parameterizing them according to the user's instructions to define the optimization problem. A solver then generates a series of optimal UI layouts, which our framework validates against the user's instructions to adapt the UI with the final solution. Our approach thus overcomes the previous need for manual inspection of layouts and the use of population averages for objective parameters. We integrate multiple agents sequentially within our framework, enabling the system to leverage their reasoning capabilities to interpret user preferences, configure the optimization problem, and validate optimization outcomes.