Lighthouse RL: Sample-Efficient Circuit Optimization via Strategic Reset Points
2026-07-15 • Machine Learning
Machine LearningHardware Architecture
AI summaryⓘ
The authors present Lighthouse RL, a new method that makes training AI to size analog circuits more efficient. Unlike usual methods that explore a lot of less useful options, their approach re-starts training from previously found good solutions called "lighthouses," which helps focus on better options. They tested their method on simulated problems and real circuits, showing faster learning and better success rates than other techniques. Their strategy can also improve other AI optimization methods by guiding exploration more smartly.
reinforcement learninganalog circuit sizingsample efficiencyblack-box optimizationBayesian optimizationexploration-exploitationreset strategyoptimization performancegeneralization
Authors
Mustafa Emre Gürsoy, Stefan Uhlich, Ryoga Matsuo, Yağız Gençer, Arun Venkitaraman, Chia-Yu Hsieh, Andrea Bonetti, Eisaku Ohbuchi, Lorenzo Servadei
Abstract
In this paper, we introduce Lighthouse RL, a sample-efficient reinforcement learning (RL) approach for analog circuit sizing. Traditional methods lack generalization across different performance targets, while standard RL approaches waste resources exploring unpromising regions. Our method addresses these inefficiencies through a strategic reset strategy that initializes episodes from high-performing configurations discovered during training, called "lighthouses". These states, which are closer to the target objectives, guide exploration toward promising regions. When compared to RL and Bayesian optimization methods from the literature, we demonstrate the effectiveness of our approach on a 2D benchmark problem and on two analog circuits, showing significant improvements in sample efficiency (up to 1.72x faster), optimization performance (100% vs. 0-87% success rate), generalization (75% vs. 0-50% extrapolation success), and objective maximization. This efficiency is particularly valuable for computationally expensive black-box optimization problems, and our reset strategy can be used as a plug-and-play enhancement for any RL-based optimization approach.