An LLM-powered Agentic Recommendation System for Connected TV Content Discovery
2026-07-10 • Information Retrieval
Information RetrievalArtificial Intelligence
AI summaryⓘ
The authors created a recommendation system for Connected TV that uses large language models (LLMs) to better handle different kinds of context like news, events, and user activity without needing lots of manual data setup. Since LLMs alone aren’t always fast or precise enough for all parts of the recommendation process, they combined LLMs with traditional machine learning methods in a hybrid system. This approach lets each task be done by the best-suited method, improving overall performance while keeping the flexibility of LLMs. They also addressed practical challenges like the speed of making recommendations and shared lessons from their engineering efforts.
recommendation systemslarge language modelsConnected TVcontextual signalsmachine learningfeature engineeringhybrid systemsinference latencypersonalizationagentic architecture
Authors
Lei Shi, Di Wang, Harry Tran, Helsing Xu, Yuchen Lu, Dhara Ghodasara, Wilson Chaney, Xueting Liao, Jerry Yu, Huayu Ding, Mingze Gao, Shike Mei, Shuo Tang, Zhe Zhang, Jianming He, Abhishek Kumar, Haotian Wu, Hamed Firooz, Li Li
Abstract
Recommendation systems, from traditional multi-stage to recent unified generative architectures, face challenges in incorporating diverse contextual signals, such as trending topics, breaking news, cultural events, and cross-surface user activities, into their ranking pipelines. These systems are designed to consume structured behavioral signals with consistent schemas, and lack the reasoning capability to naturally process unstructured or heterogeneously formatted contextual information. Incorporating such signals typically requires feature engineering, bespoke data pipelines, and carefully tuned heuristics. In this paper, we present an LLM-powered agentic recommendation system designed for Connected TV (CTV) content discovery that addresses these limitations. Our system leverages the reasoning capabilities of large language models to naturally process and synthesize diverse signals across varying schemas and structures, eliminating much of the manual integration inherent in traditional ranking and retrieval systems. Recognizing that current LLM-based solutions still fall short of traditional machine learning models in several recommendation tasks, including retrieval efficiency, personalization precision, and scalability, we adopt an agentic architecture that orchestrates specialized components, allowing each sub-task to be handled by the most suitable method, whether LLM-based or traditional ML. The main contribution of this work is our engineering approach to successfully overcoming the practical limitations of enabling LLM for recommendation, particularly inference latency. We share insights from our work and discuss the trade-offs and lessons learned in building a hybrid system that combines the flexibility of LLMs with the performance of established recommendation techniques.