Rethinking Sales Lead Scoring with LLM-based Hierarchical Preference Ranking
2026-06-03 • Information Retrieval
Information RetrievalArtificial Intelligence
AI summaryⓘ
The authors studied how to better score sales leads in complex industries like automotive, where decisions take a long time and involve many steps. They found that usual methods struggle because data is limited and difficult to interpret. They created a system using large language models combined with structured customer data, plus a new ranking approach called HPRO that understands the sales process hierarchy. Their method improved accuracy in lead scoring and increased actual sales in a long online test. This shows the approach helps prioritize leads more effectively in real business settings.
Sales lead scoringLarge Language ModelsHierarchical Preference RankingBradley-Terry modelCRM (Customer Relationship Management)Pointwise learningPairwise learningLead conversionA/B testingNEV (New Energy Vehicle)
Authors
Chenyu Zhang, Yiwen Liu, Yin Sun, Xinyuan Zhang, Yuji Cao, Junming Jiao, Juyi Qiao
Abstract
Sales lead conversion in high-stakes domains (e.g., automotive, real estate) differs fundamentally from e-commerce recommendation due to prolonged decision cycles and multi-stage funnels. Traditional lead scoring methods rule-based scorecards, machine learning, or pointwise CTR models face severe challenges: sparse supervision, a semantic gap in unstructured CRM logs, and inability to capture relative lead priority. While Large Language Models(LLMs) offer superior semantic understanding of customer interactions, general-purpose LLMs are ill-suited for lead ranking: they generate text rather than comparable scores, and lack alignment with the hierarchical priorities of sales funnels. We introduce an LLM-based discriminative framework for sales lead scoring, which supports joint modeling of structured CRM features and unstructured customer interactions. On top of this framework, we propose HPRO (Hierarchical Preference Ranking Optimization), which augments sales lead scoring with a hierarchical preference ranking objective. HPRO employs a margin-aware Bradley-Terry formulation to transform sparse binary labels into dense, funnel-aware preference pairs, enabling lead scoring to leverage both pointwise and pairwise supervision. Experiments on large-scale data from a leading NEV brand demonstrate state-of-the-art classification (AUC 0.8161) and ranking performance (+39.7% precision among top-ranked leads). A 132-day online A/B test validates 9.5% sales volume uplift, confirming real-world commercial impact.