Dynamic Ranked List Truncation for Reranking Pipelines via LLM-generated Reference-Documents
2026-04-10 • Information Retrieval
Information Retrieval
AI summaryⓘ
The authors address the problem of making Large Language Models (LLMs) more efficient for reranking search results, which normally requires a lot of computing power and long text contexts. They propose generating special reference documents using LLMs that help distinguish relevant from non-relevant items in a list, making it easier to trim and rerank the list efficiently. Their method processes these lists in smarter, overlapping segments rather than fixed steps, speeding up the reranking process. Experiments show that their approach is faster and more effective than previous methods across different datasets.
Large Language Models (LLM)rerankingranked list truncation (RLT)relevance judgmentreference documentslistwise rerankingcontext lengthTREC Deep Learning benchmarks
Authors
Nilanjan Sinhababu, Soumedhik Bharati, Debasis Ganguly, Pabitra Mitra
Abstract
Large Language Models (LLM) have been widely used in reranking. Computational overhead and large context lengths remain a challenging issue for LLM rerankers. Efficient reranking usually involves selecting a subset of the ranked list from the first stage, known as ranked list truncation (RLT). The truncated list is processed further by a reranker. For LLM rerankers, the ranked list is often partitioned and processed sequentially in batches to reduce the context length. Both these steps involve hyperparameters and topic-agnostic heuristics. Recently, LLMs have been shown to be effective for relevance judgment. Equivalently, we propose that LLMs can be used to generate reference documents that can act as a pivot between relevant and non-relevant documents in a ranked list. We propose methods to use these generated reference documents for RLT as well as for efficient listwise reranking. While reranking, we process the ranked list in either parallel batches of non-overlapping windows or overlapping windows with adaptive strides, improving the existing fixed stride setup. The generated reference documents are also shown to improve existing efficient listwise reranking frameworks. Experiments on TREC Deep Learning benchmarks show that our approach outperforms existing RLT-based approaches. In-domain and out-of-domain benchmarks demonstrate that our proposed methods accelerate LLM-based listwise reranking by up to 66\% compared to existing approaches. This work not only establishes a practical paradigm for efficient LLM-based reranking but also provides insight into the capability of LLMs to generate semantically controlled documents using relevance signals.