BracketRank: Large Language Model Document Ranking via Reasoning-based Competitive Elimination
2026-04-10 • Information Retrieval
Information Retrieval
AI summaryⓘ
The authors present BracketRank, a new method for improving how computers pick the most relevant documents by having them compete in a tournament-style process. This method groups documents based on model size limits, uses prompts that require explaining why a document is relevant step-by-step, and eliminates documents in rounds until the best remain. Their tests show BracketRank works better than previous top methods on reasoning-heavy retrieval tasks. This shows that combining reasoning with a competition format helps find better results in complex searches.
document rerankinglarge language modelssemantic inferencenDCGprompt engineeringcompetitive tournamentBRIGHT benchmarkTREC datasetsstep-by-step reasoningcontext window
Authors
Abdelrahman Abdallah, Mohammed Ali, Bhawna Piryani, Adam Jatowt
Abstract
Reasoning-intensive retrieval requires deep semantic inference beyond surface-level keyword matching, posing a challenge for current LLM-based rerankers limited by context constraints and order sensitivity. We propose \textbf{\BracketRank}, a framework that treats document reranking as a reasoning-driven competitive tournament. Our approach introduces three key innovations: (1) adaptive grouping based on model context limits, (2) reasoning-enhanced prompts that mandate step-by-step relevance explanations, and (3) a bracket-style elimination structure with winner and loser tracks. This design ensures robust document advancement while enabling parallel processing across competition stages. Evaluation on the BRIGHT reasoning benchmark shows that \BracketRank achieves \textbf{26.56 nDCG@10}, significantly outperforming state-of-the-art baselines including RankGPT-4 (17.0) and Rank-R1-14B (20.5). On TREC datasets, BracketRank achieves 77.90 nDCG@5 on DL 19 and 75.85 nDCG@5 on DL 20, exceeding all baselines, establishing that explicit reasoning within competitive elimination is a powerful paradigm for complex, multi-step retrieval tasks. https://github.com/DataScienceUIBK/BracketRank