What Makes a Good Fiqh Retriever? Answer Retrieval for Arabic Islamic Jurisprudence
2026-08-20 • Information Retrieval
Information Retrieval
AI summaryⓘ
The authors studied how well computer systems can find exact answers to Islamic legal questions in Arabic, focusing on retrieving passages that clearly state the required ruling. They created a test set to compare different search methods, finding that fine-tuning models and using knowledge of specific Islamic schools (madhhabs) improves retrieval accuracy. Their analysis shows the main difficulty is telling apart passages that mention relevant topics from those that actually give the correct ruling. This helps pinpoint where improvements are needed in such question-answering systems.
Retrieval-Augmented GenerationArabic FiqhAnswer-bearing retrievalMRR@5Dense retrievalLexical retrievalHybrid retrievalMadhhab-aware filteringFine-tuningError analysis
Authors
Somaya Eltanbouly, Heba Sbahi, Samer Rashwani, Abdessalam Bouchekif, Mutaz al-Khatib, Shahd Gaben, Mohammed Ghaly
Abstract
Retrieval-Augmented Generation is used for Islamic question answering, but most systems are evaluated end-to-end, making retrieval failures difficult to isolate from generation failures. We study answer-bearing retrieval for Arabic fiqh, where a passage is relevant only if it states the ruling required by the question. We build a retrieval test collection for Arabic fiqh and use it to evaluate dense, lexical, hybrid, fine-tuned, and madhhab-aware retrieval strategies. The best retriever achieves 0.524 MRR@5, while fine-tuning improves performance to 0.553. Hybrid retrieval provides limited gains for strong models, whereas madhhab-aware filtering more than doubles MRR@5 on school-specific questions. We further present an error analysis showing that the main challenge is distinguishing answer-bearing passages from topically similar passages that do not contain the requested ruling.