If you run a chatbot over your own docs, course notes or support articles, you have probably read a dozen posts promising that one more technique (a reranker, query rewriting, hybrid search) will lift accuracy by some double-digit percentage. A new paper from IBM researchers measured what happens when you add those techniques together. The gains shrink, and the two things you care about most, finding the right passage and answering truthfully from it, move independently.
The paper
Title: Retrieval Augmented Generation Compositionality (the framework is called LASAGNA)
Authors: Ido Levy, Segev Shlomov and Leshem Choshen. The full text lists all three at IBM, with Levy and Choshen also at the Weizmann Institute of Science.
Published: 21 October 2026 on arXiv (2610.29722). 103 pages, of which nine are the main paper.
Code: lasagna on GitHub, Apache 2.0.
The finding in plain English
Most RAG techniques are tested one at a time: take a baseline pipeline, swap in a reranker, report the lift. Builders then add up those lifts and assume the stack will deliver the total. The authors ran 450 configurations across eight benchmarks, with corpora from 5K to 200K documents, to check that assumption.
It did not hold. Three retrieval techniques that each helped on their own (cross-encoder reranking, turning off MMR diversity filtering, and adding document-awareness prefixes to chunks) predicted a combined lift of about 47% in retrieval quality. Used together they delivered +17.2%, a 63% shortfall. Across 17 tested combinations, the shortfall ranged from 7% to 72%, with a median of 41%. In the paper's tests, the median combination lost 41% of the gain added up on paper.
Two more findings matter just as much:
Retrieval quality and answer faithfulness are separate dials. Across matched configurations, the correlation between retrieval quality (NDCG@10) and faithfulness was r = -0.01, effectively zero. Embedding model, index-time enrichment and reranking drove retrieval. Query augmentation and agentic quality gates drove faithfulness. Improving one tells you nothing about the other.
The best stack depends on corpus size. The simpler "standard" profile led at 5K and 15K documents. The heavier "balanced" profile led from 25K onward, and at 200K documents it beat standard by 13.7% relative (p = 0.003). Simpler pipelines degraded steadily as the corpus grew; the heavier ones stayed roughly flat.
Faithfulness behaved a little differently. Its predicted gain from the best single changes was +28% and the realised gain was +32%, so on that metric the components did roughly add up. The disappointment sits on the retrieval side.
How they did it
The team built LASAGNA, a framework that lets you swap one stage of a RAG pipeline while holding everything else fixed. The stages and options they tested:
Embedding: MiniLM-L6, BGE-M3, E5-large, BGE-large
Chunking: fixed, semantic, sentence-window, hierarchical
Index-time enrichment: none, contextual retrieval, RAPTOR, GraphRAG
Retrieval: dense, hybrid with BM25, hybrid with SPLADE
Query augmentation: none, HyDE, RAG-Fusion, decomposition, step-back
Reranking: none, a single cross-encoder, an ensemble, MMR
Compression: none, LLMLingua
Agentic quality gate: none, CRAG, Self-RAG, Speculative-RAG
Retrieval was scored on BEIR datasets (NFCorpus, FiQA, SciFact, ArguAna, HotpotQA) and generation on question-answering sets including SQuAD v2 and MuSiQue. GPT-5.1 generated answers and acted as the main faithfulness judge using RAGAS, with Claude models as second and third judges. The base corpus of about 5,100 documents was scaled to 15K, 25K, 50K, 100K and 200K.
"Compositionality" is measured simply: add up the gains of the best single-component changes (the prediction), build the pipeline with all of them (the reality), and report how much of the prediction went missing.
The paper also bundles techniques into four profiles. Its headline table, averaged over three seeds:
Profile | Retrieval (NDCG@10) | Faithfulness | Latency |
|---|---|---|---|
standard | 0.860 | 0.739 | 5.30s |
balanced | 0.840 | 0.911 | 12.25s |
max-reliable | 0.804 | 0.916 | 14.16s |
speed | 0.541 | 0.524 | 0.70s |
The authors note that no profile leads all metrics. Balanced gives up a little retrieval quality for a large gain in faithfulness, at more than double the latency of standard.
Why it matters
This research has a direct consequence for anyone selling or running a docs bot, a paid community assistant or a "chat with my course" product. If you choose components by reading benchmark tables, you will overestimate what the finished system does. If you tune only retrieval (the easiest number to measure), you may ship a bot that finds the right page and still makes things up. And if you tune on a 2,000-document pilot, the winning setup may stop winning once your archive grows past 25,000.
It fits a pattern we have covered before: in More memory, worse agents, saving more lessons could make agents repeat their mistakes. Adding more components to a pipeline carries the same risk unless you measure each addition.
Caveats
Benchmarks, not your data. The authors call their results case-study evidence. Everything runs on one constructed English multi-domain test bed, with at most 93 retrieval and 200 generation queries per experiment.
Proprietary models only. All headline results use API models. No open-weight generators, no multilingual corpora, no private company data.
Small confirmation set. One confirmatory experiment used 43 unique queries instead of the pre-registered 200.
Mixed hypotheses. Of their three pre-declared hypotheses, only the scale one was clearly supported. The claim that interactions explain at least 20% of faithfulness variance was not met, and the idea of a single general "balanced" recipe was not supported.
Licensing. The repo says some selectable models, including the default SPLADE encoder and monoT5 reranker, are not licensed for commercial use. Check its model licence notes before you ship anything built on them.
How to use it today
You do not need 450 configurations. You need a small, honest test you can repeat. This takes an afternoon.
1. Build a 40–60 question eval set from your own documents. Pull real questions from support tickets, community posts or DMs. For each one, record the document (or section) that holds the answer and a one-line reference answer. Include 5–10 questions your docs cannot answer, so you can check the bot says so. If you have never built a held-out set, our guide to tuning with a held-out test set covers the mechanics.
2. Measure two numbers separately, every run.
Retrieval recall@5: did the correct document appear in the top five chunks? You can score this with a script, no judge needed.
Faithfulness: is every claim in the answer supported by the retrieved chunks? Use an LLM judge with a strict rubric, and hand-check 10 answers per run so you trust the judge.
Log latency and cost per question alongside both.
3. Ablate one technique at a time from a plain baseline. Start with dense retrieval and no extras. Then add one change per run: a reranker, hybrid keyword search, query rewriting, a verification step. Record what each adds on its own.
4. Then test the full stack and compare. Add up the single-change gains, build the combined pipeline, and measure it. If the combined result is well below the sum, drop the component with the smallest solo gain and re-test. Keep whatever earns its latency.
5. Re-test as the corpus grows. Re-run the same eval set each time your document count roughly doubles, or at least every quarter. The paper's crossover between 15K and 25K documents is a reason to schedule this.
If you want a ready-made harness, the repo installs with pip install lasagna-rag (the package called lasagna is a different project), accepts your own JSONL, CSV or BEIR-format data, and ships speed, standard, balanced and max-quality profiles you can compare directly:
pip install lasagna-rag
python -m nltk.downloader punkt_tab
lasagna configure --profile standard
lasagna upload handbook.pdf
lasagna search "what is the refund window?"
For non-builders using a hosted "chat with your docs" tool, the same test still works. Write 20 questions you know the answers to, ask them, and note separately whether the tool cited the right source and whether the answer matched that source. Repeat after a big upload.
Sources
Levy, Shlomov, Choshen, "Retrieval Augmented Generation Compositionality", arXiv 2610.29722, abstract page, 21 Oct 2026: https://arxiv.org/abs/2610.29722
Same paper, full text (HTML), including affiliations, Table 2, scale and correlation results, limitations: https://arxiv.org/html/2610.29722
LASAGNA framework README (install, profiles, licensing): https://github.com/dolev31/lasagna