RAG and Knowledge Engineering
Query Decomposition — Stack
Query decomposition › Stack
Technology landscape as of 2026-10-10. Knowledge cutoff May 2026.
Selection criteria
You are choosing an orchestration shape and a planner, and the operational properties matter more than the planner's cleverness.
- Parallel fan-out support. Whether independent branches actually execute concurrently, since a serial implementation turns a four-way split into four times the latency.
- Budget enforcement. Hard caps on branches, hops, total retrievals and total admitted passages, enforced by the framework rather than by the prompt.
- Dependency expression. Whether a plan can state that one sub-query consumes another's result, and whether partial failure of a dependency is handled.
- Constraint propagation. Whether filters applied to the parent query are carried to every branch structurally, rather than being re-derived from text.
- Join control. Whether recombination is a step you own — deduplication, per-branch summarisation, structured extraction — or an opaque concatenation.
- Termination guarantees for iterative plans: hop limits, repeated-query detection, and a defined answer when the budget is exhausted mid-chain.
- Plan observability. Whether the plan, each sub-query, its results and the join are all visible on a single trace.
- Incremental cost. The number of model calls the planner itself adds, separate from the retrievals it causes.
Options
| Option | Best for | Watch out for |
|---|---|---|
| Model-generated sub-query lists executed in parallel | Comparisons and compound questions where the branches are independent | Planner quality is unmeasured by default; branches that should have been dependent run on stale premises |
| Iterative retrieve-then-reason loops (ReAct-style, IRCoT-style interleaving) | Genuine multi-hop questions where the next query depends on the last answer | Serial latency per hop; needs explicit hop limits and repeated-query detection to terminate |
| Retrieval frameworks with sub-question engines (LlamaIndex sub-question query engine, LangChain multi-query and self-query retrievers) | Getting a working plan-and-join quickly with inspectable components | Defaults concatenate results; deduplication and constraint propagation usually need adding |
| Agent orchestration frameworks used for the plan (LangGraph, DSPy pipelines, Temporal-backed flows) | Products that need the plan to be a durable, inspectable, resumable object | Heavier than the problem for simple comparisons; adds a second orchestration surface |
| Structured decomposition against a schema (text-to-query over typed attributes, then one retrieval per value) | Corpora where the branching dimension is a known attribute — region, product, year | Only works where the attribute is populated and trustworthy on the documents |
| Graph or multi-hop retrieval instead of decomposition | Questions whose hops follow explicit relations rather than free-text searches | A different build-time cost entirely; see the graph capabilities rather than treating it as a planner option |
| No decomposition, with a deeper single retrieval and a reranking stage | Most traffic, most of the time | Genuinely multi-hop and comparative questions are answered arbitrarily rather than refused |