Query Decomposition — Patterns

Branch on an enumerated attribute rather than on free text

Problem. A model asked to split a comparison invents the branches, and the branches it invents are not guaranteed to be the right set — it may compare two of three regions, or produce two sub-queries that differ only in phrasing and retrieve the same passages.

Approach. Where the comparison dimension corresponds to a structured attribute in the corpus, enumerate the values from the index rather than from the model, and issue one retrieval per value with that value as a hard predicate. The plan becomes a fan-out over a known set, each branch is guaranteed to be about a distinct subject, and a missing branch is detectable because the enumeration said it should exist.

Trade-off. This only applies where the dimension is an attribute, which excludes most comparisons in a prose corpus. Enumerating from the index also means the plan's completeness depends on the attribute being populated everywhere, and a document missing the attribute is silently excluded from the comparison.

Per-branch reduction before the join

Problem. Concatenating every branch's passages into one context multiplies length, buries each branch's evidence among the others, and lets the strongest-worded branch dominate the synthesis regardless of relevance.

Approach. Reduce each branch to a small, attributed result before combining — a short extract with its citations, or a structured value where the question is quantitative — and perform the comparison over the reduced results rather than over raw passages. Each reduction is grounded in its own branch's evidence only, which also keeps the attribution clean.

Trade-off. Reduction is a model call per branch, so a four-way split costs five calls rather than one, and anything the reducer drops is unavailable to the final synthesis no matter how relevant it turns out to be. It also introduces a second place where a fabrication can enter.

Explicit gaps carried into the answer

Problem. A branch that returns nothing is indistinguishable, by the time the join runs, from a branch that was never issued. The synthesis fills the hole with a confident general statement and the answer claims a completeness it does not have.

Approach. Make each branch's result a record with a status, so that an empty retrieval is a present object marked as such rather than an absence. Carry those records into the synthesis and require the answer to state which parts of the question could not be evidenced, and give the caller a structured signal so it can decide whether a partially evidenced answer should be shown at all.

Trade-off. Users dislike answers that enumerate what the system could not find, and product pressure to suppress the gap statement is constant. There is also a real ambiguity the status cannot resolve: an empty branch may mean the corpus lacks the material or that the sub-query was badly formed, and those warrant different responses.

Budget as a hard ceiling on the plan, not a prompt instruction

Problem. Fan-out and hop count are the two variables that make decomposition expensive, and both are typically left to a model instructed to be reasonable. Under unusual input the model is not reasonable, and the cost is unbounded.

Approach. Enforce the limits in the executor: a maximum number of branches, a maximum number of hops, a maximum total number of retrievals, and a maximum number of passages admitted to the final context. Exceeding a limit truncates the plan and marks the result as budget-limited rather than failing, so the answer is degraded and attributed rather than absent. Record the realised breadth and depth on every request so the distribution is visible.

Trade-off. Truncating a plan mid-execution produces exactly the partially evidenced answers the previous pattern has to disclose, and the limits that are safe for the common case will occasionally cut off a question that genuinely needed more. The limits are also a product decision disguised as a configuration value.