RAG and Knowledge Engineering
Query Rewriting — Stack
Query rewriting › Stack
Technology landscape as of 2026-10-10. Knowledge cutoff May 2026.
Selection criteria
Most of this is a prompt, a small model, or a dictionary, so the criteria are about latency, blast radius and whether the effect can be measured at all.
- Added latency at the median and the tail, measured against the retrieval it is meant to improve. A rewrite that costs more than the search is rarely worth it.
- Reversibility. Whether the original query is retained and can be retrieved alongside the rewrite, which is what makes a bad rewrite recoverable.
- Determinism. Whether the same query and the same history produce the same rewrite, which decides whether caching is possible and whether a regression is reproducible.
- Multilingual behaviour, weighted by real traffic rather than by the number of languages claimed.
- Controllability. Whether constraints the user stated — dates, product names, quantities — can be pinned so a rewrite cannot drop them.
- Observability. Whether the rewrite, its inputs and the decision not to rewrite are all emitted as attributes on the request.
- Vocabulary maintenance for dictionary-based expansion: who owns the synonym and abbreviation lists, and how a new internal term gets added.
- Cacheability. Whether rewrites for repeated queries can be reused, since query traffic is heavily repetitive.
Options
| Option | Best for | Watch out for |
|---|---|---|
| Instructed small language model for context resolution | Multi-turn products where follow-up turns are the dominant query shape | A model call on every turn; resolution errors are confident and invisible without a trace |
| Generated hypothetical answer used as the query (HyDE-style) | Corpora of declarative prose where questions and passages are stylistically far apart | The generation can invent specifics that dominate retrieval; worthless when the corpus is already question-shaped |
| Controlled vocabulary and synonym expansion (engine-side synonym maps, curated abbreviation lists) | Organisations with heavy internal jargon and stable terminology | Maintenance has no natural owner; over-broad synonyms dilute precise queries |
| Spelling correction and normalisation (SymSpell, engine-native fuzzy matching) | High-volume consumer-style traffic with typos and transliteration | Corrects identifiers and product names into the wrong thing; needs an allow-list of protected tokens |
| Multi-query generation with fusion of the result sets | Ambiguous questions where several readings are plausible | Multiplies retrieval cost; the fusion must deduplicate or the budget fills with one passage |
| Query classification before rewriting (small classifier or rules) | Avoiding the cost and risk of rewriting queries that do not need it | Another component with its own error rate, in front of the one it is protecting |
| Framework-provided rewriters (LangChain and LlamaIndex query transformation components) | Assembling and comparing several strategies quickly | Defaults are generic; the prompt is the whole behaviour and needs to be owned, not inherited |