← Query Engine

Boundary with Knowledge Store

query-engine · workzone

Retrieval (search and access rights) belongs to Knowledge Store; Query Engine is the RAG layer on top. This page holds the seam: where the store ends and RAG begins — the call boundary; how identity goes down as the filter's input; which access signal the store reads for staleness; and where the live rights check augments the pre-filter in v2. The module's reciprocity node: this is where the inbound links from KS, Auth, and Harvester converge.

The call boundary: query down, candidates up

When the chat model calls search_knowledge, Query Engine calls Knowledge Store's assembled hybrid result in a single query — this is the body of the tool. The four search primitives (vector · lexical · graph · sql), their fusion (RRF), and the ACL pre-filter live entirely in KS; QE neither repeats them nor assembles across stores. The contract is narrow: a query with an identity goes down, a ready list of candidates under access rights comes up — and from there RAG begins.

The boundary is not crossed on an empty base. If Knowledge Store is empty (is_empty), search_knowledge is not offered to the model — there is nothing to call, no downward call is made, and QE simply returns the model's answer from general knowledge.

Knowledge Store retrieval
  • Four primitives: vector · lexical · graph · sql.
  • Fusing the lists — RRF into a single result.
  • ACL pre-filter — rights as a query condition, before ranking.
  • Returns candidates: fragments and entities, under access rights.
query + identity ▾ ▴ candidates under access rights
Query Engine RAG
  • One hybrid-result call — no assembling search of its own.
  • Rerank of candidates — the RRF order from KS.
  • Packing context from the best fragments.
  • LLM answer AI generation.
Identity — the filter's input, not a rights scheme at QE

Query Engine passes the user's identity down — the email and its mapping to source accounts. Knowledge Store expands it into a resolution join and filters candidates in a single SQL before ranking.

Query Engine passes identity
──▶ email → source accounts
Knowledge Store resolution join + pre-filter
──▶ single SQL · before ranking
candidates under access rights RAG input

The source of the rights is with other modules; QE only presents them. Harvester captures rights and identities from the sources and writes them into KS; expanding them into accessible entities is KS's resolution. The platform side of access is held by Auth & Security — the “data ACL” layer: in v1, Hybrid, where the source itself determines access; a platform interpretation on top of source rights is v2.

The hidden-item marker is content-free. The pre-filter removes the forbidden entirely: in the list under rights there is nothing hidden, and “how many were removed” is not visible from it. So alongside the candidates the store returns a light marker of filtered-but-relevant — whether there was a hidden candidate, in which source system, and who its author is (author_principal_id, resolving to an email), without title or content. On it rests the hint about hidden access: the author-contact of “whom to go to” exists in v1; the formal grantor of “who grants access” would require a new field in the rights model — v2.

The model's arguments are untrusted input. search_knowledge takes the query and filters as values, not as SQL, and the identity — from the session, not from the model's arguments. So a prompt injection is powerless: it won't grant the model someone else's access (rights are not its argument), and it won't build a SQL injection (a value does not become code — that is the retrieval layer's concern). We deliberately do not offer text-to-SQL — it would open both vectors.

Access signal for staleness → access_counter

What is queried and how often — only Query Engine sees it. This signal is needed by Knowledge Store's staleness-decay as the “access frequency” input: what is in demand keeps its weight, what is forgotten sinks. There is deliberately no carrier for it in the KS model — the only one who sees the accesses is this module; it also holds the access_counter counter (hits, last_accessed_at per entity). The decay itself and its formula are with Knowledge Store.

An “access” ≠ a raw retrieve. We count it when an entity actually entered an answer or its source was clicked. A raw retrieve is noisy: landing among candidates is not yet a benefit, and counting it would reinforce something that merely surfaced once. Each counted access is a cheap upsert into access_counter (hits++, last_accessed_at); there is no cross-module call on the hot path of the answer.

The signal is not sent — it is read on recompute. Decay is an absolute recompute of trust_score in the Curation Pass from the current inputs. It takes demand the same way it takes authority: by JOIN to the carrier table at run time — authority_tier from Harvester, access frequency from Query Engine's access_counter. There is no separate “send” schedule: the signal's cadence = the Curation Pass cadence, and the hot answer path and the cold curation path are decoupled. What has not been queried for a while (last_accessed_at long ago) decays toward neutral — the forgotten sinks on its own, without deletion.

access_counter · QE hits · last_accessed_at
──▶ JOIN on recompute · like authority_tier
Curation Pass · KS authority × freshness × demand
──▶ absolute recompute
entities.trust_score ranking weight
Live rights check on outputv2

A pre-filter by the stored ACL answers fast, but the cache lags the source: revoked access may still hang in our base — Harvester syncs rights eventually. To keep a stale revocation from leaking into an answer, for the final top-N candidates we do a live rights check at the source (late-binding security trim): we confirm access right before the display.

pre-filter By the stored ACL — the whole index, one SQL. Fast, but eventually consistent.
rerank → top-N Candidates narrowed to a handful of finalists.
live-check at the source We confirm access to each finalist before display — only on the top-N.

The source did not respond — we close (fail-closed). If at check time the source is unavailable (a failure, a timeout), we do not show the candidate: unconfirmed access is treated as denial — the same fail-safe default as across the entire access layer.