← Knowledge Store

Tests

knowledge-store · workzone
Decisions made
One database — tested with one SQL, not projection mocks The load-bearing pillar of the design is body, vectors, and graph in one Postgres, with rights enforced by a single JOIN. So the core of coverage is integration against a real Postgres with pgvector: ACL pre-filter, graph traversal, and upsert are verified on a live database, not on a substituted store. Mocks cover only pure logic without a database (chunking, fusion, decay).
Type sets the level, priority is set by markers Unit — deterministic logic without a database, on every PR. Integration and API — against a test Postgres+pgvector in Docker, as a separate step. P0 holds the load-bearing invariants (ACL never leaks, upsert is idempotent), P1 — curation and modes; priority is orthogonal to type (pytest -m p0).
The coverage boundary matches the module boundary KS is responsible for retrieval and storage: primitives, fusion, ACL filter, lifecycle. RAG on top (rerank, packing, the LLM call) belongs to Query Engine and is not tested here. Rights capture and the Entity contract belong to Harvester; at the seam we only verify that what arrives is persisted correctly and read back under rights.
Stack and infrastructure
In place done pytest, pytest-asyncio, pytest-cov, httpx
Database testcontainers — a test Postgres with the pgvector extension (vectors + recursive SQL in one database); Alembic migrations are applied when the container starts
Factories factory_boy for Entity / chunks / entity_edge and the ACL five-tuple — building the graph and scene rights without going through Harvester
Embedding a fake embedding model (deterministic vector from text) — so that similarity search is reproducible, without real inference in the automated run
Other time-machine (age for staleness decay), Curation Pass steps invoked directly — without spinning up the scheduler
Markers @pytest.mark.unit / @pytest.mark.integration / @pytest.mark.api — by type; @pytest.mark.p0 / p1 — by priority, orthogonal to type
Unit Pure logic without a database deterministic · every PR
test_chunking.py
Slicing the body into chunks, ordinal, content hash
Unit → chunks
Cases
body → chunksa long body is sliced into pieces; a short one — a single chunk; an empty one — none
ordinal in orderchunks are numbered consecutively from the start of the text — UNIQUE (entity_id, ordinal) holds
content_hash per chunkthe hash is computed over the piece's text; the same text → the same hash
re-embedding only on text changeediting one chunk → its hash changes, the neighbors keep theirs; only the changed one is re-embedded
token_countthe piece's length in tokens is set; the routine respects the token budget per chunk
test_fusion.py
Fusing four ranked lists into one
Cases
four lists → onevector / lexical / graph / sql merge into a single ranked result over constructed lists
common key — the entitychunk hits (vector / lexical) collapse to the parent by entity_id, the best chunk stays with the entity; one entity appearing in several lists is deduplicated
RRF by reciprocal rankweight comes from position, not the raw score (incomparable across paradigms); a candidate shared by several lists rises
weighted fusionprimitive weights shift the final order predictably
determinismthe same inputs → the same output bit-for-bit; the tie-break is stable
partial inputsone primitive is empty / returned fewer — fusion does not fail, the rest are counted
test_emptiness.py
The derived is_empty property — whether there is anything to search
Cases
no chunks → is_emptynot a single row in chunksis_empty is true; the source of truth is the presence of chunks to search, not the count of sources or entities
first chunk → togglethe first chunk accepted → is_empty is false; the property changes by itself, without a separate command or flag
embedder assigned, no chunks → still emptyan assigned harvester_embedding with zero chunks does not make the base non-empty — emptiness is defined by chunks, not by ingestion readiness
derived, not a stored flagthere is no separate state column; is_empty is computed from the presence of chunks (the implementation may cache), never set by hand — like is_available() on SMTP
test_staleness_decay.py
The decay function that lowers trust_score
Cases
authority — the baselineat equal age and demand, a source with a higher authority_tier (high > normal > low) yields a higher trust_score; the tier is read by a JOIN on source_id
age → loweran old source_updated_at yields a lower trust_score, a fresh one — higher (time-machine)
access frequencythe unrequested drops; what is queried holds its weight; demand is read by a JOIN from access_counter (hits · last_accessed_at), long-unqueried decays toward neutral
monotonicitythe function does not grow with aging; no jumps or negative values
weight only, not accessdecay changes trust_score, does not touch status / is_deleted — the stale stays visible
test_traversal_builder.py
The recursive-CTE builder for graph traversal
Cases
depth 1–3the assembled recursive-CTE limits the traversal to the given depth of steps
cycle protectiona cycle in the edges does not loop the traversal — visited nodes are not repeated
direction and rel_typetraversal along src→dst with a filter on relation type is built correctly
slot for the ACL JOINthe template leaves room for a JOIN of entity_acl on visited ids — rights land in the same query
Integration · P0 Load-bearing invariants against Postgres + pgvector testcontainers · separate CI step
test_acl_prefilter.py
ACL pre-filter in one SQL — the disallowed never arrives
IntegrationP0 → ACL is applied once
Cases
vector under rightssimilarity search over chunks returns only chunks of allowed entities — the selection by the same SQL, before ranking
lexical under rightsfull-text search (text_tsv + GIN) is filtered by the same ACL JOIN — an exact word from a disallowed entity does not leak
chunk inherits ACLa chunk has no ACL of its own; access is resolved through entity_identity_acl
group grant vs directvisible via membership (group_membership) OR a direct grant (entity_acl.source_principal_id); the resolution chain from users
public — a wildcard on the rowa grant with scope='public' (both principal/group NULL) is visible to all — to a user with no memberships and to an anonymous caller under the source's rules; it goes into the filter as scope='public', without a synthetic group
scope ↔ fields consistentCHECK scope IN (group/principal/public) plus consistency: groupsource_group_id filled, principalsource_principal_id, public → both NULL; a malformed combination is rejected by the DB
revoked does not leakremoved from the group / grant withdrawn → the entity disappears from results without a separate post-filter
empty accessa user with no identities / memberships → empty results, not an error and not a leak
test_graph_traversal.py
entity_edge traversal under the same ACL JOIN
IntegrationP0 → entity_edge
Cases
1–3 step traversalrecursive SQL from the start nodes brings nearby context: the thread, hierarchy, mentions
single ACL on the traversalthe traversal is covered by the same JOIN of entity_acl — nodes without access do not enter the result, without a separate rights projection
an inaccessible node breaks the patha path through a disallowed entity does not "jump over" it and does not leak further
cycle in the grapha real edge cycle on a live DB does not loop the CTE
fanout cap per stepa hub node with thousands of edges: the fanout cap limits the number of children expanded from one node per step — the traversal does not blow up into tens of thousands of edges; the cutoff fires before the ACL JOIN, not after
filtering by weightedges below the weight threshold are dropped from the traversal — weak links are not pulled into context; the filter applies before ACL, holding back inflation from a hub node alongside the fanout cap
test_entity_upsert.py
Idempotent upsert by the source key
IntegrationP0 → entities
Cases
one key — one rowby UNIQUE (source_id, source_type, source_entity_id) a repeated upsert updates, does not spawn duplicates
projections in orderbody → chunksentity_edge in one transaction; FKs hold
content_hash changeediting body → re-embedding of the affected chunks; unchanged ones are not re-embedded
chunk set syncthe text was shortened → surplus chunks are deleted, none orphaned (CASCADE on entity_id)
author → SET NULLdeleting a source_principal nulls entities.author_principal_id (ON DELETE SET NULL) — the entity survives; deleting a source carries away all its content via CASCADE
status — from the dictionary onlya non-empty status value outside (draft/final/archived) is rejected by CHECK; NULL is allowed — until the classify step
test_soft_delete.py
is_deleted hides from results, does not delete physically
IntegrationP0 → entities
Cases
hidden from resultsis_deleted = true → the record and its chunks drop out of all four primitives, the row is physically alive
chunks carry a mirror of the flagthe denormalized chunks.is_deleted is set in the same transaction; partial HNSW/GIN (WHERE NOT is_deleted) exclude the deleted from the index traversal
a separate axis from statussoft-delete does not touch status (draft/final/archived) — the axes are independent
restoreclearing the marker returns the record to results with its previous status
deleted_at set by reconciliationthe moment of disappearance is stamped when it vanishes from the source, not on the increment
test_identity_bridge.py
Auto-linking identity↔users by exact email
IntegrationP0 → identity ↔ users
Cases
link from both sidesAuth creates users → KS sets identity.user_id by exact lower(email); Harvester upserts identity → KS backfills the link via users; KS holds both points
case normalizationALICE@CORPalice@corp link up — reconciled by UNIQUE (lower(email)), no case collisions
1:1 bridgepartial unique (user_id) WHERE user_id IS NOT NULL — one identity per users; a repeated link is idempotent, does not spawn a second binding
no match → NULLthe email was not found (a different address / absent) → user_id = NULL, the identity stays unmatched for manual linking in Admin, not an error
link is synchronousthe bridge is set at the moment of creation/upsert, does not wait for the next sync wave — on the very first request access resolves through user_id
Integration · P1 Curation Pass, refresh, journal, recall, backup testcontainers · separate CI step
test_curation_pass.py
Graph curation steps over the already-assembled base
IntegrationP1 → Curation Pass
Cases
cross-source edgea mention materializes into entity_edge with origin = curation once both nodes are already in the graph; an entity_ref request until then, and after materialization the row is deleted
forward reference within a sourcea deferred reference to a not-yet-loaded target of the same source waits as an entity_ref request; the run materializes the edge (origin = curation) after the target arrives in the next increment — the request is cleared
a user target resolves into the bodya request with target_kind = user (e.g. author → identity) is settled by writing author_principal_id into the entity body; no entity_edge is created — not every request becomes an edge
node mergetwo entities recognized as one object collapse: edges are moved over, the duplicate is marked duplicate_of
fuzzy identity reconciliationunreconciled source_principal under different emails are matched, identity_id is set; the bridge to users is backfilled across different emails
decay writes trust_scorethe staleness step records the lowered entities.trust_score into the DB
retention v2by policy: archival hides from hot results, deletion purges physically together with chunks via CASCADE
steps are idempotenta repeated run does no harm; a partial failure of one step does not break the rest
the unreconciled — for reviewwhat could not be fuzzily linked stays unmatched (identity_id = NULL) for manual review, not discarded
test_embedding_refresh.py
Re-embedding on a model-change event
IntegrationP1 → Embedding refresh
Cases
model change → regenerationa mismatch of chunks.embedding_model with the current model → a bulk run recomputes chunks.embedding
embedding_model updateafter the run, the chunks' embedding_model = the platform's current model
unchanged model — nothingthe model did not change → the run recomputes nothing (the index is already consistent)
outside the schedule chaintriggered by an event, not a timer; ordinary ingestion embeds only changed text by content_hash
dimension change — not this paththis run covers a model change at the same dimension N (row-by-row recompute); changing N itself (halfvec(N)) is a migration (new column → CREATE INDEX CONCURRENTLY → atomic swap), not an in-place refresh, and is out of scope here
test_curation_runs.py
Control plane: run visibility
IntegrationP1 → curation_runs
Cases
one row per runthe start writes running + started_at; the finish — succeeded / failed + finished_at
statistics in stepswhich steps ran and with what values — in a flexible JSONB: edges materialized, duplicates reconciled, entities lowered
failure in errora failure → state = failed + a brief reason in error; steps already run are reflected in steps
CHECK on statea value outside (queued/running/succeeded/failed/cancelled) is rejected by the DB
one active runa second unfinished run is rejected by the singleton partial unique index — at most one queued/running on the platform
cancel → cancelledcanceling a running run (UI "Cancel") terminalizes it into cancelled + finished_at, releases the singleton lock; the next run starts fresh
CHECK on triggera value outside (schedule/model_change/manual) is rejected by the DB; event-driven re-embedding is written as a row with trigger = model_change
test_coordination.py
Mutual exclusion of lanes: ingestion ∥ curation
IntegrationP1 → Lane coordination
Cases
destructive step → sync queuedwhile a merge/retention holds the lock, a run of the affected source goes to queued, does not fail, and starts when the step finishes
additive runs in paralleledge materialization / decay / refresh do not block a Harvester upsert — concurrent writing is survived, edges are not orphaned (ON CONFLICT DO NOTHING)
merge under contentionan upsert of the merged entity during a merge leaves no orphaned edges — they are reconciled by the next run, duplicate_of is consistent
a stale heartbeat releases the lockthe holder died without closing the run → on heartbeat_at going stale the lock is taken away, a new run starts; fencing rejects a late write
refresh without the lane lockduring an active re-embedding, new chunks are written with the current model; mixed embedding_model generations converge, ingestion does not stall
test_hnsw_recall.py
HNSW under-recall: is_deleted lifts off the partial index, the ACL-JOIN — for a manual measurement
IntegrationP1 → chunks
Cases
is_deleted — out of the traversalthe denormalized chunks.is_deleted + a partial HNSW (WHERE NOT is_deleted) keeps the deleted out of the index — this predicate causes no under-recall, it is not a post-filter
the ACL-JOIN under-recallsa hard ACL comes as a JOIN on entities → the top-K of the approximate HNSW is partly cut off already after the scan: fewer allowed candidates than requested, without an error
iterative_scan does not cure ACLhnsw.iterative_scan (pgvector 0.8.0+) only tops up by conditions on chunks itself; a predicate via a JOIN on entities is not compensated by it — hence ACL-recall does not close automatically
metric — cosineon constructed normalized vectors the proximity order goes by cosine (operator class vector_cosine_ops), not by pgvector's L2 default — the metric is fixed as a contract, not inherited by accident from the index
the exact value — by handthe test catches the fact of under-recall under ACL and the boundary of the cure; the absolute recall on real data is left to the manual measurement below
test_backup_restore.py
The backup and restore cycle
IntegrationP1 → Backup & restore
Cases
snapshot is consistentone database → body, vectors, graph, and rights in the backup as a single consistent point
restore brings everything backafter restore the three projections and the ACL match the originals; search and traversal work
vectors survive the cyclechunks.embedding and the HNSW index are restored — proximity computes as before the backup
snapshot journaltaking a snapshot writes a backup_snapshots row: running + started_atsucceeded + finished_at + size_bytes + location; the CHECK on state rejects a foreign value
rotation by retention_countsnapshots beyond backup_settings.retention_count → the oldest is removed both from the journal and from storage
a stuck snapshot does not lock the schedulethe worker died leaving a snapshot in running → on heartbeat_at going stale (~90s) the watchdog reaps it into failed, the single-flight lock releases, the next scheduled one starts
one active snapshotpartial unique WHERE state = 'running': a second backup on top of an unclosed one is rejected, no parallel dump
write-only credentialsdestination_creds_enc is stored encrypted, not returned in the API response or data export (like api_key_enc)
settings — singletonbackup_settings CHECK (id = 1): a second row is rejected; the application reads/updates, does not create
API · P1 Retrieval endpoints — the HTTP contract httpx · ASGI app · Postgres+pgvector · → HTTP API → Conformance
test_api_retrieval.py
The four primitives individually and the assembled hybrid
Cases
primitives as toolsvector / lexical / graph / sql are called individually → a ranked list of their own paradigm
assembled hybridone call → a fused result (fusion inside KS), ready for RAG retrieval
request shape and validationa malformed parameter (traversal depth outside 1–3, an unknown filter) → 422, not 500
RAG does not happen herethe endpoint returns candidates; rerank / packing / the LLM call belong to Query Engine, not exposed outward
top-K limita primitive returns no more than K hits; K is bounded by a server cap — a request over the cap is truncated to it, not unbounded
retrieval response shapea list of hits with a score and chunk/entity ids, ordered by descending relevance; an empty result → 200 with an empty list, not 404
test_api_access.py
Results are always ACL-filtered
Cases
filtered by the callerretrieval is always narrowed by the caller's rights; two users see different things for the same query
the disallowed does not leakan inaccessible entity does not come back in any primitive, nor in the fused result, nor in the graph traversal
anonymouswithout a token → 401 (parametrized across all retrieval routes)
no "bare" routesevery retrieval endpoint is genuinely behind a guard and an ACL filter — a parametrized coverage audit
API · P1 KS-admin — the HTTP contract of runs httpx · ASGI app · → HTTP API → Conformance
test_api_admin_ops.py
Launching maintenance and source status over HTTP
Cases
launch a run → 202reindex / re-embed / backup "now" → 202 + run id; a curation_runs / backup_snapshots row in queued, a task in SAQ
source statusGET → a per-source view: state, entity/chunk counters, the last-run timestamp
restore from a snapshotrestore from backup_snapshots202; an unknown snapshot → 404
lock on destructive opsmerge/retention under the target-lane lock — a concurrent launch → 409, not data corruption
repeated launch under the lock → 409a second POST reindex / re-embed / backup while a run is already queued/running409 CONFLICT (single-flight over curation_runs / backup_snapshots) — not a second run and not 500
validation as 422a malformed run / schedule parameter → 422, not 500
test_api_admin_access.py
Maintenance runs — under a role only
APIP1
Cases
Owner/Admin onlyOwner/Admin launches a run / reads status → 200/202; Member → 403
anonymouswithout a token → 401 (parametrized across all KS admin routes)
no "bare" routesevery admin endpoint is genuinely behind a role guard — a parametrized coverage audit
Manual Recall measurement on real data — outside CI manual measurement · cannot be closed in an automated test

The absolute recall of the approximate HNSW index under a hard ACL filter depends on the distribution of real data and the shape of rights — general benchmark estimates cannot be trusted, and synthetics do not reproduce it. The automated test catches only the fact of under-recall and the boundary of the cure: hnsw.iterative_scan tops up by conditions on chunks itself, but the ACL comes as a JOIN on entities and is not compensated by the iterative scan — so the value itself, that the ACL filter does not drop completeness below acceptable, is taken by hand on a representative corpus. The purpose is to raise recall under the actual load via index parameters (m, ef_search), since iterative_scan does not close the JOIN predicate — not in the automated run.

Structure Test file structure

Split by type: unit/ — pure logic without a database, runs on every PR. integration/ and api/ — against a test Postgres with pgvector in Docker, as a separate, rarer step. Priority (P0–P1) is orthogonal to the folders and set by markers (pytest -m p0). The live recall measurement stays a manual tool and does not enter the automated run.

  • tests/knowledge_store/module directory
    • conftest.pytestcontainers Postgres+pgvector · Entity/chunks/edge factories + the ACL five-tuple · fake embedding model · Curation Pass steps directly
    • unit/pure logic without a database
      • test_chunking · test_fusion · test_staleness_decay · test_traversal_builderslicing, fusion, decay, traversal builder
    • integration/against Postgres+pgvector
      • test_acl_prefilter · test_identity_bridge · test_graph_traversal · test_entity_upsert · test_soft_deleteP0 — load-bearing invariants: ACL, identity bridge, traversal, upsert, soft-delete
      • test_curation_pass · test_embedding_refresh · test_curation_runs · test_coordination · test_hnsw_recall · test_backup_restoreP1 — curation, refresh, journal, lane coordination, recall, backup
    • api/HTTP contract — httpx + ASGI + Postgres+pgvector
      • test_api_retrieval.pyprimitives as tools + the assembled hybrid
      • test_api_access.pyresults are always ACL-filtered
    • manual/outside CI — a live recall measurement on real data