← Knowledge Store

ACL and identities

knowledge-store · workzone

Knowledge Store is the home of the permissions and identities that Harvester captures from sources and writes here, and that → Query Engine applies at retrieval. This page holds the model: how the platform gets from a signed-in user to the entities they are allowed to see — through identities and source permissions — and where the v1/v2 boundary runs. Permissions are stored in the source's original terms, without interpretation into platform-level tiers.

Resolution join: from user to allowed entities

Permissions apply at query time as a pre-filter: candidates are narrowed by access before ranking. To answer “what is available to this user,” the platform walks a chain — from the platform account to its identities, their memberships in source groups, and the grant on an entity. The application itself lives in → Query Engine; here are the links and how they are stored.

users identity.user_id identity
identity source_principal.identity_id source_principal
source_principal group_membership source_group
source_group entity_acl entity

A direct grant on an entity to a specific person short-circuits the chain via entity_acl.source_principal_id, bypassing the group. A scope = 'public' grant is visible to everyone — it is added to the filter directly, bypassing the membership chain.

Harvester
writes
Captures permissions and people from the source, upserting them into these tables in the same sync modes as the content.
Knowledge Store
stores
The relational source of truth for permissions and identities; builds the resolution join and runs the pre-filter next to the vectors — in one SQL with semantic search.
Query Engine
applies
Passes the caller's identity at query time — KS runs the resolution join and filter; Query Engine does not duplicate them.
Identities: source person → identity → account

The same connector API that returns content also returns the source's people. Each source person is a source_principal; those matching by email are merged into one identity. If an identity also signs into the platform, identity bridges to Auth · users.

1 Identity The canonical identity — the merge target.
identity merged identities
UNIQUE (lower(email)) UNIQUE partial (user_id) FK → users
id BigInteger PK —
email Text NOT NULLUNIQUEIDX merge key · normalized lower(email)
display_name Text NULL display name
user_id BigInteger FK→usersNULLUNIQUE bridge to the platform account · 1:1 pinned by UNIQUE partial · NULL = identity with no platform sign-in
created_at DateTime(tz) DEFAULT now()
updated_at DateTime(tz) DEFAULT now() + trigger
A bridge to Auth, not a duplicate
A source “identity” and a “platform account” are different entities. identity.user_id is the only link between them: set when the identity also signs into the platform. The sign-in channels live in Auth · identity_mapping (Slack bot source='slack', SSO Okta / Azure); here, Jira / Slack / Confluence act as content sources. The same Slack plays two roles — sign-in (Auth) and content (here); don't conflate them.
Populating the bridge — auto-link by email → Auth & Security → Harvester
identity.user_id is set automatically by exact lower(email) — a strict 1:1 link: pinned by UNIQUE partial on user_id, and both tables are also unique by email, so there are no collisions (unlike the source_principal → identity merge). It fires synchronously at two points: when a platform account is created (invite accepted, setup, admin creating / changing an email) — Auth asks to link an existing identity of the same email; and when Harvester upserts an identity — linking it to an already existing users row. This way a signed-in user immediately sees all their source accounts, without waiting for a sync. The linking operation is owned by Knowledge Store — the owner of identity; Auth and Harvester call it. The operation is idempotent and transactional: a repeat call with the same email and account is a no-op (guarded by the UNIQUE keys and the upsert on lower(email)), and linking runs in one transaction in the shared database — with no window where the bridge is half-set. The outcome is explicit: either linked, or no match (no identity for this email yet, or sign-in address ≠ source address) — the unlinked goes to manual review, never lost silently.
sign-in email ≠ source email
If a person appears in the source under a different address (alice.smith@corp vs. sign-in alice@corp), auto-link by exact email won't fire — the identity stays without a bridge (user_id = NULL). An administrator links such an identity to the right user manually on Admin · Account linking — the same path as unmerged source_principal rows. Auto-merge across different addresses (deep entity resolution) is v2, a Curation Pass step; see the iteration boundary.
2 Source person The account as the source returns it.
source_principal people from sources
FK → sources FK → identity UNIQUE (source_id, source_user_id)
id BigInteger PK —
source_id BigInteger FK→sourcesIDX source · CASCADE
source_user_id Text NOT NULL native id at the source
email Text NULL for merging · absent from some sources
display_name Text NULL name as in the source
identity_id BigInteger FK→identityNULLIDX NULL = unmatched, awaiting review in Admin
created_at DateTime(tz) DEFAULT now()
updated_at DateTime(tz) DEFAULT now() + trigger
Merge by email · unmatched go to review → Admin · Account linking
Source people with the same exact email are merged into one identity. Those that couldn't be merged (no email, ambiguity) keep identity_id as NULL: the row goes to manual review in Admin, where it is linked to an existing identity or a new one is created. The fuzzy “different emails, same person” case is deep entity resolution: v2 (a Curation Pass step).
Source permissions: groups, membership, entity grant

Permissions are taken in the source's original terms (a Jira project, a Confluence space, a Slack channel) — without interpretation into platform-level tiers. Access rests on two facets: the tag — which group an entity belongs to (entity_acl), and membership — who belongs to the group (group_membership); without both you can't answer who can see the entity.

3 Source group A container in the original terms.
source_group source groups and containers
FK → sources UNIQUE (source_id, source_group_id)
id BigInteger PK —
source_id BigInteger FK→sourcesIDX source · CASCADE
source_group_id Text NOT NULL native container id
name Text NOT NULL display name · “Backend project”
kind Text NULL container type · project · space · channel · set by the manifest
created_at DateTime(tz) DEFAULT now()
updated_at DateTime(tz) DEFAULT now() + trigger
4 Membership Who belongs to the group.
group_membership group membership
FK → source_group FK → source_principal PK (group, principal)
source_group_id BigInteger PKFK→group leading column of the composite PK (the PK index covers group lookups) · CASCADE
source_principal_id BigInteger PKFK→principalIDX second PK column · separate IDX for the principal→groups resolution · CASCADE
created_at DateTime(tz) DEFAULT when the membership was observed
A snapshot of current membership
One row per “group × person” pair. Reconciliation checks membership against the source and brings the snapshot into line: added to a project — the row appears; removed — it is deleted. It is a snapshot of the present, not a change feed. Membership is flat — direct pairs; expanding nested groups (group-in-group, transitive membership) is v2, see the iteration boundary.
5 Entity grant The tag: who can see the entity.
entity_acl entity permissions
FK → entities grant: group · principal · public UNIQUE partial (source_group_id, entity_id) UNIQUE partial (source_principal_id, entity_id) UNIQUE partial (entity_id) WHERE public
id BigInteger PK —
entity_id BigInteger FK→entitiesIDX KS entity · CASCADE
scope Text NOT NULLCHECK grantee · group · principal · public
source_group_id BigInteger FK→groupNULLIDX grant to a group · set when scope=group
source_principal_id BigInteger FK→principalNULLIDX grant to a direct person · set when scope=principal
created_at DateTime(tz) DEFAULT when the grant was observed · no updated_at — a snapshot with no payload
A grant — to a group, a direct person, or everyone (public)
scope distinguishes three forms of grant, and CHECK keeps them consistent: group — source_group_id is set (access for a container/group), principal — source_principal_id (addressed to a person), public — both NULL (content open to everyone in the source: a public Slack channel, anonymous Confluence access). Public is a row-level wildcard grant, without bloating membership with a synthetic “everyone” group; at resolution it is added to the filter as scope = 'public'. The relational table is the source of truth for permissions; how this grant, in a single JOIN, covers both search and graph traversal across the three projections of one database is held by the data model. The entity node (entities) is designed with the rest of Knowledge Store — only the permissions facet is fixed here.
Indexes for the resolution direction · also the dedup key
The hot path is “the user's set of groups → accessible entities,” so the grant carries unique partial indexes (source_group_id, entity_id) WHERE source_group_id IS NOT NULL and (source_principal_id, entity_id) WHERE source_principal_id IS NOT NULL (and for public — (entity_id) WHERE scope = 'public'), not just single-column FKs: the lookup is index-only, without a heap visit for entity_id. The same pair of structures does double duty — the order (grantee, entity_id) matches the resolution direction, and uniqueness provides a conflict target for upsert reconciliation (the grant's natural key given a polymorphic grantee and a surrogate id): a repeat capture is ON CONFLICT DO NOTHING, duplicates excluded. The partiality (WHERE …) is because each grant form (scope) sets its own set of columns. When the set of accessible groups is already known (resolved at query time by → Query Engine), the resolution join from users collapses into a filter over that set. Denormalizing permissions into an array on the entity is a separate heavy lever, deferred (open-questions).
v1 / v2 boundary

v1 is raw capture of permissions and identities, deterministic merge by exact email with manual review of the unmerged, pre-filter application, and graph curation by a background run (Curation Pass). v2 leaves the platform interpretation of permissions on top of the source level, fuzzy (machine-guess) identity merging, and hard guarantees at retrieval.

v1 — raw capture + pre-filter

The five tables above · identity merge by exact email · the identity ↔ users bridge via auto-link by exact email (synchronous, both sides) + manual linking of the unmerged in Admin (no email, ambiguity, sign-in address ≠ source address) · grant to group / principal / public · membership and tag in the source's original terms · pre-filter at retrieval (resolution join). Permissions are eventually consistent: a revocation takes effect by the next sync.

v2 — interpretation and inference v2

A platform ACL on top of source permissions (double filter, container-override, platform_acl_rules, the → Admin · Platform ACL screen) · a live permission check at retrieval (late-binding security trim) · fuzzy cross-source identity merging and auto-linking of the bridge by different emails (deep entity resolution — a Curation Pass step) · expansion of nested groups (group-in-group): transitive membership — in v1 membership is flat, one row per “group × person.”