← Harvester

ACL and identities

harvester · workzone

Alongside the content, Harvester pulls two accompanying datasets from the source — access rights and people's identities — and keeps them up to date on a par with the entities themselves. Harvester produces them; the table model, storage, and resolution are held by Knowledge Store. Rights are saved in the source's original terms, without interpretation; people are merged by email; permission sync is part of the general sync modes, with no dedicated mechanism of its own. Collecting both goes through the connector — the fetch mechanics are held by sources.

1 Access rights The source's ACL travels along with the content.
ACL in original terms
Harvester saves the source's rights exactly as the source itself expresses them: a project in Jira, a space in Confluence, a channel in Slack. There's no mapping onto platform access levels here — rights are pulled and stored alongside the entity one to one. The capture is two-sided: both the group tag on the entity (which project, space, or channel it lives in) and group membership — who belongs to it; without both facets you can't answer who has access to the entity. This is a deliberate boundary: Harvester records who had access to what in the source, and doesn't try to decide what that means on the platform.
Interpretation — not here
Translating the original ACL into platform access levels is the job of other layers; Harvester doesn't decide who should see what on the platform. The contract is simple: rights travel with the content and are stored undistorted.
Applied at retrieval — pre-filter
Rights are applied at query time as a pre-filter: candidates are narrowed by access before ranking by proximity — not “rank, then discard what's forbidden” (that skews the counters and empties pages). Access is resolved by a join of user → their identities → memberships in source groups → the entity's group ACL; a chunk inherits the entity's ACL and has none of its own. ACL and membership sit in the relational database next to the vectors — so filtering by rights runs in the same query as semantic search. The application itself is behind the retrieval layer.
→ Query Engine
2 Identities The same connector returns people too; merged by email.
One API — both content and people
Users are returned by the same connector API as the content, so no separate channel for people is needed. When a source is connected, a full import of its users runs; from then on incremental picks up new ones as they appear — by the same mechanisms as entities.
Merge by email · resolving the unmatched
People from different sources are merged into one identity by email — an explicit, reliable key. Anyone who couldn't be matched by email Harvester doesn't guess at: such records go for manual review in Admin Panel, where they're linked to an existing identity or created as a new one. On identity upsert, Harvester also sets a bridge to the platform account with the same email, if one already exists (identity.user_id) — the same auto-link by exact email; the rule is held by Knowledge Store.
→ Admin Panel · Account linking
Fuzzy merge — not here
A merge of the “different emails, same person” kind — fuzzy matching of identities across sources — Harvester doesn't do. That's the job of deep entity resolution in Knowledge Store, beyond our boundary; here we merge only on an exact email match.
→ Knowledge Store
3 Permission sync Part of the general sync modes, not a separate mechanism.
Rights — on the general sync modes
Rights change — someone added to a project, someone removed from a channel — and those changes need tracking. Harvester introduces no dedicated mechanism for this: ACL sync runs in the same modes as content sync.
Incremental · reconciliation
Incremental updates the ACL pointwise — by webhook or polling, just like entities. Reconciliation does a full rights reconciliation: it catches what incremental missed (dropped events, access revocations) and brings the stored ACL back in line with the source.
Revocation takes effect with a delay
The stored ACL is eventually consistent: there's a window between a change in the source and its sync. For granting access this is safe — until the new right is synced, a person sees less than they're already allowed, not more. For revocation the window means the reverse: what was revoked stays visible until the next incremental, partial re-sync, or reconciliation — this is an accepted trade-off, not an oversight. The hard guarantee that “revoked won't leak into an answer” comes not from Harvester but from the final rights check at retrieval itself — which sits behind the query layer.
→ Query Engine