Achilles — Cache & Workers: scheduler

Who runs tasks on a schedule. Time-based triggers are born here; event-driven triggers (a module calls the queue directly) bypass the scheduler. Where a task lands and who works it off — across three lanes; what stops a double run — uniqueness.

Scheduler

Built-in SAQ cron. SAQ handles CronJob out of the box — no second scheduler needed. It lives on a dedicated singleton service (one replica), separate from the horizontally scalable worker pool: were cron on every worker, with N replicas it would fire N times. The scheduler only publishes a task to a lane — the workers execute.

Singleton · 1 replica

Scheduler

SAQ cron · ticks on schedule

publishes, doesn't execute

Pool · N replicas (horizontal)

Worker

work the lanes off in parallel

The scheduler is fed by two trigger sources — both publish to a lane, and from there the task's path is shared:

fixed cron

A CronJob by the calendar — one tick publishes one task: sync, reconcile, health probe, curation, backup.

next_run_at scan

A periodic pass over agent schedules — one scan publishes every run whose next_run_at has already arrived. The owner sets the time, the scheduler only wakes it.

Perfect reliability isn't required — in either direction. A double tick (restart, race) is safe: the uniqueness lock in Postgres absorbs it — the first run starts, the second catches a conflict and goes to skipped. A missed tick (the singleton was down during the scheduled window) is acceptable too — we don't backfill, we wait for the next window: the cadences are coarse, and one miss is made up by the next run. The scheduler is responsible for “when”; the correctness of a single instance is on uniqueness.

Windows are in the organization's timezone, the tick is in UTC. Schedules are stored naive, without a zone (HH:MM, minute of the week). At launch the scheduler unfolds the window to UTC via platform_settings.timezone (IANA); agent schedules — via the owner's timezone (NULL → the organization's zone). A single platform contract: set and display in local time, store and tick in UTC — as in Harvester and Agent Engine.

v1 schedules are simple: “every N hours”, “weekly at HH:MM”. A rich / dynamic cron (per-source windows, editing from the UI) — v2.

Schedule registry

Schedule	Cadence	Lane	Consumer
Incremental sync	6 h (webhooks) / 15 min (polling-only)	background	→ Harvester
Reconcile	weekly	background	→ Harvester
Source health probe	once a day	background	→ Harvester
Curation pass	one platform-wide schedule	background	→ Knowledge Store
Backup	its own schedule	background	→ Knowledge Store
Agent scan	the scheduler scans `next_run_at`	agents	→ Agent Engine

KS re-embedding is queued by an event — a model change, not cron alone: the change triggers a run immediately, the schedule here doesn't hold it.