Achilles — Cache & Workers: Stack and structure

Backend

stack — libraries

Queue and workers → Worker topology

SAQ · queue SAQ workers

Scheduler → Scheduler

SAQ cron (CronJob)

Rate-limit → Shared primitive

Lua script (atomic)

Storage / state

redis-py · async pools → Two instances

SQLAlchemy 2.0 · asyncpg → Uniqueness

structure — files · infrastructure layer

infra/             shared layer · not a domain with entities
├── *.py           thin client, Redis pools (durable + cache)
├── worker/        SAQ app, task registration by lane
├── scheduler/     singleton cron service
├── rate_limit.py  Lua window / counter primitive
└── lifecycle.py   heartbeat · retry · reaping helpers

A layer, not a domain module

The layer has code, but it's not modeled as a domain with its own entities and migrations: it's a shared service plus a thin client that the other modules import. So the package lives in the shared achilles/infra/ — where it's natural to import it from (from achilles.infra.worker import …).

Container orchestration

One backend image runs in different roles. The API scales separately; workers are split across three lanes with different concurrency, and the scheduler and cache are separate services.

Service	Role	Lane / command	Replicas
`api`	HTTP · response streaming and the notification push channel (SSE)	uvicorn	≥ 1, scales separately
`worker-interactive`	background tasks on the live-request path	SAQ · interactive	≥ 1, moderate concurrency
`worker-background`	heavy deferred runs	SAQ · background	≥ 1, high concurrency
`worker-agents`	autonomous agent runs	SAQ · agents	≥ 1, bounded
`scheduler`	cron service · singleton	SAQ cron	exactly 1
`redis-durable`	queues, task state, locks	Redis	1
`redis-cache`	ephemeral query cache	Redis	1

Splitting queues and cache into two Redis instances — so that memory-based cache eviction doesn't touch the durable queues.

Frontend

Cache & Workers has no frontend of its own. The admin sees the state of queues, workers and Redis on the monitoring dashboard → Admin Panel.