← Cache & Workers

Rate-limit

cache-workers · workzone

One rate-limiting primitive for the whole platform: modules don't count limits themselves — they ask a shared counter. The counter and its consumers are separate concerns; the first is covered below, then the second.

Shared primitive

A shared rate-limiting service for the whole platform. Atomicity — a Lua script in Redis counts and debits in a single pass, with no races between workers. The service offers two strategies for different jobs: token-bucket (burst plus average rate) and sliding window (exact event count over an interval) — the consumer chooses, the counter infrastructure is shared. The threshold is set statically (platform default plus a config override, no hardcoding) and dynamically — a consumer may change the rate at runtime, which adaptive consumers need (see Harvester below). The counter lives in redis-durable: it survives restart and is never evicted under memory pressure. A coarse network rate-limit at Nginx — Auth & Security — stands as an outer barrier ahead of the application one: it cuts a packet storm before it reaches the app. This primitive is the home of the “rate limit” rule in the HTTP API conventions.

1 Capacity — the maximum tokens in the bucket (peak burst).
2 Refill — tokens drip in over time up to capacity (average rate).
3 A request takes a token — one available → let through, token debited.
4 Bucket empty → reject; wait for refill.

Consumers

Consumer What it limits Threshold nature Strategy Window / TTL Failure mode
Harvester request rate to external sources adaptive — set by the external source token-bucket with AIMD — gentle ramp-up, sharp drop on rejection hours, per scope (account / site) fail-open
Auth & Security login brute-forcing fixed — set by security sliding window per IP + per-account delay (email) 15 min (sliding) fail-closed
Auth & Security application API (keys) fixed — set by the key's policy token-bucket, single cap per key (per-role quotas — v2) minute fail-open
Messenger inbound (Slack · Telegram · Mattermost) inbound event volume — the public webhook endpoint; Mattermost has no webhook, its listener stream is capped per channel fixed — set by security sliding window per IP / source / channel minute (sliding) fail-closed
The failure mode is a property of the consumer, not the primitive. When redis-durable is unavailable, throttling lets the request through (the limit is merely a rate optimization), while protection blocks it: for brute-force, fail-open would open a guessing window, so it's fail-closed. The safe default is fail-closed; a relaxation to fail-open is something the consumer declares deliberately.
A rejection to an application call is returned as HTTP 429 with Retry-After — the client learns when to retry; to an internal consumer (Harvester) the same rejection is a signal to back off.
A separate bucket for expensive LLM calls — v2: inference rate is capped not on a par with the ordinary API, but by its own cost-based limit.