Skip to content

Router.Africa — Future Optimizations Register

Principle: start simple, but build the hook now so scaling later is a small change, not a rewrite. Each row names an optimization we are deliberately not doing at launch, the hook that must exist in v1 to keep it cheap, what tells us it’s time, and what skipping the hook would cost.

The schema (#6) and backend scope (#4) work must satisfy every “Hook to build now” below. Decisions themselves live in decisions-log.md; this file only tracks what is deferred and why it stays cheap.

Figures are estimates from 2026-09-20 (assumptions: ~500 B per request-log row, ~4 KB per stored request, launch volume about 2–20 req/s). Re-check against real traffic.


Optimization Hook to build now Trigger to act Cost if the hook is missing
Partition the request-log tables Every log row has a non-null created_at; every query is time-bounded; primary and unique keys include created_at (e.g. (created_at, id)) Largest log table passes ~100M rows or ~50 GB Postgres requires the partition key in any unique constraint, so an id-only primary key means rewriting the table to partition later
Move analytics to its own Postgres instance Separate schema from the ledger; no foreign keys from log tables to ledger tables (ids are plain columns); store interface in the queue consumer; separate Hyperdrive config or connection limit for analytics writes Analytics load shows up as ledger contention or Hyperdrive pool pressure Foreign keys and shared connection config would have to be untangled while the ledger is live
Move aggregates to ClickHouse Log tables are append-only; dashboard aggregates read from rollup tables (e.g. usage by model by hour), not raw rows; writes go through the store interface Aggregate query latency degrades despite rollups and partitioning Update-in-place logs and dashboard queries hard-wired to raw tables would need rewriting at the same time as the store
Optimization Hook to build now Trigger to act Cost if the hook is missing
Batch many requests per R2 object (cuts the $4.50 per million write cost) Pointer on the request-log row is object key plus optional offset and length columns (empty at launch); content writes go through a single write interface R2 write costs pass ~$100 a month (about 10 req/s sustained) A schema migration on the largest table, plus changing every reader
Customer deletion of stored content (deferred) Key layout {account_id}/{yyyy}/{mm}/{dd}/{request_id}.json.gz, so it’s a prefix operation; content status on the row so deleted content can be marked A customer asks, or a legal or enterprise requirement appears Scanning every object, or maintaining a separate index
Purge existing content when an account turns saving off (deferred; turning it off currently stops future capture only) Same account-prefix key layout; opt-out changes audit-logged with timestamp so we know what predates the opt-out A privacy requirement or customer expectation appears Can’t tell which stored content predates the opt-out, or delete it cheaply
Retention window (replace “indefinite”) Date in the key path, created_at on the row R2 storage cost, a privacy requirement, or a customer ask Can’t expire by prefix or lifecycle rule
Infrequent Access storage tier Date-prefixed keys so a lifecycle rule can target old objects Storage cost exceeds write cost and old content is rarely read Low; not adopted at launch because writes cost double and there’s a 30-day minimum
Handle very large or partial responses A content status on the request-log row (e.g. stored, partial, truncated, none); a maximum stored size per request Long-context traffic grows Can’t tell whether missing content was a bug, an opt-out, or a truncation
Use Bifrost for content logging instead Content capture sits behind the write interface; Bifrost’s disable_content_logging stays on Bifrost adds per-key content logging and deletion Turning it on early would store every customer’s content regardless of opt-out
Track storage per account Bytes and object counts recorded per account in the rollup tables Cost review, or a customer disputes usage No way to see which accounts drive cost without scanning R2
Optimization Hook to build now Trigger to act Cost if the hook is missing
CLI and MCP support (delegated OAuth tokens: device flow for a CLI, OAuth 2.1 with PKCE for a remote MCP server; see auth-security.md section 8) Keep the credential planes separate as designed (R1 to R5); sessions.audience and other enums stay text with a CHECK so a new audience is a one-line migration; authorization is read live from the database, never from a token or cookie; do not enable the bearer or JWT plugins globally; reserve the mcp.router.africa hostname; decide now which actions are never delegable (money movement, member changes, stored-content reads) A customer or partner asks for a CLI or an MCP server Enabling the bearer plugin globally to get a CLI working would make the session cookie usable as a Bearer on every dashboard route, undoing the one-credential-per-surface rule
Optimization Hook to build now Trigger to act Cost if the hook is missing
Split the queue consumer into its own Worker (already noted in the decisions log) Keep the queue() handler in its own module in apps/dashboard-api, with no imports from dashboard route code Analytics processing needs to scale or deploy independently Untangling shared code from the dashboard’s routes
Move production to PlanetScale (from the cluster Postgres) The app reaches Postgres only through Hyperdrive; standard Postgres features only; same major version The load test (P9.10) shows the droplet can’t hold the launch load, or availability needs failover Extensions or settings PlanetScale doesn’t offer; connection strings outside Hyperdrive
Ledger tier upgrade Load test under sustained write concurrency before launch; Hyperdrive pool size is the watched capacity number Load-test results, or hold/settle latency rises Guessing the tier
Datadog cost control Sampling and log-volume settings configurable, not hard-coded Monthly Datadog bill grows with traffic Cutting signals in a hurry during an incident
Assign orgs to gateway instances (e.g. a dedicated Bifrost for a large customer, or spreading load across bifrost-1 and bifrost-2) gateways table separates instance from adapter; credentials keyed (key_id, gateway_id); gateway_id on request logs, reconciliation rows and rollups; Postgres can rebuild any gateway’s keys. To add: organizations.gateway_id with a default instance for new orgs, and a move procedure (create credentials on the new instance, flip gateway_id in one statement, in-flight requests finish on the old one, delete old credentials) A customer’s load or isolation needs warrant its own instance, or one instance nears capacity Log and rollup tables with no gateway column; gateway-specific ids baked into api_keys
Gateway failover (two instances, traffic moves when one is down) Same hooks; every call goes through the Gateway interface, so failover is one place Gateway downtime becomes a real availability problem (single droplet today) Each key would need credentials on both instances and the request path a health-aware choice; not possible if credentials were one-per-key columns
Log the client IP on request logs Nullable logs.request_logs.client_ip created at launch and never written A security or abuse need outweighs the privacy cost (an IP is personal data; country and ASN are already logged) A schema change on the largest table
Per-instance model lists (gateway_models) models.gateway_model_id is separate from our public model id Instances need to serve different models Customer-facing model ids tied to one gateway’s naming
Per-model routing across gateway types (some models on Bifrost, others on another gateway, for the same org) Same hooks plus the per-instance model list A model is only available through a different gateway type Routing logic scattered outside the Gateway interface
Bifrost replicas within one instance Instances are separate Helm releases, so replicas are a values change One instance needs more throughput Verify first whether Bifrost’s rate-limit and budget counters are shared across replicas; if they are per replica, limits become inaccurate

  • created_at non-null on every log table, and part of every primary or unique key
  • No foreign keys from log tables to ledger tables
  • Log tables append-only (no update paths)
  • Rollup tables defined for the dashboard’s aggregates
  • Request-log row has object key plus optional offset and length for R2 content, and a content status
  • Per-account storage bytes and object counts tracked
  • Analytics writes use a separate Hyperdrive config or connection limit
  • Queue consumer writes through a store interface; content writes through a write interface

Add a row here whenever a decision says “later” or “revisit”. If a deferred optimization has no hook listed, that is the gap to close before the schema is final.