Router.Africa — Architecture Decisions Log
A running record of what’s decided versus still open. Update this as decisions get made — an item sitting in “open” indefinitely usually means it’s been silently deprioritized rather than resolved.
Decided
Section titled “Decided”| Decision | Detail |
|---|---|
| Model gateway | Bifrost, unchanged, deployed on the existing K3s droplet. Not rebuilt or replaced. |
| Ingest/auth/orchestration layer | Single Node/TypeScript service, deployed as Cloudflare Workers. Chosen over Go and Python — at target throughput (100–500 req/s) raw performance isn’t the deciding factor; team fluency and type-sharing with the frontend were. |
| No Go in the main app | Considered and rejected splitting request handling across Node (auth/balance) and Go (business logic/analytics/Bifrost call). Rejected because it introduces a distributed-transaction problem on the hold/settle path with no offsetting performance benefit at this scale — the workload is I/O-bound, not CPU-bound. |
| Bifrost call stays synchronous | Standard chat completion requests are never queued. Clients expect real-time response, including streaming (SSE) — a queue is the wrong shape for that. Flow: Worker → hold → call Bifrost directly → settle → return. |
| What is queued | Only post-response work: usage events, analytics, audit log. Fire-and-forget to Cloudflare Queues, consumed via a queue() handler inside apps/dashboard-api (not a separate Worker — see Queue consumer placement below). The client never waits on this. |
| Batch/async endpoint | Explicitly out of scope for now. Standard request path only. Can be revisited later as a separate product surface (similar to OpenAI’s Batch API), not retrofitted onto the standard path. |
| Credit hold/settle mechanism | Two-phase: hold before calling Bifrost, settle after. Router.Africa does not count input tokens. Hold = min(available balance, effective_max_tokens × output_rate + input_allowance). effective_max_tokens is the client’s max_tokens capped at a per-model ceiling, or a per-model default when the client omits it (never the model’s full output limit, which would over-reserve massively on large models). The per-model ceiling, default and input_allowance are configured in admin. A request is rejected only if available balance is below a configurable minimum hold. Settle: charge actual usage as reported by Bifrost at the locked rate and release the unused hold. If actual exceeds the hold, charge the full actual amount and floor the balance at zero, absorbing the shortfall. This is an accepted risk: occasional slight overage on the last request, in exchange for a simple hold; the next request is blocked. Monitor absorbed shortfalls and tune the input allowance. |
| Per-organization hold toggle | Hold is switchable per organization in the admin app (revised 2026-09-20; earlier per account, then per workspace). It applies to every workspace in the org, and the request path reads it through the workspace’s org, so we can test which accounts actually need one. On (default): the full hold/settle flow above. Off: no hold is placed. The request is allowed if available balance is above the minimum hold threshold, and settle charges actual usage (Bifrost-reported tokens × the published rate in effect when the request started) with the balance floored at zero. Unheld concurrent requests can each pass the balance check, so overage is bounded only by concurrency and request size; the shortfall is absorbed, same as the accepted-risk rule above. The idempotency insert still runs. Toggle changes are audit-logged, and absorbed shortfalls are tracked per account so we can see whether holds-off accounts cost us anything. |
| Hold/settle concurrency safety | Atomic conditional UPDATE in Postgres (WHERE balance - held >= :min_hold, setting the hold to LEAST(balance - held, :computed_hold)), not a Durable Object. A DO was originally proposed to solve the race condition, but once hold/settle are themselves durable DB writes, the DO adds a network hop without adding a concurrency guarantee Postgres doesn’t already provide row-locally. |
| Ledger database | PlanetScale Postgres, accessed via Cloudflare Hyperdrive for connection pooling. Uses flat cluster-tier pricing, not per-operation billing (that model applies to PlanetScale’s separate Vitess/MySQL product, not Postgres). HA tier (~$148/month+) is the realistic production floor; exact tier depends on load-test results under sustained write concurrency. |
| Analytics/dashboard read store | Start simple: the existing PlanetScale ledger cluster, in a separate schema, not a new instance and not ClickHouse (revised 2026-09-20). Written by the queue consumer in apps/dashboard-api; never the source for billing or rate limiting. Sized for a launch assumption of about 20 req/s or less, likely much lower, not the 100–500 req/s ceiling. Request-log metadata is retained indefinitely. No partitioning at launch, but the schema is built so partitioning, moving to its own instance, or moving to ClickHouse are cheap later. Rules that keep that true: (1) every log row has a non-null created_at, and every query is time-bounded; (2) primary and unique keys on log tables include created_at (e.g. (created_at, id)), because Postgres requires the partition key in any unique constraint, and an id-only primary key would force a table rewrite to partition later; (3) no foreign keys from log tables to ledger tables, ids are plain columns, which also lets the schema move to another instance; (4) log tables are append-only; (5) dashboard aggregates read from rollup tables (e.g. usage by model by hour), not by scanning raw rows; (6) the queue consumer writes through a small store interface; (7) analytics writes use their own Hyperdrive config or connection limit so they can’t starve hold/settle, and dashboard reads use a replica or the rollup tables. The request-logs view reads rows directly, with a pointer to R2 for stored content. Triggers to revisit (indicative, tune later): the largest log table passes roughly 100M rows or 50 GB, dashboard query latency degrades, or analytics load shows up as ledger contention. First step then is partitioning; next is a separate instance; ClickHouse if the aggregate queries outgrow Postgres. |
| API key security model | Keys stored as hashes (never raw), per-key scoping/spend caps to bound blast radius on a leak, explicit rotation/revocation, HTTPS-only, and explicit redaction from request logs given the logging/analytics pipeline already in the design. |
| Dashboard auth model | Better Auth (open-source, MIT-licensed, self-hosted TypeScript auth library), chosen over fully custom and over WorkOS (managed). Social login is a stated requirement (fast signup for developers and target customers), which ruled out fully-custom-from-scratch as the default (est. 1.5–2.5 weeks to build OAuth flows, session rotation, and security hardening per provider). Better Auth provides social login (40+ providers), password auth, and session/refresh-token management out of the box as a library, not a hosted service — estimated integration effort ~2–4 days. Chosen over WorkOS specifically because it keeps user data in Router.Africa’s own PlanetScale Postgres from day one, eliminating vendor migration risk (forced password resets, OAuth app re-registration, no confirmed path to export password hashes if ever moving off a managed provider). Cloudflare/stack compatibility confirmed: runs on Node natively; on Workers, requires Hono as the routing layer (Better Auth’s official Workers support path) — deployed in the dedicated apps/dashboard-api Worker, not the external-facing apps/api Worker (see Worker split decision below). A community integration (better-auth-cloudflare) wires up Workers + Hyperdrive + D1/KV/R2, directly compatible with the existing PlanetScale-via-Hyperdrive setup. Trade-offs accepted: no managed infrastructure — Router.Africa owns uptime, security patches, and library upgrades; API stability across minor versions is still evolving; no SOC 2 certification to hand an enterprise customer’s security questionnaire if that becomes a procurement requirement later. Hard constraint carried from the Niobi web rewrite’s cookie/token bug: whichever credential mechanism is issued must be the only valid auth path end to end in the app’s own request handling — no code path where a stale or leftover credential from a prior mechanism can silently win and authenticate as the wrong user. |
apps/api Worker routing framework |
Hono, routing the entire apps/api Worker — ingest, hold/settle, and the Bifrost call. Fills a gap that was never previously decided, not a replacement of an earlier choice — prior discussion of the Worker described it generically without naming a routing framework. |
| Domain structure | Four separate origins, split by consumer and security profile: router.africa (marketing site, Astro), app.router.africa (dashboard SPA + its own backend, including Better Auth — CORS scoped tightly to this one origin), api.router.africa (external, OpenAI-compatible developer API — API-key auth, no CORS since it’s server-to-server only), admin.router.africa (staff-only admin console for rates, markup rules, FX, keys, rate limits, ledger adjustments and audit — its own session cookie, CORS scoped to this one origin, staff flag separate from org roles; added 2026-09-20, see the Build Inventory). Chosen over path-based namespacing (e.g. router.africa/api) specifically to keep the external API’s security boundary clean — a shared /api/* namespace risks a permissive dashboard CORS policy accidentally applying to the money-moving external endpoint. OpenRouter itself uses a path-based pattern (openrouter.ai/api/v1), but this doesn’t constrain compatibility since OpenAI-compatible clients always set base_url explicitly. |
| Worker split: external API vs. dashboard backend | Two separate Workers, not one. apps/api handles only the external developer-facing API (ingest, hold/settle, Bifrost call) at api.router.africa. A new apps/dashboard-api Worker handles Better Auth and the dashboard’s own backend needs (usage stats, billing, key management) at app.router.africa. Chosen over keeping both in one Worker (which was the prior default) for isolation: the external hot path and the auth/session logic no longer share deploys or blast radius, and each Worker’s middleware (CORS, auth type) is scoped to what it actually needs rather than requiring route-group-level care to avoid cross-contamination. |
| Queue consumer placement | Folded into apps/dashboard-api as a queue() handler, not a standalone apps/queue-consumer Worker. Cloudflare Workers support both an HTTP fetch() handler and a queue() handler in the same script, so a separate deployable wasn’t required to keep the async-decoupling benefit. apps/dashboard-api is the natural owner since it’s already the Worker reading usage/billing data back out for the dashboard — the same Worker now writes it. Trade-off accepted: this couples the queue consumer’s deploy to the dashboard-api Worker’s deploy; if analytics-processing volume ever needs to scale or deploy independently of the dashboard’s own backend traffic, splitting it back into its own Worker is the natural next step — not a starting assumption at current scale (100–500 req/s). |
| Scheduled jobs | Scheduled work runs as Cloudflare Cron Triggers on apps/dashboard-api, alongside its queue() handler, not in a separate Worker: the daily rate computation (Bifrost cost, FX, markup, round up, publish, only for models that are on), the model catalog sync from Bifrost, key reconciliation between Postgres and Bifrost (feeds the admin discrepancy view), and low-balance alerts. Same trade-off as the queue consumer: deploys are coupled, and splitting out is the natural next step if volume demands it. (Decided 2026-09-20.) Refined 2026-09-20: the jobs are not each their own Cron Trigger. See Scheduled jobs admin page. |
| Data residency / KYC / AML | Out of scope for Router.Africa’s own compliance work — handled via Niobi’s existing API signature/auth mechanism. |
| Repo structure | Single Turborepo monorepo: apps/api (external API Worker, api.router.africa), apps/dashboard-api (Better Auth + dashboard and admin backend + queue consumer, app.router.africa and admin.router.africa), apps/dashboard (Vite frontend, served from app.router.africa alongside its backend), apps/admin (separate Vite frontend for admin.router.africa, its own build and deploy so admin code never ships in the customer bundle; talks to the shared apps/dashboard-api backend), shared packages/types, packages/db, packages/config. Bifrost’s config (not source) may live under infra/; Bifrost itself is not part of this repo. |
| Testing approach | Vitest for units (matches existing Vite tooling). @cloudflare/vitest-pool-workers for integration tests — required specifically because Workers run in a V8 isolate with different constraints than plain Node, so plain-Node integration tests would give false confidence. PlanetScale branching for DB integration tests rather than mocking Postgres, since the hold/settle logic’s correctness depends on real row-locking behavior. k6 for load testing. A dedicated concurrency-correctness test (N concurrent requests against one account, assert no over-approval) is treated as the single highest-priority test in the suite. |
| Threading/concurrency model | No manual thread/pool sizing needed anywhere in this stack — Workers and Node use an event-loop model (async I/O, not thread-per-request), Cloudflare auto-scales isolates, Bifrost uses goroutines internally. Hyperdrive’s connection pool size is the one real capacity number to watch. |
| Billing/pricing model | See the dedicated Pricing & Markup Model document (backlog item #11): daily published per-model, per-token-type rate in local currency, from Bifrost’s highest-cost-across-providers estimate, mid-market FX (Frankfurter.app), a scoped flat/percentage markup rules engine, per-workspace percentage discounts applied at charge time, and round-up. Rate locked at hold time, used for settle. No direct provider integration beyond Bifrost. The actual launch markup values are set later in the admin app (see Markup configuration), not a build blocker. |
| Idempotency key design | Two-part design, since OpenAI’s own chat/completions spec has no idempotency concept to build on (client SDKs won’t send one by default). Part 1 — automatic, no client action required: on every request, before the hold write, compute hash(account_id + exact request body) and attempt INSERT ... ON CONFLICT DO NOTHING into an idempotency_keys table (hash, account_id, status, response_body, created_at) within the same transaction as the hold, not a separate write — this keeps the cost at effectively zero given PlanetScale’s flat cluster pricing, versus the ~$1,300–6,500/month a Cloudflare KV-based version would have cost at stated volume. Dedup window is short (seconds, not 24h) — long enough to catch a genuine SDK-level retry after a network blip, short enough not to falsely block a legitimate identical prompt sent minutes later. If the insert conflicts: an in_progress original → return 409 Conflict; a completed original → return its stored response, skip hold/settle entirely. Part 2 — optional, client-supplied: an Idempotency-Key header, documented explicitly as a Router.Africa extension (not part of OpenAI’s spec) in the developer docs (#10), settable via extra_headers in any OpenAI SDK, for developers who want an explicit, longer-lived guarantee beyond the automatic short-window protection. Not a substitute: Bifrost’s semantic/exact-hash caching is a different mechanism (reduces Bifrost’s own compute cost on similar/identical prompts) and does not protect Router.Africa’s ledger — without the dedup above, a client retry hitting Bifrost’s cache would still produce a second hold/settle against the customer even though Bifrost only did the work once. |
| Mid-stream client disconnect policy | Checked Bifrost’s own behavior before designing this (maximhq/bifrost release notes) rather than building from scratch. Bifrost already has proactive SSE disconnect detection (v1.6.8 — detects disconnects actively during streaming, not only on a failed write) and is actively engineering correct partial billing on cancelled streams (“Billing on Failed Streams” fix, confirmed for Anthropic and Bedrock specifically — not confirmed complete across all 15+ providers Bifrost fronts). Design: trust Bifrost’s reported partial usage (usage.total_tokens, usage.cost) as the primary source for settling a disconnected request — not Router.Africa independently counting streamed tokens as the ground truth. Router.Africa’s own token count is kept as a reconciliation/sanity check, not the primary mechanism, specifically because provider coverage for Bifrost’s partial-billing fix isn’t confirmed complete — a divergence between Bifrost’s reported number and Router.Africa’s own count on a given provider is a signal to investigate, not something to silently trust either source blindly for. |
| Worker → Bifrost timeout/retry policy | Checked Bifrost’s own behavior before designing this. Bifrost already implements bounded per-provider timeout (not accumulating per-retry the way naive proxies do — e.g. ~2s max per provider attempt, so a two-provider fallback chain completes in ~4s, not 30-60s) and its own configurable retry/fallback across providers (max_retries, retry conditions, explicit fallback chain) — confirmed explicitly that a provider failure only becomes a request failure if no fallback is configured. Design, revised from an earlier version that duplicated this at the Worker level: the actual decision is how to configure Bifrost’s own timeout/retry/fallback settings (per-provider timeout, retry conditions, fallback chain), not a separate Worker-level retry on top of Bifrost’s internal one — a Worker-level retry stacked on Bifrost’s own retry/fallback risks compounding rather than helping. The Worker’s role simplifies to: set an overall request timeout comfortably longer than Bifrost’s worst-case internal fallback time, and handle whatever Bifrost ultimately returns — success, or a final failure after Bifrost has already exhausted its own configured resilience (release the hold on final failure, no further Worker-initiated retry). |
| Rate limiting / abuse prevention | Enforced in Bifrost, configured from Postgres. Every Router.Africa API key maps 1:1 to a Bifrost virtual key (see API key ↔ Bifrost virtual key mapping), so Bifrost’s own per-VK request/token rate limits do the enforcement. Bifrost doesn’t know about Router.Africa users, so Postgres holds the limit configuration per organization (decided 2026-09-20, replacing per-key; set in the admin app) and apps/dashboard-api pushes it to Bifrost’s governance API. Bifrost enforces rate limits at the virtual-key level only (no aggregate across keys), so the org’s limit is copied onto each of the org’s keys’ virtual keys: new keys inherit it at creation, and changing an org’s limit fans out to all its keys through the same sync path (drift shows in the admin discrepancy view). Accepted: the limit is effectively per key, so an org with many keys can exceed it in aggregate; revisit if that is abused. Limits remain admin-only, invisible to the customer (see Dashboard Scope). Supersedes the earlier design — a Postgres counter table (account_id, window_start, count) checked in the same transaction as the hold and idempotency claim — which is dropped: the hold transaction now covers only the idempotency claim and the hold. Consequence: a Bifrost 429 arrives after the hold is written, so the Worker must release the hold on any Bifrost rejection (same path as a final provider failure). Rate limiting still must not be read from the async request/analytics log — its write lag means a burst isn’t visible until it has already gone through. |
| Request logging | Every API request logs metadata — timestamp, account, endpoint, model, status, latency, tokens, cost — feeding the dashboard’s request-logs view and the audit trail. This is the existing async pipeline (fire-and-forget to Cloudflare Queues, written by apps/dashboard-api’s queue consumer) — already decided under Async/Post-Response Work, restated here explicitly as a schema requirement (#6) rather than left implicit. Explicitly not the source for rate limiting — see the entry above for why the async write lag makes it unsuitable for a real-time check. |
| Observability tooling | Datadog, chosen over Sentry — a stronger fit than initially assessed, not just preference: Bifrost has a native Datadog plugin (APM traces via dd-trace-go, already built, not a bolted-on SDK), Cloudflare Workers has native OpenTelemetry export (automatic instrumentation, exports to any OTel-compatible backend — Datadog included), and apps/niobi-proxy (TypeScript on Node, per the Niobi payments decision) can use Datadog’s Node APM library (dd-trace) or OTel export. This means Datadog serves as one genuinely unified platform across the whole stack — Bifrost via its native plugin, Workers via OTel export, niobi-proxy via the Node APM library — which removes the earlier “defer a unified view until later” caveat entirely; Datadog gets you that unification natively, not as a future project. Cost tradeoff worth naming: Datadog is typically pricier than Sentry at scale — acceptable given the architecture fit and stated preference, but worth watching as usage grows. |
| CI/CD pipeline | Two coordinated pipelines, not one — Cloudflare Workers Builds only covers what’s Cloudflare-hosted, so K3s-hosted pieces need a separate mechanism. Cloudflare-hosted (apps/api, apps/dashboard-api): two environment Workers per app on the same repo (e.g. -staging / -production), each with its own [env.staging]/[env.production] Wrangler config block (separate bindings, secrets), triggered by pushes to a staging branch and main respectively, each running lint + test in the build command before wrangler deploy --env <env> — both environments live and concurrently addressable, not a promote-after-staging model. apps/dashboard (Pages) gets a stable named staging branch/environment alongside its existing per-PR previews, for parity. K3s-hosted (apps/niobi-proxy, Bifrost config): a separate GitHub Actions workflow — push to staging/main runs TypeScript tests + lint (apps/niobi-proxy), builds the Docker image, pushes to a registry, and deploys via Helm to a staging/production K3s namespace respectively, mirroring the Workers-side pattern as two concurrent, separately-addressable deployments on the same droplet. Bifrost’s own config changes (infra/bifrost) go through this same GitHub Actions + Helm mechanism, staging validated before promoting to the production release. |
| Niobi payments static IP handling | Niobi’s payments integration requires server-to-server calls from a static IP for allowlisting. Cloudflare Workers have no static outbound IP by default (egress can originate from any Cloudflare edge IP). Resolved with a small, separate relay service (apps/niobi-proxy, TypeScript) on the existing K3s droplet — apps/dashboard-api calls it over a private connection (e.g. Cloudflare Tunnel), and the relay makes the actual outbound call to Niobi using the droplet’s stable IP. Deliberately minimal scope: a relay only — verify the caller, forward to Niobi, return the response; no business logic (that stays in apps/dashboard-api). Language: TypeScript, not Go — initially considered for consistency with Bifrost on the same droplet, but reconsidered since the droplet doesn’t require every service on it to share a language; TypeScript instead enables sharing request/response types with apps/dashboard-api via packages/types. Security requirements: caller verification (shared secret or mTLS over the tunnel, not an open endpoint) and Niobi’s own credentials held here, separate from apps/dashboard-api’s secrets. Staging support: must be able to mock Niobi calls (env-flag-driven mock mode within the same service) so staging doesn’t need live payment credentials or risk real transactions. Signing implementation: Niobi’s official reference implementation/SDK for request signing exists and will be ported into apps/niobi-proxy (exact integration details to follow) rather than implemented from a spec from scratch. Containment requirement: all Niobi signing logic — the algorithm, the credentials it uses, everything related to authenticating to and from Niobi — lives exclusively inside apps/niobi-proxy; apps/dashboard-api and every other service in the stack never sees or handles it directly, only calls the relay. Inbound verification placement: verifying signatures on inbound Niobi-authenticated traffic also happens inside apps/niobi-proxy, not as a gatekeeping step in front of the Worker or inside another service’s auth flow — consistent with keeping all Niobi-specific auth logic (both directions) in one place. Chosen over Cloudflare’s native fix (Workers VPC binding + Gateway egress with a dedicated IP) because that requires Enterprise-tier Cloudflare, premature spend for one integration point; also chosen over a third-party static-egress-IP proxy service (e.g. QuotaGuard) to avoid a fourth vendor relationship for something the existing droplet already solves. |
| Model availability enforcement | Raised by the Dashboard Scope doc’s onboarding flow (customers choose which models are enabled for their account/key at signup, editable later). This is a backend enforcement requirement, not just a frontend selection — apps/api must check the requested model against the account’s/key’s allowed_models list before routing to Bifrost, rejecting or falling back appropriately if a request targets a model outside the selection. Sits alongside the idempotency/rate-limit/hold transaction already decided in the request path. Scoped per key — allowed_models is a field on the API key record (see Model selection scope). Because each key maps 1:1 to a Bifrost virtual key, the same list can also be pushed to Bifrost’s deny-by-default model allowlist as a second line of defence. |
| API key ↔ Bifrost virtual key mapping | Router.Africa issues its own API keys (hashed in Postgres, per the API key security model) and maps each 1:1 to a Bifrost virtual key, so Bifrost’s governance features — rate limits, budgets, model allowlists — apply per key without being rebuilt. The Worker authenticates the customer’s key against Postgres, then calls Bifrost using the mapped virtual key. apps/dashboard-api provisions, updates and revokes the virtual key alongside the Router.Africa key. Resolves the design doc’s “per-customer virtual keys” wording against the own-keys model above: both are true, linked one-to-one. Follow-ups this creates: the mapping needs a defined sync/failure path (key created in Postgres but not in Bifrost, and vice versa), and it reopens the currency-unit question in openrouter-api-comparison.md #30 — Bifrost budgets are denominated in USD while the ledger is local currency, so decide which system is authoritative for spend caps. |
| Key ↔ Bifrost sync and failure path | A key is never shown to the customer until its Bifrost virtual key exists. Postgres and Bifrost can’t share a transaction, so “atomic” is implemented as an ordered create with compensation, in apps/dashboard-api: (1) insert the key row in Postgres as pending (hash only, not yet usable for auth); (2) create the Bifrost virtual key with the key’s rate limits and allowed_models; (3) on success, mark the row active and only then return the raw key to the frontend; (4) on any failure, delete or roll back whatever was created on either side and return an error, with no key shown. A pending key can never authenticate, so a crash between steps leaves no usable half-key. Updates (rate limit, models) follow the same order and revert on failure. Revocation marks the key revoked in Postgres first (auth is Postgres-side, so the key is dead immediately), then deletes the Bifrost key with retry. Admin discrepancy view: a section in the admin app lists every mismatch: keys stuck pending, keys active in Postgres with no Bifrost key, Bifrost keys with no Postgres row, revoked keys still live in Bifrost, and config drift (rate limits, allowed models, admin cap differing between the two). Each row supports re-sync or cleanup by an admin, and actions are audit-logged. A periodic reconciliation job populates the view. (Decided 2026-09-20.) |
| Spend cap authority | Two caps, two owners, split by who sets them. The admin-set cap (set by staff in the admin app) is owned and enforced by Bifrost as a virtual key budget, denominated in USD, and pushed by apps/dashboard-api with the key. The user-set cap (set by the customer for their account or key) is owned, calculated and enforced by Postgres in credits, against the ledger, in the hold path. Neither system tries to enforce the other’s cap, so no USD/local-currency conversion of a customer-visible amount is needed. A request must pass both. Resolves openrouter-api-comparison.md #30. Consequence: the admin cap’s USD figure is set by staff, not derived from the ledger, so it won’t track FX moves on its own. Staff set it as a backstop limit, not a precise customer entitlement. (Decided 2026-09-20.) |
| v1 model list | Not a fixed list; it’s configuration. Models are configured in Bifrost. The admin app reads the configured models from Bifrost’s model listing endpoint (to be verified against Bifrost’s API) and shows them. Staff choose the default models available to all users there, which also sets the default pre-checked models in onboarding. Per-key allowed_models (see Model selection scope) stays the enforcement field, so staff add or remove models for a specific customer or key through the admin app, which writes Postgres and pushes to Bifrost per the sync path above. A model is “on” only when staff have explicitly enabled it and it has a published rate (Pricing & Markup Model). Enabling is the single gate: staff set the rate first, then turn the model on. Customers can only add or select models that are on, and apps/api rejects requests for any other model before reaching Bifrost, so usage on a model with no pricing (Bifrost reports cost 0.0 when it has no pricing entry) can’t happen. Bifrost provider keys use explicit model lists, never ["*"], so Bifrost also denies anything not turned on. Newly released models appearing in Bifrost’s catalog via its 24-hour pricing/model sync do not become available until staff turn them on; the default scheduled sync is accepted. Bifrost upgrades are for features, not for new models. The Maxim datasheet URL is a hosted dependency; Bifrost now makes BIFROST_MODEL_PARAMETERS_URL configurable (UI, config.json, Helm), so an override is possible if egress is ever restricted. Closes the split-out question from Launch scope. (Decided 2026-09-20.) |
| Ledger source of truth | Router.Africa’s own Postgres ledger holds money: balances, holds, settlements. Bifrost’s governance layer (virtual keys, budgets, rate limits) is used for enforcement on the gateway side, not as the ledger. This closes the evaluation flagged in infra/plugins-and-extensibility.md and the README (“how much of Bifrost’s governance can be reused before building custom ledger logic”) — answer: reuse it for rate limits, budgets and allowlists via the 1:1 key mapping; build the currency ledger and hold/settle ourselves. |
| Price source of truth / repricing cadence | The design doc (§5) governs balance semantics: balances are held as credits (equivalent to tokens, never labelled with a currency, non-refundable), priced at time of consumption, with repricing applying immediately and uniformly (no grandfathering). The Pricing & Markup Model doc governs the mechanics: a daily published rate, locked at hold time. The business model doc’s monthly review cadence is superseded by the daily rate. The business model’s price table and margin figures are illustrative history, not the current price sheet. |
| Audit trail & prompt/response retention | An audit trail is a v1 requirement, correcting the design doc’s earlier non-goal (“no audit logs”). Request metadata is logged as decided under Request Logging. Prompt and response content is stored by default for every account; customers can opt out (revised 2026-09-20, replacing the earlier opt-in-only position). The setting is per workspace (decided 2026-09-20; surfaced under Data & privacy for the current workspace, see Dashboard Scope), on by default when a workspace is created, and changes are audit-logged with a timestamp. Opting out stops future capture. Retention is indefinite for now; a retention window, customer deletion of stored content, and whether opting out purges existing content are deferred to future-optimizations.md (the key layout supports all three). Redaction: no PII redaction in v1 (unreliable, easy to oversell). Message bodies only are stored; headers and API keys are never stored, and content is kept out of Datadog traces and Bifrost’s own logging. Access: stored content is readable only by admins, both Router.Africa staff admins and customer org admins; other roles, including members, have no access to stored content. Every read of stored content is audit-logged. Audit trail store: not a separate store; audit events live in the same store as the request log (decided). Because capture is on by default, the signup flow, terms and privacy policy must disclose it clearly. Storage: Cloudflare R2, keyed by request ID with a pointer on the request log row, written by the Worker after the response (not via the usage queue, whose message size is too small for content), for every account that has not opted out. Write path (start simple): one object per request holding prompt and response together, gzip-compressed, written once after the response completes and off the hot path, behind a small write interface. Key layout {account_id}/{yyyy}/{mm}/{dd}/{request_id}.json.gz, so per-customer deletion or export is a prefix operation. Streaming responses are buffered and written when the stream ends; on client disconnect, write what was buffered and mark the row partial. A maximum stored size per request is set, with anything larger truncated and flagged. Opted-out accounts write nothing. The pointer on the request-log row is an object key plus optional offset and length columns (empty at launch), so batching many requests into one object later needs no schema change; revisit when R2 write costs pass roughly $100 a month (about 10 req/s sustained at $4.50 per million writes). Infrequent Access storage is not used at launch (writes cost double, 30-day minimum). Bytes and object counts are tracked per account in the rollup tables. Token counts and cost stay in Postgres, never in R2. A failed content write must never fail the request. Message bodies only; headers and keys are never stored. Bifrost’s own content logging (disable_content_logging) stays on the disabled setting: it is a global switch (not per key), has no documented retention or delete path, and would write every customer’s prompts to the droplet Postgres regardless of opt-in. Revisit if Bifrost adds per-key logging and deletion. (Decided 2026-09-20.) |
| Launch scope (markets) | All markets live with Niobi at launch — no phased rollout by market. The v1 model list is a separate question and remains open. |
| Model selection scope | Per key. A customer’s model selection lives on each API key (allowed_models on the key record), so onboarding’s model-selection step applies to the first generated key, and the API Keys screen edits it per key. Closes the previously open account-vs-key question. |
| Dashboard and dashboard-api deploy | Together — apps/dashboard and apps/dashboard-api deploy as one unit from the monorepo. Closes the open sub-decision in the tech stack. |
| Marketing site location | Already built, in a separate repo (Astro), not part of this monorepo. The copy in docs/product/website-copy.md is source material for it. |
| Charge basis vs cost basis | The customer is charged on the token counts Bifrost reports (input and output separately) × the published rate locked at hold time. Bifrost’s usage.cost is recorded per request as Router.Africa’s actual cost (COGS), for margin tracking and reconciliation only; it never determines what the customer is charged. Router.Africa does not count tokens itself for billing. |
| Credit balance & non-refundability | Balances are presented as credits, equivalent to tokens, and are never displayed with a currency symbol or code (dashboard, API, docs, copy). This avoids implying unspent balance is refundable cash. Credits are consumed by usage only. Ledger amount at top-up is 1:1 with the credit amount purchased. |
| Top-up transaction fee (Niobi markup) | Customers choose a credit amount; the collection request is for credit amount + transaction fee (e.g. 1,000 credits + 40 fee → 1,040 collected). Only the credit amount is credited to the balance; the fee is recorded separately as revenue and excluded from the customer-balance side of reconciliation. Fee values are configured per vendor in the admin app, not hardcoded. The fee breakdown is shown before the customer confirms. Distinct from the per-token model markup below. |
| Markup configuration | The system is built to configure markup every supported way (flat and percentage; global, currency/market, model scopes — per the Pricing & Markup Model; account-scoped and tiered markup were dropped 2026-09-20, see Workspace discounts). The launch values are not a build dependency: the schema and rules engine ship first, and the actual amounts and percentages are entered later through the admin app as configuration (decided 2026-09-20). They must be set before the public price sheet is published. |
| Workspace discounts | Rates, FX and markup are never per workspace. Workspace-specific pricing (negotiated or volume) is a percentage discount on usage, set per workspace, for all models or one model (model-specific wins, no stacking), on or off with a reason, audit-logged. Charge = Bifrost-reported tokens × published rate × (1 − discount). The discount is locked with the rate at hold (at request start when hold is off), and the hold is priced the same way. Replaces account-scoped and tiered markup in the Pricing & Markup Model; a volume tier can return later as a tiered discount. (Decided 2026-09-20.) |
| Hold close-out | Settle and release each close their hold in the same transaction as the balance change, so a completed request never leaves a hold behind. No hold expiry and no sweeper job. A Worker that dies between hold and settle leaves an open hold and a raised held; the ledger invariant check flags the stuck hold, and the usage reconciliation (next entries) finds the uncharged call, which an admin charges or releases. (Decided 2026-09-20.) |
| Top-up status check | Pending top-ups are resolved by a scheduled workflow that queries the transaction status from Niobi through niobi-proxy, alongside the callback. Both paths run the same guarded code (UPDATE ... WHERE status = 'pending'), so a top-up is credited once. A top-up still pending past a threshold is flagged in the admin Transactions view; nothing auto-fails. (Decided 2026-09-20.) |
| Log-write and audit rules | The queue consumer is idempotent: INSERT ... ON CONFLICT (created_at, id) DO NOTHING, rollups incremented only when the insert happened, created_at taken from the event. Privileged audit events (ledger adjustments, key and hold-toggle changes, content-logging and discount changes, releasing a stuck hold) are written directly by apps/dashboard-api, not through the queue. A stored-content read writes its audit event before returning content; no event, no content. Content deletion, when it ships, is recorded as a content.deleted audit event and never updates the request-log row. (Decided 2026-09-20.) |
| Ledger invariant check | A scheduled check per workspace: balance = SUM(ledger_entries.amount) and held = SUM(open holds); any difference alerts. Expired idempotency rows are deleted by the same housekeeping run. (Decided 2026-09-20.) |
| Usage reconciliation (Bifrost vs ledger) | An hourly Cron Trigger on apps/dashboard-api finds calls Bifrost served that the ledger never charged. apps/api sends our request id to Bifrost as x-request-id on every call, so a Bifrost log’s id is our request id. The job pages Bifrost’s GET /api/logs (start_time, end_time, min_tokens=1, limit up to 1000) from its last watermark up to now minus a 15-minute grace, and checks each log id against core.settlements.request_id. No settlement: a not_charged row in usage_discrepancies. Settlement with different token counts: a token_mismatch row (the sanity check the disconnect policy calls for). An admin resolves a not_charged row by charging it or releasing it without charge, reason required, audit-logged. Charging uses the normal settle code path at the rates and discount locked on the request’s open hold; with no hold (hold off), the published rate in effect at the Bifrost log’s time and the workspace’s current discount. The settlement is flagged source = 'recon' and the hold is closed. Nothing is charged automatically; automatic charging for calls with an open hold is a possible later step. Bifrost still logs tokens, cost and status with disable_content_logging on, so content logging stays off. To verify in staging before relying on it: x-request-id becomes the Bifrost log id, and log rows carry the virtual key id. (Decided 2026-09-20.) |
| Scheduled jobs admin page | The admin app lists every scheduled job with its status and lets staff change when it runs and switch it on or off. Cron Trigger schedules can’t be edited at runtime, so apps/dashboard-api has one Cron Trigger firing every minute (a dispatcher), and each job’s schedule (cron expression, UTC), enabled flag and config live in core.scheduled_jobs. The dispatcher runs whatever is enabled and due, holding a lease per job so runs never overlap (a lease older than the job’s max runtime is treated as a dead run and reclaimed). Each run is recorded in logs.job_runs (started, finished, status, summary counts, error). Screen: per job last run, last status, next run, running now, recent run history; edit schedule and config (validated); enable or disable (reason required, warns that disabling means stale rates, undetected unbilled usage, and so on); Run now. Admin role can edit, Support role read-only. Every change and manual run is audit-logged. The job set is fixed; jobs are not added or removed from the screen. Minimum schedule granularity is one minute. (Decided 2026-09-20.) |
| Item | Why it’s unresolved | Blocks |
|---|---|---|
| None | All open decisions are closed as of 2026-09-20. Deferred work is tracked in future-optimizations.md and launch follow-ups in the backlog (item 17). |
— |
Deferred optimizations and the hooks that keep them cheap are tracked in future-optimizations.md; add a row there whenever a decision says “later”.
Keep this file current as decisions close — an item moving from Open to Decided should be a deliberate edit, not something that quietly happens elsewhere and never gets reflected here.