Phase 3 — Request Path (`apps/api`)
Goal: the OpenAI-compatible API, end to end: authenticate the key, gate the model, claim idempotency and hold credit, call the gateway privately, settle, then log off the hot path.
Milestone M1: in staging, a real chat completion with a hand-seeded workspace, key and balance is charged exactly tokens × the locked rate × (1 − discount). The hold closes, the ledger balances, and the concurrency test passes.
This is the money path. P3.09 (the concurrency test) is written alongside P3.04 and P3.06, not after them.
| ID | Task | Who | Needs | Status |
|---|---|---|---|---|
| P3.01 | Bifrost adapter: inference | Agent | P1.09, P0.10, P0.13 | merged — code #71 (inference and management; against a scripted Bifrost v2.2.3; live checks wait on P0.13, then the five checks in #71’s description) |
| P3.02 | Key authentication | Agent | P1.10, P1.08 | merged — code #23, fix code #61 |
| P3.03 | Model gate and request read | Agent | P3.02, P2.05 | merged — code #36 (Bifrost adapter code #71 in production, mock gateway code #69 in staging) |
| P3.04 | Idempotency claim and hold | Agent | P3.03, P2.06 | merged — code #36; hold fallback of 100,000 output tokens code #50; hold as one function core.claim_and_hold code #105 |
| P3.05 | Gateway call | Agent | P3.01, P3.04 | merged — code #36 (Bifrost adapter code #71 in production, mock gateway code #69 in staging) |
| P3.06 | Settle and release | Agent | P3.05 | merged — code #36; exact absorbedShortfall code #74; close the hold first, one lock order code #96; open unbilled usage checked inside settle code #101 |
| P3.07 | After the response | Agent | P3.06 | merged — code #36 (Bifrost adapter code #71 in production, mock gateway code #69 in staging) |
| P3.08 | Endpoints | Agent | P3.07 | merged — code #36 (chat, models, key; /v1/messages and /v1/responses are launch endpoints, built after P0.13) |
| P3.09 | Hold/settle concurrency test | Agent | P3.04, P3.06 | merged — code #40; runs on real Postgres in the deploy gates and manual Gates runs (code #54) |
| P3.10 | API conformance tests | Agent | P3.08 | local part merged — code #46 (OpenAI SDK, conformance item 20); the staging part waits for a deployed staging stack |
| P3.11 | M1: first charged request in staging | You + agent | P3.08, P3.09 | todo |
| P3.12 | GET /v1/generation (request lookup) |
Agent | P3.08 | todo |
P3.01 — Bifrost adapter: inference
Section titled “P3.01 — Bifrost adapter: inference”Who: Agent · Needs: P1.09, P0.10, P0.13
Implement the adapter’s chat completion, streaming and not, through the inference VPC binding:
-
send the decrypted virtual key and our
x-request-id -
set the Worker’s timeout comfortably above Bifrost’s worst-case fallback time
-
do no Worker-level retry
-
map Bifrost errors to the catalogue from P1.08
-
strip every Bifrost-identifying response header
-
Spec: decisions log, Worker → Bifrost timeout/retry policy, Gateway abstraction, Private service access;
../architecture/backend-scope.mdsection 3 step 4 -
Done when: against staging, a streamed and a non-streamed completion return correctly, no
x-bf-*or other gateway header reaches the client, and a provider failure maps to a catalogue error.
P3.02 — Key authentication
Section titled “P3.02 — Key authentication”Who: Agent · Needs: P1.10, P1.08
Hono middleware:
-
Authorization: Beareronly (plusx-api-keyonPOST /v1/messagesalone): key format and checksum checked locally first, so malformed keys never hit the database. -
Lookup: SHA-256 hash, then the key must be
activeand unexpired, its workspace must not befrozenordeleted, and its org must not besuspended(the “account suspended” error); resolve the workspace and org. -
Responses: identical
401s for every failure, with the masked key only in messages. The key is never logged. -
Rejected-key cache: about 60 seconds, rejected hashes only, so revocation stays immediate.
-
Spec:
../architecture/auth-security.mdsections 1 and 2;../architecture/api-key-security.mdsection 6; decisions log, Edge rate limiting, Org suspension -
Done when:
- the auth items of the conformance suite that apply to
apps/apipass - a revoked key fails on its next request
- keys in a frozen workspace and a suspended org are refused on their next request
- a burst of the same bad key hits Postgres once per cache window
- the auth items of the conformance suite that apply to
P3.03 — Model gate and request read
Section titled “P3.03 — Model gate and request read”Who: Agent · Needs: P3.02, P2.05
The single unlocked read:
- the key’s allowed models
- the model is on, with a current rate in the workspace’s currency
- the discount
- hold settings
- the content-logging flag
Apply the max_tokens ceiling and default.
- Spec:
../architecture/db-schema.mdsection 5 step 1; decisions log, Model availability enforcement - Done when: a model not on the key, a model that is off, and a model with no rate are each rejected with no database write. The read uses only primary-key or unique lookups.
P3.04 — Idempotency claim and hold
Section titled “P3.04 — Idempotency claim and hold”Who: Agent · Needs: P3.03, P2.06
One statement, core.claim_and_hold (since code #105, 2026-11-05; it was one short client-side transaction), in this lock order:
- Idempotency claim: automatic
hash(workspace + exact body), plus the optionalIdempotency-Keyheader. - User spend-cap window.
- Atomic conditional hold at the locked, discounted rates, with the minimum hold per currency and the input allowance. With hold off for the org: a plain balance check and no
holdsrow.
Conflicts return 409 or the stored response.
- Spec:
../architecture/db-schema.mdsection 5 step 2; decisions log, Credit hold/settle mechanism, Hold/settle concurrency safety, Idempotency key design, Per-organization hold toggle - Done when: P3.09 passes, a repeated body within the window returns the stored response with no new hold, and over-cap or below-minimum requests roll back cleanly.
P3.05 — Gateway call
Section titled “P3.05 — Gateway call”Who: Agent · Needs: P3.01, P3.04
- Decrypt the key’s gateway credential (AES-256-GCM, with key id and gateway id as associated data).
- Call the adapter and pass SSE through.
- On client disconnect, keep Bifrost’s reported partial usage for settle and keep the Worker’s own count as a sanity check.
- Spec: decisions log, Bifrost virtual key secret storage, Mid-stream client disconnect policy;
../architecture/db-schema.md4.2 - Done when: a ciphertext moved to another row fails to decrypt (test), and a disconnected stream settles on the reported partial usage.
P3.06 — Settle and release
Section titled “P3.06 — Settle and release”Who: Agent · Needs: P3.05
Settle, in one transaction:
- charge = tokens × locked rates × (1 − locked discount)
- debit
LEAST(charge, balance), recording any absorbed shortfall - reduce
held - write the settlement and ledger rows
- close the hold
- adjust the spend window
Release on any failure or gateway rejection (including 429), with no retry.
- Spec:
../architecture/db-schema.mdsection 5 step 3; decisions log, Charge basis vs cost basis, Hold close-out - Done when: after any mix of successes and failures,
balance = SUM(ledger)andheld = SUM(open holds), checked in tests.
P3.07 — After the response
Section titled “P3.07 — After the response”Who: Agent · Needs: P3.06
Off the hot path, with ctx.waitUntil:
- Queue: publish the v1 usage event, metadata only, carrying the gateway cost as COGS.
- R2: write content when the workspace hasn’t opted out: one gzip object per request, keyed by the specified path layout. Buffer streams, mark a disconnect
partial, truncate past the size cap. A failure never fails the request. - Idempotency: store the response and mark the row completed.
Decide in the PR: whether a failed queue publish retries or is only alerted on (backend-scope.md section 7; ledger correctness doesn’t depend on it).
- Spec: decisions log, What is queued, Audit trail & prompt/response retention;
../architecture/db-schema.md4.5 - Done when:
- events land on the staging queue
- content objects appear in R2 for a logging workspace and not for an opted-out one
- forcing an R2 failure still returns
200
P3.08 — Endpoints
Section titled “P3.08 — Endpoints”Who: Agent · Needs: P3.07
POST /v1/chat/completionsGET /v1/models, scoped to the keyGET /v1/key: the key’s cap, usage and expiry, and the workspace balance, all in creditsPOST /v1/messages(launch; acceptsx-api-keyas well as Bearer, errors in Anthropic’s format,api-errors.mdsection 7) andPOST /v1/responses(launch), both verified end to end in P0.13
No CORS. Every response carries the request id.
- Spec:
../product/api-surface.mdsections 1, 2 and 4; decisions log, Credit balance & non-refundability (wording) - Done when: the OpenAI and Anthropic SDKs work against staging with only a base-URL change (for each endpoint that ships), and no response shows a currency symbol for a balance.
P3.09 — Hold/settle concurrency test
Section titled “P3.09 — Hold/settle concurrency test”Who: Agent · Needs: P3.04, P3.06
N concurrent requests against one workspace, on real Postgres (the CI container, and once on staging), asserting no over-approval: total holds never exceed balance minus held, and the ledger balances afterwards. Include concurrent settles and releases, and the spend-cap window. Run it in the PR gates if it’s fast enough (cicd.md section 9), otherwise on every merge.
- Spec: decisions log, Testing approach;
../architecture/cicd.mdsection 2 - Done when: it passes at N = 100, and fails when the conditional
WHEREis removed (to prove the test catches it).
P3.10 — API conformance tests
Section titled “P3.10 — API conformance tests”Who: Agent · Needs: P3.08
Every conformance-suite item that applies to apps/api, plus the key-lifecycle tests in api-key-security.md section 9. Add:
-
conformance test 20: the test key appears in no log or output
-
a test that the client’s
Authorizationheader never reaches the gateway -
Spec:
../architecture/auth-security.mdsection 6;../architecture/api-key-security.mdsection 9 -
Done when: they run in the PR gates and pass.
P3.11 — M1: first charged request in staging
Section titled “P3.11 — M1: first charged request in staging”Who: You + agent · Needs: P3.08, P3.09
- Agent: add a canary script that sends one tiny request through the full path using the seeded internal workspace, then checks the settlement and ledger rows. It becomes the post-deploy smoke check and, later, the production canary.
- You: confirm the charge against the published rate by hand once.
- Spec:
../architecture/cicd.mdsection 6;../architecture/observability.mdsection 6 - Done when: the pipeline’s smoke step runs the canary on every staging deploy and it passes.
P3.12 — GET /v1/generation
Section titled “P3.12 — GET /v1/generation”Who: Agent · Needs: P3.08
Public at launch (decided 2026-11-07). GET /v1/generation?id=<request id> returns the metadata of one past request, read from request_logs:
- request id, time, endpoint, model, status code and error code
- latency, whether it streamed, input and output tokens
- the charge in credits (no currency symbol, no money wording)
Rules:
- Scope: only requests from the calling key’s workspace; any other id returns
404, the same as an unknown id. - No content, ever. Prompts and responses are readable only by admins, and every read is audited (decisions log, Audit trail & prompt/response retention). There is no
/generation/content. - Not fields we don’t log.
request_logshas no attempt or fallback count, so the response has none until the schema adds one. Do not promise “attempts” in the docs. - Lag: rows are written after the response through the queue (P3.07), so a lookup straight after a request can return
404briefly. Document it; clients retry. - Standard errors from
api-errors.md; no CORS; every response carries the request id. - Spec:
../product/api-surface.mdsection 2;../architecture/db-schema.md(request_logs);../architecture/api-errors.md - Done when: a key reads its own request after a charged call in staging, gets
404for another workspace’s id, no content or money symbol appears in any response, and it is in the curated OpenAPI spec and its contract test (D.06).