Skip to main content
kRouter

Architecture

What happens to one request inside kRouter: access check, format detection, routing, translation, token savers, the upstream call and the reply.

This page follows one request from the moment your tool sends it to the moment the reply streams back.

The stages

Your tool
 │  POST /v1/chat/completions, /v1/messages or /v1/responses
 ▼
1.  Access check          local caller or valid API key
 ▼
2.  Format detection      OpenAI, Claude, Responses or Gemini shape
 ▼
3.  Model resolution      combo, model alias or provider/model
 ▼
4.  Account choice        Zenith (default) or another strategy
 ▼
5.  Translation, savers   into the provider's format; RTK and friends
 ▼
6.  Upstream call         proxy-aware, per-account concurrency limit
 ▼
7.  Reply                 translated back, recorded, streamed to you

1. Access check

Calls to /v1/* from the machine kRouter runs on pass without a key unless REQUIRE_API_KEY=true. Every other caller needs a valid API key or gets a 401. "Local" is decided from the connection's peer address, never from the Host header, and a request carrying X-Forwarded-For or X-Real-IP (as a reverse proxy adds) always counts as remote. Dashboard pages and /api/* need a dashboard login, and /api/* routes are refused unless they are on an allow-list.

2. Format detection

The path decides first: /v1/responses is the Responses API and /v1/messages is Claude Messages. Otherwise the body decides: input without messages means Responses, contents means Gemini, OpenAI-only fields such as response_format or stream_options mean OpenAI, and Claude-style content blocks mean Claude. The detected format picks the translator used in stage 5.

3. Model resolution

The model field is read as one of:

  • a combo name: the combo's list becomes the fallback chain (see Combos & fallback)
  • <alias>/<model-id>: everything before the first / names the provider
  • a model alias you defined

kRouter also looks at what the request carries. If it contains an image (or a PDF, audio or video) and the chosen model cannot read it, the capacity adapter brings in a model that can. The image pool is on by default; the others are opt-in.

4. Account choice

kRouter keeps every active account in an in-memory cache, re-synced from SQLite every 10 seconds, so the hot path does not query the database. It drops accounts that are cooling down or banned and picks one of the rest. The default strategy, Zenith, scores accounts on recent success rate and response time, scales down accounts under 30% remaining quota, and takes the highest score. Unless the provider uses round-robin, later turns of the same conversation go back to the same account when it is free, to keep the provider's prompt cache warm. See The Zenith Routing Engine.

5. Translation and token savers

The request is converted from your tool's format into the provider's. For example, OpenAI messages become Claude messages and tools, or Gemini contents with a systemInstruction, and Responses input items become chat messages. Some providers need more: Kiro answers in the binary AWS EventStream format, which kRouter turns back into a normal stream.

The token savers then run on the translated body: RTK (on by default), then Caveman or Ponytail, Headroom and PXPIPE when you have turned them on. They are skipped when a client sends X-9Router-Token-Saver: off, and when Claude Code calls the Claude Code provider directly, so that Anthropic's prompt cache keeps working.

6. Upstream call

The request goes out through proxyAwareFetch, which honours HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, NO_PROXY and per-account proxy pools.

Each account of some providers has a limit on requests in flight, so a busy tool cannot flood it: Claude Code 5, Kiro 4, Codex 3, Antigravity, Gemini CLI and Qoder 2 each, OpenCode Free and MiMo Code Free 1. A request that cannot get a slot waits (5 to 20 seconds, depending on the provider) before it tries another account. Other providers have no limit unless you set a per-account maximum.

Expired OAuth tokens are refreshed before use, and concurrent refreshes of the same token are merged into one.

If the provider returns an error, kRouter classifies it (the full table is in Combos & fallback), stores a cooldown on that account in SQLite so it survives a restart, and tries another account, then the next combo entry.

7. Reply

Streamed chunks are translated back into your tool's format as they arrive; non-streaming replies are converted the same way. Token counts go to the Usage page. With observability on (the default), the last 1,000 requests are also kept for the dashboard's request log; see Security for exactly what that stores.

File layout

In the kRouter repository:

/open-sse/handlers/     # request entry, dispatch
/open-sse/services/     # account picker, combos, fallback, quota
/open-sse/executors/    # per-provider HTTP calls
/open-sse/translator/   # format conversion
/open-sse/rtk/          # token savers
/open-sse/config/       # provider models, error rules
/src/sse/services/      # auth, token refresh
/src/mitm/              # MITM intercept layer
/src/app/api/           # API and dashboard routes
/src/app/(dashboard)/   # dashboard UI (Next.js App Router)

Where to go next