How the Zenith routing engine eliminates AI rate limit stalls

If you route AI traffic across a dozen free and paid accounts, you eventually run into the Cascade Problem.

In legacy AI routers (like LiteLLM or vanilla 9router), fallback logic is sequential. When Account 1 runs out of quota, it returns a 429 error. The router catches the error, marks the account dead, and tries Account 2.

But what if Account 2 is also out of quota? The router hits a 429 again. By the time the router tries Account 5, 20 seconds have passed. Your IDE times out. The AI agent crashes.

We built the Zenith Score Engine in kRouter v0.5.75 to solve this completely.

Dumb Routing vs. Intelligent Scoring

Legacy routers wait until an account fails before switching away from it. Zenith anticipates the failure.

Before every request, Zenith evaluates every available account in your fallback chain using a mathematical scoring function. It reads two live metrics directly from the HealthCache RAM layer (sub-millisecond reads):

1. TTFB Latency -- the rolling average Time-To-First-Byte over the last 10 requests for that account.
2. Quota Headroom -- the percentage of daily or weekly quota remaining, pulled from the provider's rate-limit response headers.

The Scoring Formula

Each account receives a Zenith Score computed as:

score = (ttfb_weight * ttfb_score) + (quota_weight * quota_score)

Where:

• ttfb_score = normalized inverse of the average TTFB (lower latency = higher score). An account averaging 200ms TTFB scores higher than one averaging 800ms.
• quota_score = percentage of remaining quota, normalized to 0-1. An account with 80% daily quota remaining scores 0.8.
• ttfb_weight and quota_weight are configurable but default to 0.4 and 0.6 respectively. Quota is weighted higher because running out of quota causes a hard failure (429), while higher latency is just slower.

The 30% Penalty Threshold

When an account drops below 30% remaining quota, Zenith applies an aggressive penalty multiplier. The quota score is not just 0.3 -- it gets squared, pushing it to 0.09. This exponential penalty shoves the account to the back of the line before it actually hits 0%.

The effect: traffic is proactively migrated away from dying accounts while they still have headroom, preventing the cascade problem entirely.

Concrete Example: 5 Accounts

Suppose you have 5 accounts configured in a combo:

Kiro Free #2 has 22% quota remaining -- below the 30% threshold. Its quota score gets penalized from 0.22 to 0.05 (squared), pushing it to rank 4 even though its latency is excellent. Zenith routes traffic to Kiro Free #1 first, not because it is listed first, but because it has the best combination of low latency and high quota headroom.

By the time Kiro Free #2 actually hits 0%, your traffic has already moved elsewhere. No 429. No stall. No IDE timeout.

The v0.5.77 Wiring Fix

In v0.5.77, we completed the Zenith integration by rewiring the account selection pipeline. Previously, auth.js handled account selection directly using a simple round-robin. Now, auth.js delegates to accountSelector.js, which calls the Zenith scoring engine.

This means Zenith scoring applies to every request path -- chat completions, TTS, embeddings, image generation, everything. Any endpoint that uses account-based routing gets the Zenith brain automatically.

The Result: Zero Rate-Limit Stalls

Because Zenith routes traffic away from dying accounts before they actually die, you never hit the 429. Your IDE never sees a connection stall. The fallback is entirely invisible.

And when you do have multiple healthy accounts with 100% quota, Zenith routes traffic to the one with the lowest live latency, ensuring your autocomplete responses stay sub-second.

How to Use Zenith

Zenith is the default routing strategy in kRouter as of v0.5.77. You do not need to configure anything.

If you previously set up your combos to use fill-first or round-robin, they have been automatically upgraded to the Zenith engine. If you prefer the old sequential behavior, you can manually revert a combo to fill-first in the dashboard, but we do not recommend it. Zenith is the brain that makes 20+ account proxy setups feasible.

# Verify your installation is running the Zenith engine
npm install -g @sifxprime/krouter@latest
krouter -v
# Output should be v0.5.75 or higher.

For the full technical details, see the changelog. For a deeper look at the RAM layer that makes Zenith's sub-millisecond reads possible, read Building a sub-5ms failover proxy.