One token over the line: the long-context price cliff

Nothing in your setup changed. Same repository, same agent, same model id in the same config file. But this week's invoice is five times last week's, and there is no error in any log to explain it.

The usual suspects come up clean. No retry storm, no runaway loop, no accidental switch to a bigger model. What actually happened is that your average request drifted across a line in the price list. On Claude Haiku 5.5, as of 19 October 2026, that line sits at 100,000 prompt tokens, and crossing it by a single token reprices the entire request.

The price list has two rows for one model

Anthropic's pricing page lists Haiku 5.5 twice, as "Claude Haiku 5.5 (for prompts up to 100,000 tokens)" and "Claude Haiku 5.5 (for prompts over 100,000 tokens)". The two rows are identical in shape and differ by exactly five times in every column:

Batch processing splits the same way: $0.05 input and $0.25 output below the line, $0.25 and $1.25 above it.

The mechanism is spelled out in the page's long context pricing section, and it is the part people miss:

A request's prompt length counts all of its input tokens, including cache reads and cache writes. Each request is priced on its own: a request over the threshold pays the higher prices even when part of its prompt is a cache hit, and earlier requests keep the prices they were billed at.

Three things follow from that sentence. The surcharge is not marginal, so it is not "the tokens above 100,000 cost more" but "all of them do". Cached tokens count toward the measurement. And nothing is retroactive, so a single large request does not reprice the rest of your month. Your bill is the sum of independent per-request decisions, which is exactly why a slow drift in average prompt size can multiply it without any single moment that looks like a change.

Every vendor draws the line somewhere else

There is no industry threshold. Each vendor picked a number, and the numbers are not close to each other.

That last column matters more than it looks. Anthropic states the full-request rule in prose. xAI's model page carries a note under its tables saying that "requests whose prompt reaches the listed token threshold are billed at the higher rate for all tokens in the request". Google and OpenAI publish both price sets and define the boundary — OpenAI's page says only "Short context: ≤272K input tokens. Long context: >272K input tokens" — but neither says whether the higher rate reprices everything or only the overflow. Plenty of third-party write-ups assert the expensive reading for both. That claim is not on either vendor's page, so treat it as unconfirmed: budget as if it were whole-request, and check it against a real invoice before you rely on either answer.

One more reversal worth absorbing, because it inverts an old habit. The same Anthropic page says that "Claude 4.6 and later models (except Claude Haiku 5.5) ... include the full 1M token context window at standard pricing. (A 900k-token request is billed at the same per-token rate as a 9k-token request.)" So on Anthropic's current line-up the cheapest model carries the cliff and the expensive ones do not. "Long context costs extra on big models" is now backwards.

The cache does not save you

This is the trap that catches agent workloads specifically, because agents are built around a large stable prefix that the cache is supposed to make nearly free.

Anthropic's usage fields do not overlap. input_tokens is documented as the number of input tokens "which were not read from or used to create a cache", and the total is cache_read_input_tokens + cache_creation_input_tokens + input_tokens. The pricing page says the threshold is measured against that total. So a loop whose cached prefix has grown to 99,000 tokens crosses the line on a 1,200-token turn.

The cache is not being ignored — those 99,000 tokens really are billed as cache reads rather than fresh input. They are simply billed at $0.05 per million instead of $0.01. You keep the caching discount and lose five times more than it saves.

And the multipliers stack. Anthropic's page notes that pinning inference to the US with inference_geo applies a 1.1x multiplier to every pricing category, and says in so many words that on Haiku 5.5 it also applies to the over-100,000 prices.

What does not work

Budgeting as if the surcharge were marginal. Where vendors have said, it is not. A 101,000-token request on Haiku 5.5 costs almost exactly five times a 99,000-token one, not 1.02 times.

Watching the context window instead of the prompt length. These are different numbers and they are not close. Haiku 5.5 has a one-million-token context window; its price threshold is at 100,000. A request can be comfortably inside the window and ten times past the price tier. kRouter's own model catalog records Haiku 5.5 at a 1,000,000-token window, and that figure is a capability, not a billing boundary.

Trusting the token count your client prints. kRouter adds a fixed 2,000-token buffer to the usage object it hands back to the client, so a client that does its own context-window arithmetic errs on the safe side. That buffer is applied only on the way out; the figures kRouter logs and stores are the provider's own, unmodified. Compare the logged numbers against the threshold, not what your editor shows you.

Expecting a router to notice the line for you. kRouter does not route on measured prompt size against a billing threshold, and it would be dishonest to imply otherwise. It has a request-complexity classifier, but it is advisory and log-only by design, and its length signal saturates at around 10,000 characters of user text — two orders of magnitude below any of these thresholds. It also does context-budget arithmetic at 80% of a model's window, but that runs only when a request falls back to a capability-pool model with a smaller window, and it trims history to fit a window rather than to land under a price tier.

The fix: measure, then choose

Measure first. kRouter writes one usage line per request from the provider's own reported numbers, which is the same accounting the provider bills from:

[USAGE] CLAUDE | in=1200 | out=430 | account=3f9c2a81... | cache_read=99000

For an Anthropic model, the number to compare against 100,000 is in plus cache_read plus cache_create, not in alone. If the upstream returned no usage at all, kRouter estimates it and marks the line as estimated — do not budget on those.

Then pick a move. Once you know where your requests actually sit, the decision is short:

Compressing, honestly. kRouter's token savers are the lever here, and the cliff is what makes them worth configuring: near a threshold their payoff is the multiplier, not the percentage. RTK is the only one on by default. Headroom and PXPIPE are opt-in and need extra local components. A client can send X-9Router-Token-Saver: off to exempt one request from all of them when byte-exact context matters.

There is a real conflict to call out, and it lands exactly on the model with the sharpest cliff. When a Claude client talks to the Anthropic API through kRouter, kRouter skips every token saver by default, because any mutation of the body changes Anthropic's prompt cache key and costs you the cache. So on that path you get cache stability and no compression — and because cache reads count toward the threshold, the cache you are protecting will not keep you under the line. You can set the cache-control mode to never on the dashboard and compress instead, but you are then trading the cache discount for the tier. Which one wins depends on your ratio of cached prefix to new tokens, and the usage lines above are how you find out. kRouter will not make that call for you.

Common questions

Does the higher price apply only to the tokens above the threshold?

Not on the vendors that have said. Anthropic states that a request over the threshold pays the higher prices on its whole prompt, and xAI's model page says requests at or above its threshold are billed at the higher rate for all tokens in the request. Google's and OpenAI's pricing pages define their thresholds and publish both price sets but do not state how the rate is applied, so assume the expensive reading until an invoice tells you otherwise.

Do cached tokens count toward the threshold?

On Anthropic, yes, explicitly: prompt length counts all input tokens including cache reads and cache writes, and a request over the line pays the higher prices even when most of its prompt was a cache hit. This is why long-running agent loops with a big stable prefix are the workload most exposed to the cliff.

Does kRouter keep my requests under the threshold automatically?

No. kRouter has token savers that reduce prompt size and it logs the provider's own token counts per request, but it does not measure a prompt against a billing threshold and route on the result. Staying under a line is something you configure deliberately after looking at your own numbers.

Which token count should I compare against the threshold?

The provider's reported input accounting for that single request, summed across its cached and uncached parts. On Anthropic that means adding the cache-read and cache-creation counts to input_tokens. Do not use your client's displayed figure if it came through kRouter, because kRouter pads that number by 2,000 tokens on purpose.

Do the big Claude models still charge extra for a long context?

As of 19 October 2026, no. Anthropic's pricing page says Claude 4.6 and later models, with Haiku 5.5 as the stated exception, include the full one-million-token context window at standard pricing, and gives the example that a 900k-token request bills at the same per-token rate as a 9k-token one.

Related

• Headroom: the token saver that reclaims your context window
• RTK token compression
• Where each kRouter token saver acts on a request
• The cheapest LLMs for coding in 2026
• Claude Code's context window on other models

Get started

kRouter is a self-hosted gateway in front of 95+ providers, and it logs every request's token accounting locally so you can see where your prompts actually sit:

npm install -g @sifxprime/krouter
krouter -t

Then open the Token Saver docs for what each saver changes, or the install guide to get it running.