Duplicate AI replies? The response cache gotcha explained

You say "Hi" to your AI assistant. It replies, "Hello, how can I help you?" Then you ask something completely different, and it replies "Hello, how can I help you?" again. The same canned answer, for the wrong question.

If a response cache sits between you and the model, check it first. This exact bug was reported by a kRouter user on Antigravity in July 2026. This post explains why it happened, what kRouter changed, and how to tell whether a cache is the one answering you.

What a response cache does

Some traffic repeats byte for byte within seconds: title generation, structured-output retries, "is this model reachable" probes. A short-lived cache answers those from memory and saves the provider call.

kRouter's response cache is off by default. When it is on, each request is keyed on a hash of its shape: the model, the system prompt and instructions, the conversation (messages, input or contents), temperature, max tokens, tools, tool choice and response format. Same key, same answer, for five minutes. The concepts page summarizes it alongside kRouter's other features.

Claude Code's own warm-up and title requests never get that far. kRouter recognizes them and answers them itself, without calling a provider or touching the cache.

Where it went wrong

A cache key is only as good as the fields it reads. If the part of a request that changes the answer is not in the key, two different questions can share one entry.

That is what happened with Antigravity. Its IDE sends Gemini-format requests with the conversation nested inside a request object, and it signals streaming in the URL rather than in the body. kRouter's key reads the conversation from top-level fields. For an Antigravity request those were all empty, so every turn on the same model produced the same key, and nothing in the body marked the request as streaming. The first reply that got cached became the answer to every turn after it, until the entry expired.

Standard Chat Completions, Claude Messages, Responses and Gemini requests put the conversation where the key looks, so a new question gets a new key.

What kRouter does now

The cache refuses anything it cannot safely replay. A request is cached only when all of these hold:

1. It is not for Antigravity, Gemini or Gemini CLI. These providers never use the cache, whatever your toggle says. The check matches the provider alias as well as the id, because a request arrives as ag/gemini-3-flash-agent, not antigravity/.... The first fix, in v0.5.99, listed only the ids, so it never fired. v0.5.101 added the aliases, and the fix was confirmed against a running instance.
2. It is not streaming. Streamed replies are never cached.
3. Its temperature is 0.3 or lower. Anything higher is treated as non-deterministic. A request that sets no temperature counts as cacheable, even if the provider's default is higher. If replies to such a request should vary, send a temperature above 0.3.
4. Its max_tokens is not tiny. A limit under 32 is almost always a warm-up ping, so it is skipped.
5. It does not force a tool call. tool_choice: "required" usually means the model is in the middle of a task.
6. The reply was worth keeping. Only HTTP 200 replies of at least 100 characters are stored, so error stubs and empty completions cannot poison the cache.

Entries live in memory: five minutes each, at most 500, oldest dropped first. Restarting kRouter empties the cache.

How to tell whether the cache answered

Look for the header. A reply served from kRouter's cache carries X-Cache: HIT and X-Cache-Age-Ms. A reply from the provider has neither. Send the same request twice and compare:

curl -si http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"<provider>/<model>","temperature":0,"messages":[{"role":"user","content":"Say hi"}]}' \
  | grep -i '^x-cache'

Watch the card. While the cache is on, the Response Cache card on the dashboard's Endpoint page shows hits, misses, hit rate, tokens saved and how many requests were skipped.

Clear it. The same card has a Clear cache button that empties the cache and resets the counters. Turning the toggle off stops lookups from the next request.

What you should do

• Leave it off if you are unsure. That is the default.
• Turn it on if you send a lot of repeated, deterministic, non-streaming traffic: batch jobs, structured extraction, repeated title generation. Set a low temperature on those requests.
• On Antigravity, Gemini and Gemini CLI, the toggle makes no difference. kRouter bypasses the cache for them, so answers are correct either way.
• Coding agents mostly stream, so the cache rarely fires for them. If you want fewer tokens in a coding session, RTK compression is the saver that applies there: it is on by default and compresses the tool output inside each request, streamed or not.

Common questions

Why does my AI give the same answer to different questions?

If a response cache is in the path, its key probably does not cover the part of the request that changed. Check the wrong reply for an X-Cache: HIT header. In kRouter this was the Antigravity bug fixed in v0.5.101. Turning the cache off, or clearing it, stops it at once.

Is kRouter's response cache on by default?

No. It stays off until you switch it on in the Response Cache card on the Endpoint page. Even then it never caches streamed replies, or requests to Antigravity, Gemini or Gemini CLI.

How long does kRouter keep a cached reply?

Five minutes, in memory, with at most 500 entries. Restarting kRouter empties it.