Skip to main content

Spot the cache bug

Four versions of the same system prompt. One of them caches. The other three do not, and every one of the three returns HTTP 200 with a correct answer and no warning of any kind.

Read each variant and decide before you reveal the usage block. The guessing is the exercise.

src/prompts.ts
system: [
  {
    type: "text",
    text: `${ROLE}\n\n${POLICY_HANDBOOK}`,
    cache_control: { type: "ephemeral" },
  },
  { type: "text", text: volatileContext(opts) },
]
usage on the 2nd identical call

Does this cache? Every variant returns HTTP 200 with a correct answer, so the response body will not tell you.

The only field that distinguishes these four is cache_read_input_tokens. That is why it is the thing to alert on, and why a caching change cannot be validated by reading the diff.

Break the real thing yourself in Lab 5, Step 2, then read how the prompt is assembled in the architecture notes.