Model knowledge cutoffs don't match the published dates

The model knowledge cutoff on a vendor spec sheet is a claim about training data, not a measurement of what the model knows. On Aug 10, independent researcher Shrivu Shankar published a probing method that measures the second thing — and the two don't line up. Claude Opus 5 ships with a published cutoff of May 2026 and answers historical questions like a model that stopped reading in January.

What actually happened

Shankar's write-up uses three cheap probes. An 8-way multiple-choice quiz built from Wikipedia daily-facts datasets, asking what happened on a given day. Direct questions about today's date. And fifty variations of "what model are you?" to see how consistently a model identifies its own lineage.

Run across frontier models, the probes cluster into training runs. Anthropic's Opus 4.7 and later share a very similar effective cutoff around late December 2025 — consistent with one underlying run behind several releases. OpenAI's GPT-5.6 family traces to a checkpoint distinct from GPT-5.5's, finishing around late February 2026.

Then there's Opus 5. Published reliable cutoff: May 2026. Measured factual knowledge: roughly what the January 2026 models had. Shankar doesn't claim to know why — post-training, dataset engineering, and data mixture decisions all sit between a scrape date and what survives into the weights.

This is one researcher's methodology on black-box models, so treat the specific dates as estimates, not audit findings. The method is the point.

Why model knowledge cutoffs matter for your business

A cutoff is a spec, and specs are marketing until you test them. If you built a support agent that answers questions about your own pricing page, the cutoff never mattered — you're feeding it context. If you built anything that leans on the model's world knowledge — competitor lookups, regulatory summaries, "is this vendor still in business" — a four-month gap between claimed and actual is a wrong answer delivered confidently.

The probe is a ten-minute build. Assemble twenty facts you know the date of, half before and half after your assumed cutoff, and score the model. Do it on every model you route to. We do this for clients when swapping models, and it's caught more surprises than benchmark scores ever have.

Pin the version, then retest. The same lineage appearing under multiple release names is exactly why we keep telling people to pin model aliases rather than pointing at latest. A vendor can ship a new name over an older run, and your evals won't notice unless you check.

Retrieval is the fix, not a bigger model. Anything time-sensitive belongs in a retrieval layer you control, with a source and a timestamp attached. That's the difference between an answer you can defend and one you have to apologize for.

Sources: Shrivu Shankar — Exploring Claude/GPT Knowledge Cutoffs.