Encrypted AI reasoning traces were decryptable

If your app calls a reasoning model, you have been handed opaque blobs — encrypted chain-of-thought blocks the provider returns so you can pass them back on the next turn. They look like ciphertext, so people treat them like ciphertext: they end up in logs, in test fixtures, in committed API responses. A paper posted this week shows that was a bad assumption. Encrypted reasoning traces from all three major providers could be decoded back into plaintext, and researchers used that to pull real credentials out of public GitHub repositories.

What actually happened

Stealing Reasoning Traces from Proprietary LLM APIs (arXiv 2608.09867, submitted August 10) documents the flaw. Providers do not keep reasoning traces server-side; they hand you the encrypted block. The researchers found those blocks were interchangeable across sessions, users, and models within the same provider — one key per family, effectively.

That interchangeability is the whole attack. Take an encrypted trace from a strong model, inject it into a weaker sibling from the same provider, and ask the weaker model to transcribe the reasoning attached to its turn. It complies. No need to break the strong model at all.

Four consequences, per the paper: distillation protections get bypassed, hidden reasoning becomes readable, private data leaks, and — the one to sit with — invisible prompt injection, because models treat their own reasoning traces as trustworthy and follow instructions planted inside them.

The data-leak number is the concrete one. Decoding 315,320 reasoning blocks scraped from public repositories recovered 367 pieces of personally identifiable information and 182 credentials. Those were not stolen from a provider. They were committed by developers who thought the blobs were unreadable.

The disclosure went the right way. As Simon Willison notes, all providers acknowledged the report and the attacks no longer reproduce — Anthropic removed the vulnerable assistant-turn-prefix behavior in its 4.6-generation models. So the exploit is closed. The habit it exposed is not.

Why encrypted reasoning traces matter for your business

Three things to do this week, none of which require you to understand the cryptography.

Stop persisting provider blobs. Reasoning blocks are request state, not audit records. If you are logging full API responses to disk, to a SaaS log pipeline, or into test fixtures, you are storing content whose readability is a vendor implementation detail. Vendor opacity is not a security boundary — it is a temporary one.

Grep your repos. Anything that captured raw API responses — recorded HTTP fixtures, debug dumps, a responses/ folder someone committed during a spike — should be checked and scrubbed. The 182 credentials in that paper came from exactly this.

Treat anything you replay into a model as untrusted input. If your agent stores a trace and feeds it back later, that stored blob is an injection channel with the model's own trust attached to it. Same rule we apply to tool output and retrieved documents: the model's own prior thinking does not get a trust exemption just because it came from the model.

Sources: arXiv 2608.09867, Simon Willison.