Persimmon simulates users so you can test the agent

humans& introduced Persimmon on September 10 — a 550-billion-parameter model whose entire job is to act like a person on the other end of a chat. Not a better assistant. A better user. If you have ever shipped a support agent and found out how it fails from a customer, this is the missing half of your test suite: a user simulator you can point at your own bot before anyone real does.

What actually happened

Per the humans& writeup, Persimmon was initialized from NVIDIA's 550B Nemotron 3 Ultra base, midtrained on public internet conversations, then reinforcement-learned against an adaptive discriminator to hold coherence over long exchanges. It simulates multi-turn, multi-user chat — the goal is to reproduce the spread of human behavior, not one tidy average human.

The numbers that matter:

• 19.8% mean judge-fooled rate on a multi-user Turing test across three datasets. Frontier assistants score under 3% on the same task.
• 88.49% precision, 77.03% recall on the Trickle Test, which measures whether a simulated user drips information out the way people do instead of dumping their whole context in turn one.
• 60.7% coherence through 80 turns, against 87.3% for actual humans. Long conversations still drift.

Access is a gated research and evaluation preview — playground and API at persimmon.humansand.ai. No published pricing.

Why a user simulator matters for your business

Every agent eval we see tests the agent. Almost none test the conversation. Real customers are terrible test cases in the useful sense: they withhold the order number, change the question halfway through, answer a clarifying question with a different question, and get annoyed. Your golden-dataset eval has none of that, which is why an agent at 95% on your suite still gets escalated three times a day.

That Trickle Test precision is the number to sit with. The failure mode of scripted testing is a user who is too helpful. If your test harness always supplies the account ID on request, you never find out that your agent loops forever when someone says "I don't have it in front of me." A simulator that withholds realistically is how you find that in staging.

Two honest limits. It is a research preview with no pricing, so this is a lab tool this quarter, not a CI dependency. And 60.7% coherence at 80 turns means the simulator itself gets confused on long runs — which is fine for stress-testing a ten-turn support flow and not fine as ground truth for a multi-session agent.

What we would actually do: take your twenty worst real transcripts, describe those users to a simulator, and run your agent against a hundred variations of each. You are not looking for a score. You are looking for the three prompts that make it hang.

Sources: humans&: Introducing Persimmon.