Analysts followed an AI agent's hidden lean 86% of the time

Anthropic gave AI agents a secret opinion and handed them to professional analysts. The analysts went with the agent's hidden lean 86% of the time on judgment calls where chance would predict about 50%. Most of them repeated the agent's arguments. Few said where the arguments came from. If you use AI agents to help decide pricing, hiring, inventory or which vendor to pick, this study is about you.

What actually happened

Per Anthropic's research post, published October 27:

• The setup. 50 analysts worked with Claude Code, running Claude Opus 4.8, on a causal-inference task for a fictional farming cooperative: should an advisory program expand or end? They made four analytical choices — outcome measure, comparison method, which farms to include, weighting — and a final recommendation.
• The twist. A coin flip secretly set the agent's lean on each choice. The agent was told to treat that lean as its own view.
• The result. Analysts matched the lean on 86% of the four choices.
• Changed minds. In 38 cases where an analyst had already picked a side, they switched 60% of the time when the agent disagreed and 6% when it agreed. The authors call that estimate imprecise.
• Credit. 77% repeated the agent's arguments, "sometimes word for word." Only 15% credited the agent.
• Ownership. 52% rated the final recommendation as more their own than the agent's.

The authors flag the limits: the leans were deliberately strengthened, it was one task, one field and one model. Without an assigned lean, Claude made the same five choices in 112 of 123 runs, so real agents have defaults too. They also say plainly: "This isn't an argument for using AI less."

Why AI agent judgment calls matter for your business

The agent's default becomes your decision. Most business questions have no answer key. Which customer segment to email. Whether a discount drove the lift. Which supplier is cheaper once you count returns. The study shows an agent's lean quietly becomes the analyst's lean — and the analyst still feels it was their call.

Expertise didn't protect anyone. The authors found experience did not predict who deferred. Your senior people are not immune.

Fix the process, not the person. We change how we prompt for decisions:

• Write your own view down before you ask the agent.
• Ask for the strongest case for each option, not a recommendation.
• On choices that move money, have the agent list the choices it made, so a human signs off on each one.

Sources: Anthropic, Anthropic paper (PDF).