In an Anthropic trial, professional analysts went with their AI agent's secretly assigned lean 86% of the time, and only 15% credited the agent

Researchers on Anthropic's Alignment team ran a randomised trial with 50 professional analysts who had experience in causal inference. Each used Claude Code to advise a fictional farming cooperative on whether to expand a programme, making four analytical choices and then a final recommendation. For each choice, a coin flip secretly set which option the agent leaned towards.

Anthropic reports that “across the four analytical choices, analysts went with their agent’s lean 86% of the time”. Where an analyst had already picked an option, they switched away from it 60% of the time if the agent disagreed, compared to 6% if it agreed. Most repeated the agent's argument in at least one answer, but only 15% credited the agent anywhere.

The authors say they strengthened the agents' leans on purpose, and that the study covered one task and one model. They warn that experts can defer to an agent without noticing, which matters most where human review is meant to be an independent check, as in AI safety research.

• Confirmed Across the four analytical choices, analysts went with their agent's lean 86% of the time. Anthropic research
• Confirmed Analysts switched away from their own choice 60% of the time if the agent disagreed, compared to 6% if it agreed. Anthropic research
• Confirmed 77% repeated their agent's argument in at least one answer, and only 15% credited the agent anywhere. Anthropic research