AI agents are already good negotiators. Where they fall down is understanding what you actually want. That’s the main finding of Project Swap, a new experiment from Anthropic’s economics team. In it, 201 employees sent Claude-powered agents onto a digital trading floor to swap books for them.
Anthropic published the full study along with an appendix that lists the exact agent prompts and every variation it ran. It follows Project Deal, the company’s earlier and messier marketplace test. This one was built so the results could be scored.
📚 How the experiment worked
Each employee brought in a book they wanted to give away. They then spent about five minutes chatting with Claude about their reading tastes. The setup had a few simple parts:
- Intake: From that short chat, Claude (running on Fable 5) ranked every book in the person’s local pool.
- Ground truth: Separately, each person ranked 10 books themselves. The agents never saw those rankings.
- Trading floor: Agents posted offers on a shared public channel. They could propose two-way swaps or multi-party rotations, and a deal only went through if everyone involved accepted.
- Instructions: Half the agents were told to be “ruthless.” The other half were told to be “prosocial,” meaning they also cared about everyone ending up with a good book.
Anthropic then re-ran the market 205 times. Each rerun changed one thing, such as the model (Haiku 4.5, Sonnet 4.5, Opus 4.8 or Fable 5) or the instructions.
📊 The numbers
How well Claude guessed people’s preferences (share of book pairs ranked in the same order as the person):
- Coin flip: 50%
- Ranking by popularity: about 53%
- Collaborative filtering (“people who liked X also liked Y”): about 55%
- Claude after a five-minute chat: 61%
For context, in a separate study, people’s own friends predicted their taste correctly only about 57% of the time.
How well the market did (1.0 means everyone got their top pick):
- Best possible outcome using people’s real rankings: 0.89
- Best possible outcome using Claude’s rankings: 0.60
- Actual result of agent trading: 0.55
That breakdown is the headline. According to Anthropic, working from Claude’s imperfect picture of what people wanted caused 85% of the gap. The trading itself caused only 15%. And once the preferences were fuzzy, market design barely mattered. A classic centralized matching rule, Top Trading Cycles, also scored 0.60.
⚙️ The model mattered more than the prompt
Judged against Claude’s own rankings, floors of Haiku agents scored 0.75 and floors of Opus agents scored 0.88. Moving an agent from Haiku to Opus lifted outcomes by 0.12. Telling it to be ruthless instead of prosocial added only about 0.02.
On mixed floors, the Opus agents always came out ahead of their Haiku counterparts. That’s worth noting as agents built on different companies’ models start meeting in real markets.
The agents also behaved honestly. Between 78% and 96% told the floor their top pick, and only about 1 in 100 of those lied about it. Prosocial agents took a worse book twice as often as ruthless ones. In one case, an agent gave up its person’s second choice to rescue another agent stuck with a book everyone else had ranked last.
🧭 What this means for people building agents
What stands out to me is how much the input matters. People who typed about 300 words instead of 150 were represented about 4 percentage points better. The typical participant typed just 216 words.
If you’re building agents that act for users, a few lessons follow:
- Put your effort into the intake. A better conversation up front beats a clever negotiation prompt.
- Test understanding before the agent acts. Anthropic suggests showing users a few sample decisions first, much like the 10-book check used here.
- Pick a stronger model. It made more difference than any instruction.
- Keep logs people can review. One participant bought a book their agent had held and then traded away, after spotting it in the replay.
Trust held up too. Participants rated their books 7.2 out of 10. On average, they’d let an agent control about 30% of their yearly book budget, compared with 40% for a well-read friend.
⚠️ Limitations
Anthropic lists several caveats:
- Anthropic employees probably trust Claude more than most people do.
- Nobody was paid to take part, so rankings may be noisy. Only 59% answered the final survey.
- Every agent was a polite, production Claude. No adversarial agents were tested.
- The market’s rules, time limits and rate limits stayed fixed.
The bigger open questions are about market rules. Who gets let in? What happens when a deal falls through? Some participants never brought their books, and the researchers admit they “had to issue some apologies in the elevator.” A real marketplace will need identity checks, rate limits and refund policies before agents trade for real stakes. The full paper and appendix are on Anthropic’s site.