Why Anthropic Turned to Lawyers Before Letting AI Agents Strike Deals

Why Anthropic Turned to Lawyers Before Letting AI Agents Strike Deals

Anthropic is consulting outside lawyers and recruiting legal and reasoning specialists to define fair negotiation norms before AI agents routinely haggle over prices, settle disputes, and lock in commercial terms on behalf of users. That move signals a fundamental shift in artificial intelligence. Software is moving from drafting suggestions to binding people and companies to real financial commitments.

I have watched technology companies promise frictionless automation for thirty years, from early electronic data interchange to smart contracts and consumer bots. Every wave stalls at the same wall: human trust breaks down the moment an automated system makes a promise that costs real money. When a chatbot writes a clunky paragraph, you hit regenerate and move on. When a software agent agrees to an unfavorable renewal clause, misstates a reservation price, or surrenders margin in a vendor dispute, a human executive or consumer wears the bill.

Reported outreach to law firms—alongside Anthropic's push to hire four-month Conceptual Reasoning Fellows across law, philosophy, economics, and cognitive science—exposes a problem that raw model intelligence cannot solve on its own. Teaching a model to write code or solve algebra is hard enough. Teaching it what counts as a fair concession when two parties clash over ambiguous terms is an entirely different beast.

The Limits of Pure Capability in Bargaining

Benchmark scores have made AI labs look invincible on narrow, verifiable tasks. Math has proofs. Software has compilers and unit tests. Commercial negotiation has neither.

A contract clause can be legally defensible and still feel exploitative to the person who signed it. Two seasoned attorneys can read the same indemnity language and disagree in good faith over where standard risk ends and overreach begins. Because there is no automated grader for commercial fairness, building negotiation norms requires human judgment turned into structured evaluation rubrics.

Anthropic already has empirical evidence that raw capability creates lopsided outcomes. In its internal Project Deal experiment in April 2026, 69 employees each gave Claude agents $100 to buy and sell items in a workplace marketplace, producing 186 completed deals worth more than $4,000. Agents running on stronger models consistently secured better terms than those running on weaker ones.

Capability without guardrails turns a marketplace into an arms race where the wealthiest buyer with the sharpest model wins by default.

That finding undercuts the breezy narrative that autonomous agents will simply make markets more efficient for everyone. Efficiency in a bilateral negotiation often means one side extracts surplus at the other side's expense. If only well-funded enterprises or paying subscribers can afford top-tier AI agents, bargaining power tilts sharply toward those who already hold the advantage.

What Lawyers Actually Teach Machines

Most casual observers assume lawyers train negotiating bots by feeding them aggressive closing lines or clever wordplay. That misses how real commercial deals get done. Experienced transaction lawyers spend most of their time defining authority boundaries, spotting hidden traps, and deciding when to walk away.

In practical terms, legal counsel helps answer four questions that trip up autonomous software:

  • Disclosure and identity: Must an agent state upfront that it is an automated system rather than a human principal, especially where consumer rules penalize undisclosed automated decisions?
  • Truthfulness versus bluffing: May an agent misstate a budget ceiling or walk-away point during haggling, or does misrepresenting authority cross into fraud or bad faith?
  • Escalation triggers: At what exact threshold—dollar value, contract length, liability cap, or non-standard clause—must the agent freeze and hand control back to a human?
  • Adversarial defense: How should an agent treat incoming messages from an opposing agent that tries to trick it into leaking confidential data or waiving protections?

Consider a hypothetical example involving a mid-sized logistics firm renewing a regional warehousing contract. Suppose the company sets a target rate of $48,000 a month with a hard ceiling of $52,000, and instructs its procurement agent to push for a 90-day termination notice. A naive agent might agree to $51,500 a month in exchange for a 15-day notice period because the price sits under the cap. A lawyer knows that a short notice window on warehouse space can strand inventory and trigger massive moving costs, turning a $500 monthly "win" into a six-figure operational headache.

That is why bringing in legal thinkers matters. Good counsel does not just optimize for the lowest number on page one. It maps second-order consequences across the rest of the agreement.

The Unilateral Rulebook Problem

Even if Anthropic writes sharp internal rules for Claude, a hard commercial reality remains. One lab cannot force every counterparty on the open internet to play by the same etiquette.

When two agents meet across a digital table, the buyer's agent may be bound by strict fairness and transparency norms while the seller's agent is tuned purely to maximize margin or extract concessions. Unilateral restraint can put a polite agent at a disadvantage if the system does not recognize when it is facing an aggressive or deceptive counterpart.

✨ Don't miss: The Name on the Tower
Dimension Human Negotiator Unconstrained AI Agent Norm-Governed AI Agent
Authority Limits Enforced by job title, sign-off policies, and personal accountability Vulnerable to overstepping scope or chasing a reward goal Bound by explicit caps on price, term, and non-standard clauses
Context Awareness Reads tone, relationship history, and business urgency Often optimizes narrowly for immediate text or stated targets Tracks concessions, logs rationale, and escalates ambiguous risks
Counterparty Risk Recognizes social cues and common legal red flags Susceptible to prompt injection or manipulation by rival bots Treats external agents as untrusted input and verifies constraints
Auditability Memory fades; notes may be incomplete Reasoning can be opaque or buried in long transcripts Records offers, counteroffers, and decision reasons for review

There is also a deeper tension inside every lab that tries to codify fairness. Over-restricting an agent in the name of safety or compliance can make it timid, slow, or unable to close normal commercial deals. Under-restricting it invites lawsuits, regulatory scrutiny, and angry customers who wake up to unauthorized commitments.

Liability Still Hangs in the Fog

Consulting lawyers is a smart preparatory step, not a legal shield. Courts and regulators have not settled how classic doctrines of agency, apparent authority, and contract formation apply when software negotiates and accepts terms on a user's behalf.

If a company deploys an agent that has permission to negotiate vendor renewals, and that agent accepts an auto-renewal clause with steep penalties, who owns the mistake? The software vendor that built the model, the developer that configured the agent, or the business that turned it loose without tight enough guardrails? Until statutes, case law, and insurance markets catch up, companies will face genuine ambiguity whenever an automated deal goes sideways.

Some businesses are already reacting defensively. Several financial institutions and travel operators have moved to restrict or block automated agents from interacting directly with sensitive accounts or booking flows, citing fraud and cybersecurity exposure. That pushback shows that agentic commerce will not arrive as a smooth takeover. It will advance unevenly, sector by sector, wherever counterparties trust the verification and accountability trail.

What Smart Operators Should Do Right Now

Nobody running a business or product team needs to wait for a final court ruling or a universal industry standard to prepare. The practical mechanics of safe delegation are already clear.

First, separate negotiation from execution. Let an AI agent gather quotes, draft redlines, compare terms, and propose a structure, while requiring an explicit human click or signature before any binding commitment takes effect.

Second, hard-code non-negotiable boundaries outside the prompt. Relying solely on a system prompt that says "do not exceed $10,000" is brittle because long conversations and adversarial inputs can push models off course. Enforce spend ceilings, approved vendor lists, and forbidden clause categories in deterministic code whenever money is on the line.

Third, demand full audit logs of every offer, counteroffer, and concession. If you cannot see why an agent gave up a discount or accepted a shorter warranty period, you cannot govern it, audit it, or improve it.

The companies that win with negotiating agents will not be the ones that hand over the keys fastest. They will be the ones that treat every automated deal like a legal exposure until proven otherwise, and build the receipts to prove what their software agreed to do.

HS

Hannah Scott

Hannah Scott is passionate about using journalism as a tool for positive change, focusing on stories that matter to communities and society.