AlixPartners: Claude Code Looked Right but Missed $800K

How AlixPartners Caught Claude Code's $800K Contract Mistake

Kevin Madura and Natalia Connolly work on the AI engineering and AI and data teams at AlixPartners, the management consulting firm known for bankruptcy, restructuring and risk work. Their AI Engineer talk walks through a real engagement, replayed with synthetic data: a large healthcare company acquires a smaller specialist, and the procurement team inherits hundreds of vendors, more than 40,000 contract documents and databases that were never linked. The question the client wants answered is simple: now that we have merged, can we get better terms from a shared vendor, how much would we save, and how fast do we have to act?

Answers that looked complete: Pointed at the data, Claude Code returned well-formatted, plausible answers. The team then checked every one against the warehouse. Annual spend, the split between the two companies and contract counts were correct. Total contract value was off by about $800,000, and the deadline to stop the soonest-expiring contract from auto-renewing was wrong: "The agent said December 1st. The actual date was November 1st, right?" Waiting for the agent's date would have meant an automatic renewal.

Why the data fooled the agent: The $800,000 gap came from an orphan amendment that was never linked to its parent contract, so the agent never found it. Other traps are just as ordinary: long contracts that time out silently in preprocessing and never reach the database, rate tables that break across pages and lose their headers, and amendments that restate rather than add to the original value. Summing the extracted dollar figures gives roughly $17 million, but "the text says that every amendment restates the total contract value. So the correct answer is seven, which is the most recent amendment's value." The renewal date had a similar root cause: the agent read the termination clause when the question required the date calculated from the renewal clause.

Two weeks of pipeline, one night of processing: The fix was to structure the data before letting the agent loose. Connolly and Madura spent about two weeks on a five-stage pipeline: document intelligence and OCR, classification (master service agreement, amendment, NDA), extraction of terms, rates and spend, mapping services to spend categories, and only then analysis by Claude Code. Chart-heavy pages went through as images with DSPy for structured output. Then "40,000-plus contracts were processed overnight once it was done." The structured tables also let the team check every answer programmatically.

Business logic lives in tools, not prompts: On top of the structured data, Claude Code built its own deal-specific tools, such as fetching a rate card page or a vendor's spend, with known gotchas embedded in how they query. Those were packaged into custom MCP servers so later agents start with the accumulated knowledge instead of rediscovering it. Access is governed the simple way: "You just have a defined set of operations that you're comfortable with, and you block everything else." One lesson from the two merged entities: when an access-denied error came back as an empty or zero value, the agent treated it as real data, so tools now return explicit feedback about what failed.

Parallel agents, deterministic reconciliation: AlixPartners dispatched one agent per savings lever from the firm's library of analyses, such as shared vendors or different vendors for the same service. The results were good, but they overlap, so a reconciliation step built around DSPy deduplicates findings with a prescriptive workflow to avoid counting the same saving twice.

5 Lessons From AlixPartners on Claude Code for Contract Analysis

• Check every answer against the source - Plausible formatting hid two errors with real money attached: an $800K contract value gap and a missed auto-renewal deadline
• Structure the data first - Two weeks of pipeline work made 40,000+ contracts analyzable overnight and made answers verifiable in code
• Expect messy contracts - Orphan amendments, restated values, silent preprocessing timeouts and headerless table pages are normal, not exotic
• Encode business rules in tools - Custom tools and MCP servers carry the deal's gotchas forward better than prompt text
• Give the agent business context - Recommendations got shot down because management had already started acting on them months earlier

What the AlixPartners Case Means for AI Agents in Consulting

The failure was not the model's reasoning but what it could see: an unlinked amendment and the wrong clause. Madura's conclusion is that "context is still king", and that spending tokens and effort up front on structure and business context is cheaper than an agent confidently answering from an incomplete picture. For any firm handing agents financial analysis, the operational rule is the one AlixPartners followed: trust the answer only after the code can check it.