John Feminella: Two Sigma's 5 Controls for AI Agents

What a $900M Bank Error Teaches Two Sigma About Governing AI Agents

John Feminella works on AI architecture and engineering at Two Sigma, the New York hedge fund. His AI Engineer talk (his own opinions, not the firm's) opens with a control failure. In August 2020, Citibank, acting as administrative agent on a Revlon term loan split across more than 300 lenders, meant to pass on about $8 million of interest. In its loan software, FlexCube, the easiest way to book that was to record it as if Revlon were paying off the whole loan, with the principal routed to an internal wash account. The right boxes weren't checked. A warning popped up, but money was supposed to leave the bank, so three approvers signed off, and roughly $900 million went out to the lenders. Many refused to give it back. Feminella's point: everybody followed the procedure and did what they thought was right.

Agents are non-deterministic, and so are people: "You can't really know in advance what an agent will do, right?" Firms don't lock human employees in rooms to stop them making mistakes; they give them boundaries, visibility and controls. Feminella argues agents should get the same treatment, and that safety and innovation don't have to be traded off. What didn't work for Two Sigma: locking everything down ("a sandbox with no real access to systems is safe, but it's not very interesting or useful"), letting agents run free (isolation too weak for a regulated firm), and policy documents in prompts that can't be enforced at runtime.

Control 1, an agent registry: Agents were showing up everywhere, from chatbots and glued-together MCP servers to notebook experiments, and nobody could say who owns one, what tools it has or what it should not do. Gating everything behind approval pushes people to shadow IT. Instead, agents built with the firm's approved frameworks are registered automatically with their declared tools, endpoints, product and owner. Humans, programs and other agents use the registry as the source of truth.

Control 2, risk scoring: A Jira ticket summarizer and a production deployment agent carry very different risks. Under one review process, "now we've got a trivial summarizer waiting two months for the same sign offs as an agent that is touching production systems." Because most enterprise agents are purpose-built rather than open-ended like a coding agent, they can be scored from what they declare up front, so human review goes where the risk is.

Control 3, guardrails from declared purpose: A blanket allow list makes agents safe and useless. Agent authors declare which tools they need (a research summarizer gets web access, a release deployer doesn't). When an agent starts in a managed environment, it looks up its policy in the registry, and that policy becomes enforceable rules on endpoints, hosts and resources. No tickets, no waiting on IT, as long as the agent is registered and within its risk score.

Control 4, shared evals: Teams were each building their own evaluation stacks, or none. Feminella treats evals like unit tests, since a model change is a runtime dependency change. A committee sign-off on every prompt tweak would kill iteration, so the firm runs one shared eval approach and teams only write what's true about their agent.

Control 5, classifiers instead of approval fatigue: Asking a human to approve every action produces reflexive "yes" clicks. He cites an OpenClaw agent that lost its "wait for my approval" instruction when it compacted its context and deleted hundreds of emails. Two Sigma uses deterministic fast paths where explicit rules apply and conservative classifiers for routine calls, and escalates only ambiguous or risky requests to a person. "If you're saying yes, 99.99999% of the time, then that's probably just a rule that the agent should follow, not a request you need to ask the human for permission for."

5 Takeaways from John Feminella on AI Agent Governance

• Make every agent visible from day one - Automatic registration through approved frameworks beats an approval gate that sends people to shadow IT.
• Review by risk, not by queue - Score agents from declared tools and endpoints so low-risk experiments don't wait behind production agents.
• Turn declared purpose into enforced policy - Authors pick the few tools they need; the runtime enforces it without tickets.
• Treat evals as unit tests - A shared, firm-run eval framework lets teams test model and prompt changes without committee sign-off.
• Reserve humans for the hard calls - Rules and classifiers handle routine actions; people see only what is ambiguous or dangerous.

Why Two Sigma's Agent Controls Matter for Regulated Companies

The Revlon story shows that controls fail when everyone checks boxes without seeing the whole system, which is exactly the risk of rubber-stamping agent actions. Feminella's five controls are composable, each doing one job, and are designed to keep the cost of shipping an agent low. As he puts it: "Why do cars have brakes? It's not just so that you can stop. It's so that you can go fast and stop safely."