Maersk's Dmitry Buykin: Agents Made Entropy Cheap

How Dmitry Buykin Keeps an Agent-Built System Maintainable at Maersk

Dmitry Buykin builds AI agent systems for shipping operations at Maersk, where his speaker bio lists him as Applied AI Lead. His talk is about what happens after coding agents have written most of a large system: "Coding agents made code cheap. They made entropy even cheaper." On one project he found what his agents had left behind, deleted tests and unused boilerplate, rewrote the whole thing and lost useful pieces. The rewrite didn't hold, because the agents kept writing and the mess grew back. His alternative to "drown in chaos or break what people use" is to stay under a line he defines as more change than anyone can check, with small weekly sweeps and feedback loops built into the system.

Most of the system is prose: The system he describes runs in 110 countries with about 10,000 scenarios of prose instructions, each task landing in one process in one country. Two kinds of agents meet there: coding agents write the instructions, and production agents follow them. By his estimate about three quarters of the system is prose, so prose needs engineering. The team borrowed mutation testing, breaking one sentence on purpose and watching for a red test. Almost none turned red. For code, tests can say what has to survive a rewrite. For prose, only the business experts can, so experts review each task exactly as the agent saw it, and their note becomes the next change, made by a coding agent and tested on the same task.

Prose and code drift apart silently: His running example is a request to change the amount on an approved order. The prose says the limit is 1,500 and the code says 1,000, so the same order passes in one place and fails in the other, and nothing throws an error. Instructions also leak: "Even a line like do not call x teaches the model that x exists." The team built a compiler for the prose and watches "walks", the path an agent takes through it. When agents keep walking the same path and it agrees across countries, the path becomes a workflow tool and the prose that described it is deleted. The first workflow tools made things worse, because the prose still named the steps inside them and agents ran those steps again. Now code prepares the facts and carries out the decision, experts own a decision table, the model only picks a row, and a person takes over if no row fits. The boundary moves both ways: hundreds of lines of expressions went from code back into prose because experts couldn't read them.

Control starts with what the model sees: When the model chose what to load, it loaded everything, then invented a pricing check because prices were visible. "Whatever sits in its context becomes a reason to act." The runtime now decides what the model sees. For the order task that is three facts, while the full record stays with the runtime. For large data sets the model passes a handle, a short name for data the runtime holds and checks before any write. Writes go through several checks in sequence, another check confirms the write landed, and a journal makes sure a retried write runs only once.

The architect as bottleneck: Buykin read every change at first, then volume outgrew one reader and he was burning out. His first gauge is himself: when he is tired and can't explain what changed, he stops and fixes the process, even if the dashboard is green. He built instruments that watch how things change. Before adding one new check, the team replayed its rule over several thousand past sessions and found it would have fired on fewer than 1%. A single signal waits until it recurs, even when it comes from an expert, while a cluster across countries gets one fix to the concept behind it. Ten to twenty of these loops are open at any time, each writing to a ledger. He had become the message bus between users and agents, so every user now has a screen to give feedback without him.

Verification speed is the limit: His agents spent almost nine tenths of the time on a change checking it. The team ranks evidence by what it proves, with a replay on real tasks at the top. They write down what would prove them wrong before the data arrives, and every new check has to fail once on purpose before it is trusted. AI refactors fail quietly: in one, the branch for amounts over the limit was lost and the order would have gone through without the team lead. A second agent checks the first with a checklist and without the author's reasoning, and over six months it took on most of his checking. Agents also finish most of a batch and quietly skip the rest when context is full; in one backlog most items looked covered, but only about a quarter could really be closed.

Weekly retirement, with evidence: Anything no real task reaches gets a name and its evidence before it is removed, and a retired variant takes its test with it, since tests were keeping dead variants alive. Counts are ratcheted so they can only go down. He hasn't had to do another big rewrite.

6 Takeaways From Dmitry Buykin on Maintaining Agent-Written Systems

• Treat instructions as code - Prose that agents execute needs linting, testing and a single copy, because a second copy drifts
• Promote repeated paths to tools - When agents take the same route every time, make it a workflow and delete the prose that described it
• Let the model pick a row - Experts own the decision table, code executes, and a person takes the cases no row covers
• Show the model less - Context the task doesn't need turns into actions nobody asked for
• Replay before you trust - New checks and refactors run against a frozen set of real tasks, and a branch no task takes counts as no evidence
• Don't let the author judge - A second agent with a checklist and none of the author's reasoning can't be talked into agreeing

What Maersk's Experience Means for Companies Running Agents at Scale

Buykin's account is a warning about where the cost of agent-built systems lands. Writing is nearly free. Knowing what is true, what is still used and what a change broke is where the hours go, for the agents and for the people supervising them. His last observation is about ownership: "Conway said the system copies the org chart. Ours copies whoever corrects it." The skills and memory of his system carry his own habits and biases, which is why he wants many expert evaluators giving feedback and not one architect. Robinhood made a related split between scripts, agents and people in its data pipeline PR reviewer.