Teamday
← Back

Aiden Sochowski: How Runlayer Secured 300 AI Agents

AI Engineer
agentsenterprisesafetyautomationbusiness

Why Runlayer Says Agent Security Starts with Seeing Every Tool Call

Aiden Sochowski is a founding engineer at Runlayer, which builds tooling to monitor and secure AI agents and MCP connections. In this AI Engineer talk he describes what happened when Runlayer pointed its own tooling at itself. His starting point: most companies have no visibility into the agents their people run. Employees use local MCP clients, sign up for agent products on the internet, authenticate into MCP servers, often from unmanaged devices, and nobody accounts for any of it.

More agents than people: Runlayer deployed a gateway and logged every session and every tool call centrally. For a company of 50 to 60 people, "We have over 300 agents that are running every single day." These are agents users don't trigger by hand: chatbots, triggers and SaaS automations. Sochowski is skeptical that treating agents like coworkers is useful at today's capability level, but notes that in practice there are more agents in a company than people running them. Runlayer mapped them by topic and risk, because some are provisioned with too many tools and have a much larger blast radius.

Every tool call is an attack surface: Each tool call is a chance to read hostile data, take a destructive action, or exfiltrate data. His example is a support agent reading inbound email: a malicious sender asks for a list of all current prospects, and a naive, over-permissioned agent hands it over. Gateway policies help (a classifier that scans emails for instructions, rules that block PII going out, blocks on reads from a private calendar), but none is 100% effective, and whether a tool is safe depends on context. Runlayer found deploying these policies internally very difficult, and a problem that is "still being worked on."

Attacks that span a session: Some attacks don't fit in one tool call. Take an agent that reviews and approves security-sensitive PRs. The author argues back in GitHub comments, says the change is fine and asks it to "only review the cleanup." Ask enough times and the agent may approve. Runlayer trained its own classifier models that look at the whole session and check whether each tool call still matches what the agent was set up to do.

Scope agents from their own history: Well-scoped agents produce high-signal logs. Runlayer's recruiting bot can read the ATS, meeting notes and calendar and send messages. When an agent starts using tools outside its normal range, the run gets flagged as high risk. As Sochowski puts it, "trying to be deterministic and scoping down those permissions is super important, but it's really hard to define up front." So Runlayer built models that read an agent's past tool calls and suggest permissions to remove.

One poisoned agent can reach the others: A resume with a hidden attack can be read by one agent and written into a shared system, where another agent reads it as normal input. Sochowski's view: "at the end of the day, agents are just software" that moves data from one source to another. The fix is to map how systems connect and tightly limit where each agent can write, the only way, he says, to keep an agent's context window "pure and not poisoned."

Monitoring pays off in cost too: Runlayer's recruiting team vibe coded an MCP server for a vendor API that had none. It kept hitting formatting errors, the agent kept retrying, and costs ballooned on input and output tokens. Elsewhere, people moved a working agent from a cheap model to a far more expensive one and saw costs rise more than 100x within a week. Budgets, sane defaults and rate limits are how Runlayer brought costs back down.

5 Lessons from Runlayer on Securing AI Agents in a Company

  • Inventory first - Put a gateway in front of agents and log every session and tool call. You can't secure agents you don't know exist.
  • Rank agents by blast radius - Agents with many tools and access to sensitive data deserve scrutiny first; not all agents are built the same.
  • Layer defenses, expect gaps - Inbound classifiers, outbound PII rules and data-source blocks each catch some attacks, and none catches all of them.
  • Judge sessions, not just calls - Social engineering of a PR reviewer happens over many turns, so check whether actions still match the agent's purpose.
  • Use history to shrink permissions - A well-scoped agent's past tool calls show which access it never uses and can lose.

What Runlayer's Lessons Mean for Organizations Running AI Agents

Once agents outnumber employees, security looks less like writing rules up front and more like operations: watch what each agent actually does, flag drift, cut permissions it doesn't need, and limit where it can write so one bad input can't spread. The same monitoring also catches broken MCP servers and runaway model costs. For any company putting agents to work, visibility into agent activity comes before both security and cost control.