Teamday
← Back

Modal's Akshat Bubna: Don't Vibe Code Agent Infrastructure

AI Engineer
agentsenterpriseautomationstartupbusiness

What Agents Need From Infrastructure, According to Modal's CTO

Akshat Bubna is co-founder and CTO of Modal, the compute platform that runs sandboxes, inference and batch jobs for AI companies. His AI Engineer talk opens with a customer example: Hebbia's product lets a financial analyst run a query across a portfolio where "every cell is backed by a different agent." Each agent runs in a Modal sandbox, and the architecture separates the harness, the loop that calls the model and runs tools, from the containers where tool calls execute. Agents in the same row share a volume so they can collaborate, and the whole thing has to spin up thousands of agents in seconds and tear them down afterwards.

Workspaces, then persistence: Bubna lists what agents need from the ground up. A coding agent such as Ramp Inspect, Ramp's internal background agent, which he says pushes almost every pull request at the company, needs the repo, dependencies and dev services in one workspace. Basis, the accounting agent, needs documents, work papers and scripts. A game studio needs its engine and assets. Then the workspace has to outlive the task. With Basis, documents arrive, the agent works, then waits for an accountant who may not reply for days. "So you want to persist the state over time, potentially indefinitely. But you don't want to run the compute the entire time because compute is expensive." Modal offers several flavors: snapshot the root filesystem, snapshot one directory such as node_modules while upgrading the base image, or a shared volume.

Agents consume compute unevenly: Ramp let Modal share Inspect's resource data. CPU and memory vary widely between sandboxes and over time within one sandbox, because compiling or a heavy analysis spikes usage. Provisioning for peak would mean about 15 CPU cores and 15 GB of RAM per sandbox, which he calls economically senseless. Modal's work is letting sandboxes grab resources when needed and pay only for that duration. Agents also increasingly want GPUs: Claude Science ships with a Modal integration that offloads data preprocessing and scientific models to sandboxes, and Modal's own experiments show agents given GPU sandboxes to train models will swap GPUs and run ablations in parallel.

Collaboration, memory and containment: Sub-agents and humans share filesystems, so the filesystem needs speed and consistency. Agents save and update memory as files and run background tasks to refresh it. And because agents "don't really know when they might leak your sensitive data to a German forum or something." Teams need control over which IPs can reach the agent, which domains it can call, and credentials injected so the agent never sees them. Domain allow lists turned out to be insufficient: users want path-level rules for GitHub or Datadog, so a pattern has emerged where customers write a lightweight proxy sidecar with custom rules. Ramp does this. Observability closes the loop: when a training run fails, the agent itself should read the logs, metrics, GPU utilization and budget and iterate.

Why not build it yourself: Bubna's answer to "have an agent build us a sandbox platform" is no. "I don't think you should Vibe code a sandbox platform." Reliability is hard because tail failures are hard to reason about even with AI. Security needs someone continually defending the isolation boundary. Scale is the third reason: Modal sees 20 million snapshots a week, about eight petabytes of snapshot data a day, and 2.6 million new volumes a week. One personal-assistant customer needs 10,000 sandboxes created per second, close to a million running at once, each living two or three minutes and interactive in under a second. No single cloud supplies that, so Modal aggregates CPU capacity across hyperscalers and neoclouds.

Brains and hands: The direction he sees is "separation of the brains, which is the harness, from the hands, which is the sandbox or the runtime where stuff is running." Harnesses spin up environments on demand, snapshot them as artifacts and come back later. He frames it as the next step after pets and cattle: most teams deploy agents as homogeneous cattle today, while models are already good at reasoning about many dynamic environments at once.

5 Takeaways From Akshat Bubna on Agent Sandboxes

  • Separate harness from runtime - The model loop and the place tool calls execute should be different containers for isolation.
  • Persist state, not compute - Snapshot the workspace while an agent waits days for a human, and resume on the reply.
  • Size for variance, not peak - Ramp's data shows per-sandbox usage swings too much for fixed provisioning.
  • Allow lists by path, via a proxy - Domain-level egress rules are not enough; customers write sidecar proxies with operation-level rules.
  • Expose observability to the agent - Logs, metrics and budget are inputs for the agent's next iteration, not just dashboards for humans.

What This Means for Companies Running Agents at Work

The operational pattern from Basis is the one most businesses will hit first: an agent that works for minutes, waits for a person for days, and must resume exactly where it stopped without paying for idle compute. That is a sandbox and harness problem before it is a model problem. It is also why Basis founder Mitchell Troyanovsky argues that AI makes individuals fast but companies need new structure, and why Ramp pairs the infrastructure described here with its own model work. Bubna's closing claim is the thesis of the talk: "great primitives let your agents do more."