Teamday
← Back

Ramp Labs: How a Small Qwen Model Beat Opus on Finance

AI Engineer
agentsenterprisebusinessknowledge-workresearch

Why Ramp Runs Applied Research Instead of Waiting for Better Models

Alexander Shevchenko leads Ramp Labs, the applied research group inside Ramp, the corporate card and finance automation company. He joined Ramp's applied AI team about three years ago. His AI Engineer talk answers a question application companies hear constantly: if frontier models keep getting cheaper, faster and better, why do any research yourself? Ramp is not short of LLM work to optimize. "Over the last year, the number of use cases that use LLMs has increased 123%." Token spend on top features keeps growing, yet each feature gets cheaper per unit as new models land.

Ramp shipped agents in 2023 anyway: Shevchenko compares today to building on GPT-4, when models didn't backtrack, had no thinking tokens, and a synonym in a prompt could change an agent's behavior. Ramp shipped agents such as Ramp Tour Guide anyway rather than wait for the next model. In 2026 an agent with MCP access and broad context works well out of the box, and he argues that going one layer deeper still produces a better product, it just takes more effort.

A specialist model under a frontier orchestrator: Ramp Sheets, the spreadsheet agent Ramp Labs launched in November 2025, was slow on its hardest job: given a stack of spreadsheets and a request like building a DCF, Opus 4.6 spent a long time planning and reading every sheet, and its thinking tokens were expensive. Ramp kept the frontier model as the orchestrator and handed the retrieval and financial question answering to a smaller model. With Prime Intellect, it post-trained a Qwen 3.5 35B model that beat Opus 4.6, the frontier at the time, on exact task accuracy while serving at small-model latency. That model went to production. Ramp Labs has also published work on transferring a post-train, such as a LoRA, to newer Qwen versions or other model families, which spreads the cost of post-training over several model generations.

Context compiled overnight with cartridges: The second line of research is KV cache techniques, in particular cartridges, which compress 20 to 30 times more context into the KV cache offline. Finance work is predictable: month-end close happens on the first of every month, and the ledger, aging report and documents it needs are known in advance. "With cartridges, we're able to compress all of that work and build one overnight before the financial professional starts their work on the first of the month." Shevchenko expects labs may ship something similar in their APIs, and says Ramp would welcome it because it would already know how to use it.

10,000 agents pen testing Ramp: The third case is agentic map-reduce. Ramp Labs published how it "used 10,000 parallel agents to pen test all of Ramp", built around Ramp Inspect, its cloud agent. The swarm maps the codebase, files findings, deduplicates them with LLMs and tries to reproduce each one adversarially before a human sees it. It found seven highest-severity vulnerabilities, all patched. One was an injection sent over email where "no single component of that stack was unsafe": only an end-to-end view across components exposed it. Ramp also found that open-weight models up to five times cheaper than frontier reasoning models lose much less than five times the detection rate in a swarm, so more cheap tokens nearly match fewer expensive ones.

5 Takeaways from Ramp Labs on Applied AI Research

  • Specialists can sit under frontier orchestrators - Ramp kept Opus as the planner and gave retrieval and financial Q&A to a post-trained Qwen 3.5 35B that was more accurate on the task and much faster.
  • Predictable work can be precomputed - Month-end close context is known in advance, so cartridges build it overnight instead of the agent rediscovering it each session.
  • Swarms change the model math - In Ramp's pen testing, cheaper open-weight models in a swarm came close to frontier detection rates at a fraction of the cost.
  • Verification belongs inside the swarm - Findings are deduplicated and adversarially reproduced by agents, so only real vulnerabilities reach people.
  • Research turns into products - Ramp's token spend management and Ramp Router both grew out of Ramp Labs work on cutting latency and token cost.

What Ramp Labs Means for Companies Building Finance Agents

Shevchenko's pattern is that every project saves time or money, which he notes is also Ramp's slogan. For an application company, the lesson is to stop treating frontier progress as a reason to wait and start treating it as the orchestrator layer, with fine-tuned specialists and precomputed context underneath. For other views of finance agents in production, see Basis on opinionated accounting agents and Datasite on buying document parsing.