Teamday
← Back

Kill My SaaS: Replacing a $43K Sessionboard with AI Agents

Latent Space
agentsbusinessenterprisefuture-of-workclaudegpt

Kill My SaaS: How swyx Tested Replacing Sessionboard with AI-Built Software

swyx runs two businesses: Latent Space and AI Engineer, which organizes conferences for the AI industry. AI Engineer was shopping for speaker management software like Sessionboard, and the vendor quoted $43,000 a year for its four events. Instead of joining the "AI killed SaaS" argument on social media, swyx put real money on it. The Kill My SaaS challenge asked anyone to rebuild the tool with coding agents, and the judges were his non-technical events team, who only cared whether they would actually use it.

Build vs buy, reversed: For years swyx had a rule in his company's founding contract that "nobody could build internal software without my explicit approval because it is so easy to get not invented here syndrome". This year he changed his mind. Part of the reason is that SaaS is a bundle: "We use maybe a third of what this particular vendor was selling us and there's no discount for the stuff that we don't use." Feature requests to vendors often take three to six months, or never ship.

The challenge setup: Builders got a written brief, a walkthrough video of the real product and 42 screenshots. The winner got $10,000, and serious attempts could claim up to $500 in token reimbursement. 72 teams submitted, and most of the apps are still live and open source. swyx picked event software on purpose as an example of mid-tier business software. It isn't Salesforce, but it has to handle speakers, reviewers, sponsors, invited speakers and attendees, with one person often holding several roles. Event people are risk averse because a week's slip means people show up in the wrong place.

Where the AI-built apps broke: Three things tripped up most entries: separate portals for speakers, invited speakers and sponsors; drag-and-drop scheduling that doesn't double-book a room or a speaker; and adding an invited talk without faking a CFP submission. None of these were well specified in the docs, but anyone who used the product like a real person would find them right away.

AI judge vs human judges: Latent Space published an AI judge in which Claude logged in as each role, clicked through 20 scenarios and graded the results. 56 apps ran through it, 40 had enough coverage to rank, and the top seven landed within 2.3 points of each other. The human ratings from the events team, led by head of ops Kelsey, correlated poorly with it. Six of the AI judge's top 10 were rated weak, and the eventual winner placed 19th on the AI leaderboard. One builder said most of what they were proud of wasn't covered by the AI judge at all.

How the builders used Claude Code and Codex: The median public repo was 91,000 lines of code, and people ran agents for days. Claude Code and Codex were basically tied, and 40 of the 66 submissions studied used both. swyx saw three patterns: a smart model plans while a cheaper one writes the code, one model builds while another checks (usually whichever drives a browser better), and no model approves its own work. One team ran a Claude team and a Codex team in separate worktrees, and neither could merge until the other verified it. As swyx puts it, "Agents tend to be easier on their own work than they are on other people's work."

What the finalists did first: All three did a lot of work before writing a prompt. Marko Kraemer (TrackStage) spent eight or nine hours planning the data schema, business logic and pseudo code, then fed the walkthrough video and Sessionboard's API docs to Claude with Convex as the database. Yazin (Conferencer) crawled Sessionboard's entire help site with Firecrawl before writing a spec. Valentin De Matos (OpenRostrum) built guardrails into a reusable GitHub template, had Codex map every flow of the app, then planned which features agents could build in parallel. All three then used their apps as real users. Valentin's rule was a five-minute test: if he couldn't get through the product smoothly in five minutes, something was missing from the agents' context.

5 Lessons from Kill My SaaS on Replacing SaaS with AI Agents

  • Cloning is the easy part - Most of the 72 entries were decent first-shot clones. Someone still has to take bug reports and fix the app at 2 a.m.
  • The real price is about $11,000 plus ownership - swyx estimates about $1,000 in tokens and maybe $10,000 in AI engineering expertise to clone a mid-tier tool. After that you own it, and you have to maintain it.
  • Automated evals bootstrap, humans decide - An AI judge with recorded user flows helps builders hill-climb, but it can't measure taste or whether the parts work together.
  • Spec work beats prompting - The winners researched, planned and mapped every flow before prompting. Nobody typed one prompt and walked away.
  • Dogfood every flow - Agents built features that didn't connect, like a headshot upload that the public speaker page never read from. Only clicking through the app as a user caught it.

The Results and What Build vs Buy Means Now

Five apps cleared the bar for something AI Engineer would seriously consider adopting, and 12 more had ideas worth stealing. Marko Kraemer won $1,000 for the cleanest drag-and-drop scheduling, Yazin won $5,000 for Conferencer's click-through experience and transactional emails, and Valentin De Matos won $10,000 for OpenRostrum, which he built with a software factory he debugged along the way. AI Engineer's 2027 conferences will probably run on a mix of OpenRostrum and Conferencer.

Kelsey's main concern was accountability. When a sponsor hit a bug, the vendor's answer was to clear the browser cache, and she said "I can't ask someone who's like, you know, at a multimillion dollar company to refresh their cache because something we did broke." Owning the software makes the team responsible for fixes, but it also gives them fast turnaround. swyx says the people coding most on his team are not engineers. They are designers, domain experts and operations people, and they get minute-level turnaround on their bug reports. For organizations renting software they don't love, his conclusion is that building your own is now a reasonable choice, as long as you plan to maintain it, ideally with agents.