Steven Latré
Head of AI at imec
About Steven Latré
Steven Latré leads the AI department at imec, the Belgian non-profit research center known for advanced semiconductor R&D. His group connects AI workloads to the infrastructure that runs them, from models and harnesses to hardware choices like the CPU-to-GPU balance and KV cache optimization.
In his AI Engineer talk, Latré presents an imec study of thousands of agentic coding runs. It compares API usage with hosting open-weight models on rented or owned GPUs, measuring resolve rate, latency, concurrency and total cost of ownership.
Career Highlights
- Leads the AI department at imec
- Speaker at AI Engineer (2026) on when running your own AI models beats paying for APIs
- Launching an imec alliance that brings AI software teams together with chip and infrastructure providers
Notable Positions
On Concurrency Over Accuracy
On owned infrastructure, memory has to hold both the model and each user's KV cache. Even large open-weight models on large systems served only about four reliable concurrent users for long-horizon coding tasks. Latré calls concurrency probably more important than accuracy.
On the Break-Even Point
Low-end hardware needs about 89% GPU utilization to beat the API, while B200-class systems need about 15%. GLM-class models are roughly where infrastructure starts to win, with about 25% savings for coding assistance.
On Harness Choice
Harnesses reached similar accuracy, but token use differed a lot. With GLM, Codex used about half the tokens of the others, which halves GPU time on owned hardware. See Agent Harness.
Key Quotes
- "Concurrency is actually probably even more important than accuracy in this sense." (on self-hosting)
- "Running a Kimi K3 model only gives you about four concurrent users that can really run reliably on this." (on hardware limits)
Related Reading
- Model Routing - The step before owning infrastructure
- Agent Harness - Why harness choice changes GPU cost
- Model Commoditization - Open-weight models closing the gap