NeurIPS 2025: Jeff Dean, Yejin Choi, and the State of AI Research

What NeurIPS 2025 Reveals About AI Research Directions

This video captures the pulse of frontier AI research through candid interviews at NeurIPS 2025. The conversations reveal a field grappling with fundamental questions about what current approaches can and cannot achieve.

Jeff Dean discusses Google's 7th-generation TPU "Ironwood" and the challenging exercise of predicting ML computation needs 2.5 to 6 years out for hardware design. More interesting is his advocacy for academic research funding - he and colleagues recently published a paper tracing how Google was built on academic research (TCP/IP, the Stanford Digital Library Project that funded PageRank, neural networks from 30-40 years ago). His pitch: 3-5 year research moonshots with mixed teams are the sweet spot for ambitious but achievable goals.

Yejin Choi delivers the most provocative take: despite the excitement around reinforcement learning, it all comes back to data. RL isn't magical exploration - it's "synthesizing even more data because that much data is still not good enough." Her concern is that all current approaches are interpolating within "the neighborhood of internet data" - the artifact of human knowledge - which isn't the same as discovering new truths like how to cure cancer. She advocates for "abductive reasoning" (forming hypotheses from partial observations, like Sherlock Holmes) rather than just inductive and deductive reasoning, which she calls "regurgitation of information you already had."

Robert Nishihara's interview adds historical context - NeurIPS went from 400 people who scheduled workshops around ski breaks to 30,000 attendees. His insight on underinvestment: most research focuses on getting knowledge into model weights, but "there's a ton that can be done" with context-based approaches. He envisions continual learning breakthroughs happening without changing model weights at all.

4 Insights From Jeff Dean, Yejin Choi, and Robert Nishihara

• Jeff Dean's forecasting exercise for TPU design: predict what ML computations the field will need in 2.5-6 years, then build hardware features for things that "might be important" even if uncertain
• Yejin Choi argues RL for reasoning isn't fundamentally different from supervised fine-tuning - both are data synthesis strategies, and with enough effort SFT (like OpenThought) can beat RL approaches
• "Abductive reasoning" - forming hypotheses from partial observations - is what scientists and detectives actually do, distinct from induction/deduction which Choi calls "paraphrasing what was already in your knowledge"
• Robert Nishihara predicts breakthroughs in continual learning via context management rather than weight updates - today's reasoning gets "thrown away" after each session