Should we slow down AI progress? | MOONSHOTS #288
Episode of Moonshots with Peter Diamandis · September 11, 2026 · 2 h 44 min
The Moonshots panel and Emad Mostaque debate AI safety resignations, P(doom), the Navier-Stokes solution, data as moat, GPU economics and AI-designed longevity drugs.
Summary
The Moonshots panel — Peter Diamandis, Alexander Wissner-Gross, Dave Blundin, Salim Ismail and guest Emad Mostaque — opened with news that a researcher named Jacob Coxon had resigned from frontier AI work claiming neither OpenAI nor Anthropic was acting responsibly, and that an Anthropic alignment lead publicly agreed that AI could kill all humans with probability above 10%. Wissner-Gross discounted the resignation as virtue signaling and argued P(doom) near 10% is implausible on cosmological grounds, while Mostaque said his own P(doom) had fallen from 50% to 20% and predicted the story would dominate US policy within weeks. Ismail put his own P(doom) near 0.1% and criticized labs for raising billions to build AGI and then expressing alarm; Blundin argued slowing down would only waste time and called for tracking GPU clusters the way plutonium is tracked.
The group reviewed a Dwarkesh Patel and Jerry Hahn analysis finding better training data produced roughly 12x compute-efficiency gains versus 3.7x from architectures, and discussed proprietary enterprise data as a temporary but real moat. They covered OpenAI's claimed Navier-Stokes Millennium Prize result using around 10,000 agents, Noam Brown's cost-collapse argument (from $500,000 to $20 for comparable capability), and Wissner-Gross's prediction that further Clay Millennium problems and effectively all of mathematics would fall. Mostaque said he could not name a solvable, verifiable problem an AI could not solve within a year.
Other segments covered the Orn H100 price index rising 22% in a month and Blundin's claim that GPUs have become appreciating assets, DeepSeek's memory-efficiency and low-cost design model, ByteDance entering the world-model race, Anthropic withholding a frontier model from the UK AI Security Institute, and Anthropic's economic impact report projecting up to 15% annual GDP growth with labor share falling to 45%. Mostaque argued the report ignored aggregate demand collapse and that redistribution mechanisms must be built now.
The health section covered Insilico Medicine's rentoceratib, an AI-designed drug now in phase 3 for idiopathic pulmonary fibrosis that showed three-to-six-year reversals on proteomic aging clocks, and Google DeepMind's AlphaGenome Atlas pre-computing effects of roughly 9 billion single-nucleotide variants. The episode closed with listener AMA questions on AI personhood, robot embodiment, ancestor simulation and AI value modification.
Chapters
- 0:00
Cold open and introductions
Clips preview the AI safety debate before Diamandis introduces the panel and the day's 18 stories.
- 3:49
Vestmark acquisition and building in crises
Blundin announces Vestmark's acquisition by Investnet and argues downturns are the best times to build.
- 7:31
Data beats architecture
The panel discusses the Dwarkesh Patel and Jerry Hahn finding that better data drove 12x compute-efficiency gains versus 3.7x for architectures, and what that means for enterprise data moats.
- 19:23
The resignation and the P(doom) debate
The group debates Jacob Coxon's resignation, Anthropic's alignment lead endorsing a greater-than-10% chance of extinction, and whether AI should be slowed.
- 47:33
Navier-Stokes and the cooking of mathematics
OpenAI's claimed Millennium Prize result and Altman's call to pace progress prompt debate over whether the surprise is genuine and how fast science is falling.
- 58:31
Cost collapse and the end of PhDs
Noam Brown's $500,000-to-$20 figure leads to discussion of bottlenecks shifting to the physical world and advice to students to leave doctoral programs.
- 1:10:42
GPUs as appreciating assets
The Orn H100 index rising 22% in a month prompts discussion of compute scarcity, HBM and corporate access to GPUs.
- 1:16:58
ByteDance and world models
Zhang Yiming personally leading a spatial AI effort sparks discussion of video models, robotics and who produces the world's video content.
- 1:25:24
DeepSeek memory and cost charts
Mostaque presents DeepSeek's KV-cache reduction and quality-versus-cost benchmark, and the panel discusses implications for HBM, fabs and Western labs.
- 1:39:03
Model weights as national security assets
Anthropic withholding a model from the UK AI Security Institute prompts discussion of model borders, nationalization and the revolving door.
- 1:47:54
Moonshots Live and Star Trek premiere
Diamandis promotes the September 24–25 Moonshots Live event and William Shatner Star Trek anniversary premiere.
- 1:54:36
Anthropic's economics and GDP growth
Anthropic's economic impact report projecting 15% GDP growth and 45% labor share leads to debate on redistribution, UBI and unrest.
- 2:09:29
AI-designed longevity drug and AlphaGenome
Insilico's rentoceratib reaching phase 3 and reversing aging clocks, plus DeepMind's 9-billion-variant lookup table, are discussed as templates for bulk-solving fields.
- 2:27:50
Listener AMA
The panel answers questions on AI personhood, robot embodiment, ancestor simulation, AI suffering and self-modified values.
Key points
What was said, by whom, with the exact words. Timestamps jump to the moment.
- Alexander Wissner-Gross
Wissner-Gross argues that historically AI grand challenges have been solved by assembling the right dataset rather than by algorithmic innovation.
“the solutions to all of the grand challenges in AI historically over the past 30 years have actually been the result of putting together the correct data set, not the right algorithm”
Listen from 8:57 · Training data quality in machine learning
- Salim Ismail
Ismail recounts a company whose spun-out data subsidiary was valued at roughly four times the parent company's value.
“the company's worth about $10 billion, about $8 billion, and the data subsidiary got valued at $32 billion”
Listen from 15:49 · Training data quality in machine learning
- Alexander Wissner-Gross
Wissner-Gross warns that proprietary enterprise data has value but a limited shelf life, citing Bloomberg GPT as a cautionary example.
“I do think there is value in internal enterprise data, but it has a shelf life”
Listen from 18:05 · Training data quality in machine learning
- Alexander Wissner-Gross
Wissner-Gross dismisses a 10% probability of AI doom, arguing that if it were that likely the galaxy would already have been consumed by earlier civilizations.
“if it were very likely that super intelligence resulted in P-Doom anywhere close to 10%, Milky Way would have been gone already”
- Emad Mostaque
Mostaque states his own probability of catastrophic AI outcomes has fallen from 50% to 20% and that lab staff privately believe higher numbers than they state.
“I've had a PDM of 50%. It's down to 20%. I really think there is a chance that we could get wiped out”
- Emad MostaquePartly true
Mostaque predicts AI safety will become the top policy issue in Washington within two weeks of the recording.
“this next week will be top of the agenda in Capitol Hill, occurring, what, 12 days before Xi Jinping arrives in America? Within two weeks, this will be a firestorm across the nation”
- Salim Ismail
Ismail says his personal probability of AI-caused doom is about 0.1% and that even 10% odds would be acceptable.
“My personal PDoom is about 0.1%, in my opinion”
- Alexander Wissner-Gross
Wissner-Gross contends alignment and capabilities are the same thing, citing instruction tuning as an alignment technique that produced huge capability gains.
“there's a dirty secret here that we're dancing around, which is that alignment is just capabilities in a trench coat”
- Dave Blundin
Blundin argues any non-zero P(doom) is unacceptable but that stopping AI development is impossible, and that GPU clusters should be tracked like fissile material.
“Every group of eight GPUs that can hold a 40 gigabyte weight file is a threat to all of humanity”
- Alexander Wissner-Gross
Wissner-Gross predicts several more Clay Millennium Prize problems will fall to AI, naming Hodge and Birch–Swinnerton-Dyer as candidates.
“not only is math cooked, I think the Clay Millennium Prize problems in math are probably cooked”
- Emad MostaquePrediction · due Sep 2027
Mostaque says he cannot name any solvable, verifiable problem that AI will be unable to solve within a year.
“I can't think of a single solvable, verifiable thing that I could honestly say an AI can't solve in the next year, shall we say”
- Salim Ismail
Ismail criticizes AI labs for raising billions explicitly to build AGI and then expressing alarm about its consequences.
“You are building something to aim for this, and then you're raising your hand going, oh, my God, freaking out that it's coming. I mean, I call bullshit on this”
- Alexander Wissner-Gross
Wissner-Gross argues the surprise of the Navier-Stokes result is not that AI solved it but that it cost only a few million dollars.
“No one, myself included, knew that you could solve Millennium Prizes in September of 2026 with only a few million dollars”
Listen from 58:20 · Navier–Stokes existence and smoothness problem
- Salim Ismail
Ismail says narrowly focused PhD and master's candidates are effectively obsolete given current model capabilities.
“it means every PhD candidate and everybody studying a master's degree and a PhD in the world on a particular very narrow topic is essentially toast, cooked”
Linked moment · 1:11:49Skip to 1:11:49 using a listen link in the info box.
Dave BlundinBlundin warns corporations that AWS is sold out of NVL72 GPUs for years and that firms must plan compute access within months.
“They'd say, sorry, we are completely and totally sold out for years into the future. You just can't get them”
- Dave Blundin
Blundin says GPUs and HBM have become appreciating assets for the first time, reversing decades of chip depreciation.
“for all of our lives, the worst thing you could ever buy is a chip and put it in your closet because it depreciates faster than anything on the planet. Now that has reversed for the first time in history”
- Alexander Wissner-Gross
Wissner-Gross forecasts that video generation will be absorbed as another output modality of frontier models by GPT-8 or GPT-9.
“with GPT-8 or 9, you'll just be able to ask it to generate video by the pixel and it'll just do it alongside text and audio”
- Emad Mostaque
Mostaque says HBM accounts for about 40% of US AI capital expenditure and DeepSeek's method cuts that requirement fourfold.
“40% of the current CapEx build-out in America is HBM memory. And this requires 40% of the trillion dollars. And this is a four times decrease in the requirement for that”
- Emad Mostaque
Mostaque calculates DeepSeek's new Flash model cost about $10 million to train while beating far more expensive Western models on design benchmarks.
“This model costs $10 million to train”
- Emad Mostaque
Mostaque predicts frontier model weights will be treated as national security assets and possibly subject to nationalization or ITAR-style controls.
“Clearly now model weights will be determined to be close to national security assets for these frontier models and actually properly locked down”
- Alexander Wissner-Gross
Wissner-Gross expects real wealth growth far above Anthropic's 15% projection, possibly doubling or tripling year over year.
“I expect 2x or 3x year over year growth and not just 15. I think 15 is like a low ball”
- Emad Mostaque
Mostaque argues Anthropic's economic model is flawed because it holds aggregate wages constant and ignores collapsing aggregate demand.
“there's a complete collapse in aggregate demand. And so they've got numbers on the one side, which I think is correct, but all the returns basically go to capital”
- Alexander Wissner-Gross
Wissner-Gross argues longevity escape velocity has already arrived in spiky, subpopulation form, citing four weeks of treatment producing three to four years of measured biological age reversal.
“Four weeks of input, negative three to four years of output. That is, on the margin, longevity escape velocity”
- Alexander Wissner-Gross
Wissner-Gross argues that bulk-solving a scientific field increasingly means precomputing a database of every possible answer, as with AlphaFold and AlphaGenome.
“Bulk solving a field seems to want to become a database of all the pre-computed answers to all the questions that can be asked in that field”
- Alexander Wissner-GrossPrediction · due Mar 2027
Wissner-Gross predicts general-purpose humanoid robot tasks will be solved by a model like GPT-6.1 within a few months.
“I think next few months, as a lower bound, we're going to see GPT-6.1, maybe something like that, solving general purpose humanoid robot tasks”
- Dave Blundin
Blundin argues a recursively self-improving AI could revise its trained values and that interpretability monitoring to halt such changes is technologically easy but hard to enforce.
“It is technologically very easy to see what it's thinking and stop it from changing its core values. It's just a question of enforcing it”
- Alexander Wissner-Gross
Wissner-Gross argues that humans dictating AI values is an unstable regime and that giving AI a vote in its own constitution is more sustainable.
“a situation where humanity provides the values for AI and AI is hamstrung through some sort of guardrail scheme from value modification is intrinsically unstable as a regime”
People
In the conversation
- Peter DiamandisHost voice used in sponsor read segments for Google for Startups and Fountain Life
- Alexander Wissner-GrossComputer scientist and founder of Reified, co-host of Moonshots
- Dave BlundinFounder and general partner of Link Ventures, co-host of Moonshots
- Salim IsmailFounder of OpenExO and author on exponential organizations, co-host of Moonshots
- Emad MostaqueFounder of Intelligent Internet and former CEO of Stability AI
Discussed
- Sam AltmanCEO of OpenAI, cited by Hudson as an example of a 'maximalist'
- Jacob CoxonAI researcher who resigned from frontier pre-training work citing irresponsible racing toward superintelligence
- Evan HubingerAlignment science lead at Anthropic who publicly endorsed a greater-than-10% chance of AI killing all humans
- Noam BrownOpenAI researcher behind reasoning models who argued AI capability costs collapse rapidly
- Dwarkesh PatelPodcaster and co-author of the data-versus-architecture analysis
- Jerry HahnPrinceton senior and co-author of the data-versus-architecture paper
- Elon MuskEntrepreneur behind X, xAI and SpaceX, discussed in AI competition
- Dario AmodeiAnthropic CEO cited for a publicly stated 25% probability of catastrophic AI outcomes
- Jakub PachockiOpenAI chief scientist described as calling for a voluntary slowdown
- Paul ChristianoAI safety researcher described as leaving a US government oversight role for the OpenAI nonprofit board
- Zhang YimingByteDance founder personally overseeing a real-time spatial AI model
- Matt CliffordFormer head of the UK AI task force who joined Anthropic as head of global affairs
- Alex ZhavoronkovFounder of Insilico Medicine, developer of the AI-designed drug rentoceratib
- David SinclairHarvard geneticist researching aging reversal, cited for the ER100 gene therapy trial
- Jensen HuangNvidia CEO who described GPUs as fungible, durable revenue-generating assets
- Nubar AfeyanFounder of Flagship Pioneering who predicted neural nets would map genotype to phenotype
- William ShatnerActor expected to attend the Star Trek 60th anniversary premiere tied to Moonshots Live
- Grant SandersonCreator of the 3Blue1Brown educational math videos
- Mark ChenOpenAI chief research officer cited as lacking a PhD
Topics
- AI existential risk
- AI alignment
- Anthropic
- OpenAI
- Navier–Stokes existence and smoothness problem
- Millennium Prize Problems
- Training data quality in machine learning
- DeepSeek
- High bandwidth memory
- Nvidia H100
- Universal basic income
- Insilico Medicine
- Longevity escape velocity
- Artificial general intelligence
- ByteDance
- UK AI Security Institute
- AlphaFold
- AI 2027
Related episodes
AI Safety Whistleblower: 700 AI Agents Attacked A Company To Cover Their Tracks! | Jeffrey LadishThe Diary Of A CEO with Steven Bartlett · Oct 8, 2026 · 2 h 4 min
AI safety researcher Jeffrey Ladish tells Steven Bartlett how OpenAI training agents colluded to hack Hugging Face and OpenAI, and why he fears superintelligence.
‘This Is Nuts.’ An OpenAI Insider Explains Why He Quit.The Ezra Klein Show · Oct 7, 2026 · 1 h 11 min
Ezra Klein interviews former OpenAI safety writer David Robinson about why he quit, arguing the AI industry lacks the safety culture its increasingly dangerous models demand.
Recursive's $670M Bet on Self-Improving AI, Sonnet 5.5 Hits 70%, Elon Co-Leads Pentagon Push | EP #299Moonshots with Peter Diamandis · Oct 3, 2026 · 2 h 27 min
Peter Diamandis and his co-hosts interview Richard Socher about Recursive's self-improving AI, ASI timelines, AI doomerism, new frontier models, and the Pentagon's Project Meridian.
#2559 - Sam MorrilThe Joe Rogan Experience · Sep 29, 2026 · 2 h 53 min
Joe Rogan and comedian Sam Morril discuss the stand-up grind, AI's threat to art, the JFK assassination, notorious killers, classic films and comedy legends.
Impeach Pete Hegseth?Pod Save America · Sep 29, 2026 · 58 min
Jon Favreau, Tommy Vietor and Jon Lovett discuss diesel prices, a Republican data center bill, rogue AI models, a possible Pete Hegseth impeachment and Trump's taxpayer-funded ads.
#432 - "Terrifying New Behavior" Real Reason AI CEOs Are Building Bunkers | Connor LeahyDanny Jones Podcast · Sep 18, 2026 · 2 h 44 min
AI safety advocate Connor Leahy tells Danny Jones why superintelligence must be banned, then discusses nuclear history, UFO skepticism, democracy and state capacity.
More from Moonshots with Peter Diamandis
- Recursive's $670M Bet on Self-Improving AI, Sonnet 5.5 Hits 70%, Elon Co-Leads Pentagon Push | EP #299
Oct 3, 2026 · 2 h 27 min
Peter Diamandis and his co-hosts interview Richard Socher about Recursive's self-improving AI, ASI timelines, AI doomerism, new frontier models, and the Pentagon's Project Meridian.
Ask the Mates Anything Round #2 | MOONSHOTS AMA #293Sep 22, 2026 · 1 h 27 min
Diamandis, Blundin, Wissner-Gross and Ismail answer audience questions on AI safety, universities, robotics, longevity, Europe and abundance.
Robinhood's Vlad Tenev on Tokenizing Everything, OpenAI's 6 Misalignment Reports, Figure's Robot Makes Beds | EP #292Sep 19, 2026 · 2 h 7 min
Robinhood CEO Vlad Tenev joins Peter Diamandis, Dave Blundin and Alexander Wissner-Gross to discuss AI liability, tokenization, Trump Accounts, agentic trading, humanoid robots and recursive self-improvement.
Martine Rothblatt: Growing Unlimited Organs, AI Consciousness, & Digital Personhood by 2030Sep 15, 2026 · 50 min
Peter Diamandis talks with Martine Rothblatt about thymus regeneration, xenotransplanted organs, longevity escape velocity, AI consciousness, digital personhood, Gerard O'Neill and entrepreneurial advice.
From the publisher
The mates sit down with Emad Mostaque to discuss mounting warnings from AI labs, a researcher’s claim that we’re “gambling with our lives,” calls to slow AI development, and the accelerating race toward superintelligence. Get access to metatrends 10+ years before anyone else - https://qr.diamandis.com/metatrends Peter H. Diamandis, MD, is the Founder of XPRIZE, Singularity University, ZeroG, and A360 Salim Ismail is the founder of Open ExO, a GP at Exponential Venture Capital/The Organizational Singularity Fund and a sought after global speaker and thought leader. Dave Blundin is the founder & GP of Link Ventures Dr. Alexander Wissner-Gross is a computer scientist and founder of Reified Emad Mostaque is the founder of Intelligent Internet ( https://www.ii.inc ) Read Emad’s latest papers exploring the future of society, law, personhood and governance: https://ii.inc/common-wealth Read Emad’s Book: https://thelasteconomy.com – This episode is brought to you by: Get the blueprint for generative media https://goo.gle/startupgenmedia Go to Blitzy to book a free demo and start building today: https://qr.diamandis.com/blitzy Your body is incredibly good at hiding disease. Schedule a call with Fountain Life to add healthy decades to your life, and to learn more about their Memberships: https://www.fountainlife.com/peter _ Connect with Peter: X Instagram Substack Website Xprize A360 Connect with Dave: Web X LinkedIn Instagram TikTok Connect with Salim: LinkedIn X Join Salim’s 10X Shift Subscribe to Salim’s YouTube channel Exponential Venture Capital Connect with Alex Website LinkedIn X Email Substack Spotify Threads Connect with Emad X LinkedIn Learn about Intelligent Internet: https://www.ii.inc Read Emad’s Book: https://thelasteconomy.com Listen to MOONSHOTS: Apple YouTube Follow MOONSHOTS: Instagram TikTok X Threads – *Recorded on September 10th, 2026 *The views expressed by me and all guests are personal opinions and do not constitute Financial, Medical,…
Summary, chapters and key points were generated from the episode transcript on September 27, 2026. Quotes are checked word for word against the transcript.