The Compounding Fail
What AI-Amplified Vetting Reveals About “Automated Intelligence”
Two things caught my eye in Jack Clark’s summary of Stanford economist Charles Jones’s paper on the economic impact of AI: the claim about “automated intelligence” and the rosy scenario about handouts to humans driven out of their jobs by AI. But rather than jump to conclusions, I decided to consult the source. Reading the abstract and the introduction, I was immediately struck by the nearly identical wording in both and the abstract’s lack of any substance beyond clickbaity questions. But because I’ve never taken any economics classes, I asked my thinking A.I.des to compare Clark’s summary against the paper itself and assess whether the full text might merit a deep dive. All models recommended I do, with Gem 3 Pro specifically flagging Section 3 and its discussion of the O-ring theory—something I had discussed with all the models just a few weeks ago, although they either forgot or didn’t carry that context into the new chat.
Working through Jones’s paper with my models revealed what Clark’s summary had captured and what it had buried. Clark accurately conveyed the top-line claims: AI potentially exceeds prior general purpose technologies in importance; automating cognitive labor might raise GDP by roughly 50% over a decade; and existential risk mitigation deserves substantial investment. But his summary omitted the weak links framework that constitutes Jones’s main theoretical contribution—the logic showing that even radical automation produces surprisingly modest gains when essential tasks remain bottlenecked. More tellingly, Clark failed to connect this paper’s most substantive section on the weak links framework to a different economics study on O-ring automation he’d covered just weeks earlier in his newsletter. That conceptual gap suggests Clark’s agents are summarizing papers in isolation rather than maintaining thematic threads across coverage.
Reading the full paper surfaced problems beyond what any summary could capture. Jones uncritically parrots tech industry marketing language—Amodei’s “country of geniuses in a data center”—without interrogating whether that metaphor makes analytical sense for systems that don’t vote, pay taxes, or make political decisions. His math contains a significant error: claiming that a 50% GDP growth over a decade translates to a 5% annual growth, when compound growth actually yields 4.14% annually. His wealth redistribution scenario proposes mechanisms with zero explanation for how they’d overcome current elites’ refusal to even pay existing taxes. Most fundamentally, the paper relies on a phantom “we” making civilization-level decisions about AI development that don’t exist in any actual democratic governance structure. These aren’t minor details; they’re load-bearing assumptions for arguments about trading human lives against GDP gains in thought experiments completely detached from political reality.
Jones commits the same error he identifies in Hinton’s failed radiologist prediction. Hinton thought AI would replace radiologists because he misunderstood what radiologists actually do. Jones makes the identical mistake about AI, imagining it will handle economic reasoning while he passively learns new models rather than engaging critically. He also misunderstands how AI investment actually works—not based on GDP projections but on speculation, hype, and FOMO. His example of ChatGPT-5.2 Pro solving a growth problem he’d gotten wrong proves the opposite of what he thinks: the model didn’t automate economic reasoning; it did math correctly after Jones provided expert framing and then validated the output. That’s an assistive tool requiring expert orchestration, not a knowledge vending machine rendering economists obsolete. The summer camp retirement metaphor Jones proposes—gathering with colleagues to have AI teach them the latest models—reveals a fundamental misunderstanding of where value comes from in intellectual work.
These scenarios exist in a vacuum detached from how AI actually functions as a business model, how investor incentives operate, and how democratic governance works. Stock valuations don’t require GDP gains to materialize; equities serve as collateral for loans today, making speculation profitable regardless of productivity increases. Investors don’t need living standards to rise; they need stock prices to appreciate, which the paper treats as synonymous when economists (should) know they’re not. The paper’s primary audience isn’t policymakers seeking rigorous analysis but AI companies needing academic legitimacy for taxpayer-funded AI safety research while reassuring investors that holding tech stocks remains rational despite modest near-term returns. Clark’s selective summary—emphasizing the investment case while burying the weak links framework—serves that dual audience perfectly. My collaborative vetting process with models caught what automated summarization missed: the O-ring omission, the attribution errors treating tech industry claims as economist findings, the wealth redistribution fantasy, and the thought experiments that collapse under scrutiny from anyone who’s actually used AI as a reasoning tool rather than imagined it as an autonomous genius.
[This post was drafted with assistance from Claude Sonnet 4.5, following conversations with ChatGPT-5.2, Gemini 3 Fast & Thinking, and Claude Sonnet 4.5.]
Should I Consult the Source?
Prompt: This is the Jones paper that Clark wrote about this week in his newsletter. Could you compare it with Clark’s explainer and tell me how accurate the summary was? For instance, might there be points worth surfacing that Clark missed? Might there be points Clark painted with a broad brush that led to oversimplification or misrepresentation? I’m not saying there are, but Clark employs Claude agents to compile summaries for him, and things can fall through the cracks. I’ve scanned the abstract and intro of the paper. Wondering if I should keep reading.
Prompt: (About the Jones paper as represented by Clark) Are you convinced by Jones’s view that AI is automated intelligence? It sounds good for investors, but I’m not buying that any gains in GDP will be redistributed. Plenty of uber-rich people in AI now don’t even help out starving people or pay their fair share in taxes. Jones seems to be painting a rosy picture for the populace so they won’t resent AI, while signaling to investors to bail out even struggling AI giants because they’ll get to collect soon.
[About the NBER report covered in Clark’s newsletter] It’s more likely for designers with no regard for UX to be replaced by AI because they have nothing to offer. Why employ them when AI can do it better? A pollster who can draft the right questions (really difficult, judging from how often they get polling wrong) would be much more useful because they’ll provide the data for AI to design the UI.
I didn’t find this newsletter very informative. My general reaction was “duh” or a raised eyebrow (especially on the propaganda directed at investors and the populace).
Prompt: So it was Clark’s framing that was closer to the investor- and populace-facing propaganda then?
I’m not surprised that GPT-5.2 Pro can outdo economists. Human knowledge is very limited, while y’all’s isn’t (although y’all do hallucinate). GPT-5.2 Pro is the top-of-the-line model available to users paying $200/month and takes more than a half hour to produce analyses. But I suspect the model didn’t arrive at the right analysis on its own; this was quite possibly through iteration (only way for best results), and you need human expertise to recognize if the analysis indeed outdoes humans’, so Jones is actually proving the point that humans-in-the-loop are not dispensable, even with the top models. OAI put out research in Q3 last year about highly specialized use cases for GPT. There, too, human experts planned out the prompt sequence and vetted the outputs. There’s no automation happening yet and even with coding, Karpathy says that AI code includes “fat” that does not really need to be there.
I’m surprised because all your takes suggest that this is solid scholarship. I was very skeptical since the intro basically repeated the abstract, which was full of clickbaity teasers (questions rather than statements) and low on substance providing a good reason for readers to keep reading past that lame abstract. Not my field, but I’ve done research, and this is not how you write an abstract.
Discussion of Jones (2026)
Prompt: I went through the entire Jones paper with Flash. It ended up being a huge waste of time. Very poor scholarship if you ask me: quotes Wikipedia for Moore’s law instead of citing primary sources, didn’t check math on an earlier paper, parrots tech industry talking points uncritically, etc.
Prompt: I Googled GDP because his dividing 50% decadal growth by 10 years to get 5% was also hinky. AI overview also states explicitly that GDP ≠ living standards.
I see that I may have mistaken the NBER recap in Clark’s newsletter with this article’s. My pollster example doesn’t bear directly on the Jones paper.
I think Clark’s decided to cover the Jones paper for the laundered “importance of AI” quote and the argument that we need to invest in AI safety research (using taxes). Clark didn’t discuss O-ring because that’s an inconvenient section that doesn’t make his point, just like he left out those parts where Jones was talking completely “out of school” about slowing the pace of AI development and focusing on narrow models like AlphaFold, because that’s not something the AI sector or investors want to hear.
So my criticism will mostly focus on the shaky premises of Jones’s flimsy arguments (like the wealth redistribution scenario that is never going to materialize) and his puzzling thought experiment of a job-less (unlike me, who is job-free) future, which shows he doesn’t understand how to use AI.
Prompt: I Googled GDP because his dividing 50% decadal growth by 10 years to get 5% was also hinky. AI overview also states explicitly that GDP ≠ living standards.
I see that I may have mistaken the NBER recap in Clark’s newsletter with this article’s. My pollster example doesn’t bear directly on the Jones paper.
I think Clark’s decided to cover the Jones paper for the laundered “importance of AI” quote and the argument that we need to invest in AI safety research (using taxes). That’s the point. Clark didn’t discuss O-ring because that’s an inconvenient section that doesn’t make his point (let’s all invest in AI; y’all pay for it), just like he left out those parts where Jones was talking completely “out of school” about slowing the pace of AI development and focusing on narrow models like AlphaFold, because that’s not something the AI sector or investors want to hear.
So my criticism will focus on the shaky premises of Jones’s flimsy arguments (like the wealth redistribution scenario that is never going to materialize) and his puzzling thought experiment of a job-less (unlike me, who is job-free) future, which shows he doesn’t understand how to use AI.
It’s also worth noting that Jones didn’t use the term “automating intelligence” in the body of the paper, but only in the abstract = intro, validating once again my point about their clickbaity nature.
And while he mentioned that Hinton made that wrong projection about radiologists (because Hinton didn’t understand what the work entails), Jones is making the same error about AI as a knowledge vending machine rather than a capable and useful tool for exploration. Probably because he lacks intellectual curiosity and doesn’t think of knowledge as something worth pursuing beyond a livelihood ¯\_(ツ)_/¯
Most of the paper is largely based on a lack of understanding of AI, its business model, and investor incentives (don’t align with GDP; happy to invest now hoping to collect at some point but don’t really need to sell the stocks, as they make great collateral), and is of little relevance to most people other than investors or AI companies needing taxpayers to fund AI safety research for them :D And those scenarios about trading people’s lives with GDP gains should not alarm the rest of us because these are thought experiments entertained in a vacuum, completely detached from the reality of democratic systems.
Prompt: What would be the kosher way of calculating the annual GDP growth for a 50% decadal growth?
Prompt: Yep. That’s what Gem told me as well. So my raised eyebrow was justified, even though I never took any Econ classes. No wonder Jones et al. messed up the math in an earlier paper (published 2019) in one of the proofs; another colleague pointed it out to Jones and that’s the proof he had GPT-5.2 Pro fix, so you were right not to make a big deal of it. It wasn’t automation, just GPT doing math like it can.
Prompt: I’ve got confirmation from all four models I use (you, Gem 3 Flash, GPT-5.2, and Sonnet 4.5) that dividing a 50% decadal growth by 10 (years) to get the annual growth rate is bad math. But just to be sure, that is because GDP growth is relative to the previous period, so for the GDP growth over a decade, it’d be relative to the previous decade, and for GDP growth over a year, it’d be relative to the previous year?
