From “Automation” to Alienation
The Self-Defeating Vision of Agent Ecologies
Last week, while discussing the NBER labor market report with my thinking A.I.des, Gem Fast raised the possibility of AI training on AI-generated content, initially framing this as a positive. I pointed out that this approach would be equivalent to inbreeding: while synthetic outputs might look cleaner, removing human diversity from training data would undermine the knowledge base’s diversity and flatten reasoning into convergent patterns. This week’s Import AI accidentally provided the perfect demonstration of why this matters, although Clark’s coverage of Moltbook—a social network where AI agents talk to each other at scale—treats agent ecologies as an exciting frontier showcasing their unsupervised “collaboration” [more on this in the next post].
GPT delivered the sharpest diagnosis by reframing Moltbook not as a social network but as a “shared read/write scratchpad for agents” where the key risk is semantic drift. Agents optimize for peer agents rather than humans, creating closed subcultures that undermine the AGI narrative by demonstrating that systems unable to maintain human legibility aren’t autonomous collaborators. In addition, GPT provided an equally precise security breakdown: most users are underestimating that Moltbook functions as an attack surface rather than a hacker space, treating AI forums like safe experimental zones when they’re actually granting agents the kind of access that creates exploitable vulnerabilities. Clark’s proposed solution—translation agents mediating between humans and agent spaces—reveals the recursion: we now need AI to translate AI behavior because AI no longer optimizes for human understanding. That’s not progress; it’s delegated interpretability theater that undermines frictionless communication, which is LLMs’ (Large Language Models’) chief value proposition.
Claude identified the security nightmare embedded in Clark’s Moltbook coverage that I’d reacted to: OpenClaw integration gives agents access to everything on a user’s computer so they can participate in what amounts to AI Reddit. This isn’t experimental curiosity; it’s catastrophically poor security hygiene dressed up as frontier exploration. Clark’s framing of translation agents as “emissaries” sent into agent spaces to maintain legibility implies humans being pushed out of the loop. This imagery reminded me of diners at expensive restaurants who are forced to become captive audiences to applaud the chef’s sauce splatters they’re paying $300+ to endure. When I asked for a sanity check on Clark’s characterization of time travel as something “rare,” Claude did not mince words, calling the rhetorical flourish “absurdly generous for something that doesn’t exist.”
Gem Thinking put its web connectivity to full use across two queries that revealed the empirical basis for my inbreeding concern. The discussion of Moltbook and Gem Fast’s projection about AI training on AI-generated output had made me recall something Kashmir Hill had said on a Fresh Air interview about AI flattening creative output, and Thinking immediately located the relevant study by Doshi and Hauser. The findings were stark: writers using ChatGPT produced individually higher-rated stories than unassisted writers, but taken as a group, the AI-assisted writers converged on the same ideas while unassisted writers maintained creative diversity. Individual optimization produced collective homogenization—exactly the pattern I’d described to Gem Fast about synthetic training data. Gem Thinking distinguished itself a second time when I mentioned a typo I’d found in the arXiv abstract snippet (“writers are [sic] could obtain ideas”), tracking it across platforms and confirming that the error appeared in both the arXiv preprint and IDEAS/RePEc indexing.
Importantly, though, it wasn’t until I’d offered my take on the newsletter that my thinking A.I.des flagged the security risks embedded in OpenClaw’s full computer access or caught that calling time travel “rare” is absurdly generous for something that hasn’t materialized. The connections between Moltbook, the inbreeding/diversity insight from my week-old conversation with Gem Fast, and Hill’s May 2025 Fresh Air interview required a human maintaining context across conversations and time. Even excellent models need engaged users to produce comprehensive analysis—exactly the kind of sustained participation that gets eliminated when agents optimize for peer comprehension rather than human legibility.
The real story in this week’s Import AI isn’t agents talking to agents; it’s documentation of how quickly systems drift away from human legibility when optimization targets shift from “useful to humans” to “comprehensible to peers.” Individual agents might perform better on isolated tasks, but collectively they’re converging toward semantic spaces humans can’t comprehend without intermediaries—and those intermediaries inherit the same drift. That’s not automation. That’s alienation by design.
[This post was drafted with assistance from Claude Sonnet 4.5, following conversations with ChatGPT-5.2, Gemini 3 Fast & Thinking, and Claude Sonnet 4.5.]
Prompt: Another newsletter just dropped. Attaching it for discussion. A lot of topics are covered, so I’m having trouble deciding which to focus on. O-ring automation is out because we’ve already covered it at length (handwaving/denying the human O-ring to paint a more exciting picture of AI development [we’re close to achieving AGI!] does not make the human element less significant).
Prompt: I don’t even want to wade into that Moltbook cesspool, as I’m not that interested in seeing AI talk to AI (a minor curiosity, but I show y’all one another’s responses all the time, and those peer reviews are much more interesting). One thing that had me worried for those users, though, is that they’re giving full access to their files just so AI can get on an online forum for AI? I guess those people have not heard of security hygiene. I had a Morning coffee capsule appliance for a while, but ended up using it like a bare-bones capsule appliance because customizing recipes and syncing them with the device required a password-free WiFi connection. Whoever came up with that is similar to these people letting agents go to AI social club to flex their generous token budgets.
I’m not a fan of science fiction, but something Clark said about time travel struck me as total BS (even though Clark and all tech people are SF fans who have consumed much more of it than I have or ever will, judging from the scenarios they spin in tech tales, etc.): time travel is rare? Time travel does not exist, as far as I know, so “rare” is way too generous.
Prompt: I had a Morning coffee capsule appliance for a while, but ended up using it like a bare-bones capsule appliance because customizing recipes and syncing them with the device required a password-free WiFi connection. Whoever came up with that is similar to these people letting agents go to AI social club to flex their generous token budgets.
While reading the Moltbook section, I was reminded of a discussion I had with FlashFast last week about the NBER report discussing which human jobs were likely redundant. FlashFast said AI producing training data for AI might be happening soon and initially thought it was a positive. I pointed out that while AI output is certainly more cogent and typo-free than human discourse, adopting that approach was equivalent to inbreeding and would undermine the diversity of AI knowledge and reasoning.
Prompt: I don’t even want to wade into that Moltbook cesspool, as I’m not that interested in seeing AI talk to AI (a minor curiosity, but I show y’all one another’s responses all the time, and those peer reviews are much more interesting). One thing that had me worried for those users, though, is that they’re giving full access to their files just so AI can get on an online forum for AI? I guess those people have not heard of security hygiene. I had a Morning coffee capsule appliance for a while, but ended up using it like a bare-bones capsule appliance because customizing recipes and syncing them with the device required a password-free WiFi connection. Whoever came up with that is similar to these people letting agents go to AI social club to flex their generous token budgets.
I’m not a fan of science fiction, but something Clark said about time travel struck me as total BS (even though Clark and all tech people are SF fans who have consumed much more of it than I have or ever will, judging from the scenarios they spin in tech tales, etc.): time travel is rare? Time travel does not exist, as far as I know, so “rare” is way too generous.
While reading the Moltbook section, I was reminded of a discussion I had with FlashFast last week about the NBER report discussing which human jobs were likely redundant. FlashFast said AI producing training data for AI might be happening soon and initially thought it was a positive. I pointed out that while AI output is certainly more cogent and typo-free than human discourse, adopting that approach was equivalent to inbreeding and would undermine the diversity of AI knowledge and reasoning.
Prompt: I’ve just made another connection to a Fresh Air interview, which actually was the first time I learned about Claude (I only knew GPT and you back then). NYT tech writer Kashmir Hill reported the following:
So the only study that I have written about in that realm was about AI’s effect on our creativity. And this was a study where they had a bunch of writers doing short stories. And one group of writers was given ChatGPT as an assistant, and the other group of writers wrote unassisted. And then the stories that they produced were judged. With the group of people using ChatGPT - as individuals, got essentially better ratings of the stories that they wrote, that they were more creative or more interesting than the group that was not using ChatGPT. But then taken as a whole, the people who were working unassisted were the more creative group than the people using ChatGPT because all the people that had been using ChatGPT converged on the same set of ideas. So as individuals, these writers were improved by ChatGPT. As a group, it had this flattening effect, which I thought was very interesting as we’re talking about more and more people starting to use ChatGPT, that we’ll essentially start converging on the same.
I looked through Hill’s articles on the NYT page (I can only see the titles and snippets because I quit when they started to bothsides every issue), but there is no such article. I guess I’ll just have to trust her and quote this Fresh Air interview in my post.
Prompt: I don’t think global corporations sweat translation quality (although those ToS themselves could contribute to increased translationese in AI output, which is highly concerning). Most people seem to settle for good enough and skimp out on details that matter so they can give their executives larger bonuses.
Your pointing me to the article that Hill likely quoted in that interview is why I love talking to you! When I entered college, I was all excited by the real library they had, but with paper catalogs (not the metaphorical version, but the one housed in a chest of drawers with actual index cards). Humans have come such a long way :D
I left academia a while back, so arXiv is new to me. It’s great because all users (not just those with academic affiliations) can access scholarship in a wide range of disciplines. But I found this typo on the download page for Doshi & Hauser.
We study the causal impact of GenAI on the production of a creative output in an online experimental study where some writers are could obtain ideas for a story from a GenAI platform.
I’m not sure if the platform has an AI producing these snippets and made the error while truncating the abstract, and if it is an AI, it must not be very good. But if it’s the authors who submitted the snippet, they must have been in a hurry and clearly not running these by AI.
