Human-in-the-Loop Sucks
Well...I did it again. I wrote an entire newsletter and scheduled it to go out without copying it from Google Docs to Ghost. I had assumed I'd get around to the copying yesterday morning, after giving a keynote in Warsaw (where I am as I write this now), but I happily spent the whole rest of the day signing books and answering questions. Oh well...here's the real deal...
Whenever executives or policymakers get nervous about artificial intelligence, they reach for the same comforting incantation: “Don’t worry, we’ll keep a human in the loop.” [1] It sounds safe and responsible, but from a cognitive neuroscience perspective, it is frequently a recipe for disaster.
We have spent the last 3 years treating AI assistance as a simple linear addition: human intelligence + machine speed = super-worker. But the human brain is a ruthless miser, operating on a strict "use it or lose it" basis [2]. When you drop an effortless, hyper-fluent oracle into a workflow without redesigning the learning environment, you don't get human-AI symbiosis. You get large-scale (but not universal) cognitive offloading: a temporary spike in surface output paid for by the quiet, systematic atrophy of human judgment.
[1] "Human in the loop" is rapidly becoming the 21st-century equivalent of the medieval "royal food taster", a job description that sounds prestigious right up until you realize your sole institutional function is to absorb the poison when the kitchen makes a mistake.
[2] Or perhaps it is “utilize it or reassign it”.
Research Roundup
The Expertise Bifurcation [1]
Does working with a frontier AI assistant build professional skill by exposing novices to expert-level patterns, or does it hollow out their development? Yes. [1]
In a preregistered, 3-month randomized controlled trial, researchers gave 133 practicing patent attorneys across 11 U.S. intellectual property law firms access to a custom AI drafting assistant. Blinded expert patent lawyers graded the participants' output both while using the AI (at 10 and 90 days) and after the trial concluded, on an unassisted patent-redlining task requiring deep professional judgment.
While using the AI, treated lawyers delivered significantly higher-quality work at both 10 days (0.34 SD, p = 0.03) and 90 days (0.38 SD, p = 0.01), with the largest immediate performance boosts accruing to junior lawyers. There’s the boost every AI productivity paper shows. But then…
…when the AI was taken away at the end of the 3 months, a stark divide emerged. Treated lawyers outperformed controls by 0.32 SD overall, but that durable skill advantage was concentrated entirely among senior lawyers (0.45 SD, p = 0.02). Junior lawyers showed zero average gain, with unassisted scores bifurcating into extremes of very good and very poor.
Notably, the lawyers who gained the most while using the AI retained the least afterward. Without a pre-existing scaffold of foundational expertise, AI assistance acts as a substitute for judgment rather than a catalyst for developing it.
[1] Not Robert Ludlum’s best book.
[2] That is not the answer most reporters and policy makers are seeking. It doesn’t make good, screamy influencer content. But it is real and matches my own findings and those of others.
Human-Out-Of-Their-Own-Loop
Recent high-profile studies have celebrated the fact that conversing with an LLM can talk people out of conspiracy theories and false headlines. But does outsourcing fact-checking to a chatbot actually teach humans how to evaluate evidence on their own?
To answer this question, study participants classified news headline-and-image pairs as real or fake, discussed their assessments with an AI system, and then completed unassisted evaluations of unseen news items.
The results were…predictable. During the AI-assisted sessions, participants' accuracy jumped by an impressive 21% on average. Yet when tested unassisted on novel items at week 4, participants' independent discernment performance dropped by 15.3% compared to their own week 0 baseline.
Rather than internalizing the analytical strategies modeled by the AI, participants learned to rely on the external oracle to resolve their uncertainty. By removing the cognitive friction of independent verification, the AI intervention cured the immediate symptom (believing a specific false headline) while actively degrading the host's immune system (the ability to spot the next lie alone).
No Error, No Learning
A consensus has emerged in recent meta-analyses of human–AI collaboration: on average, a human paired with an AI performs worse than the better of the two working alone. To many researchers this was either mysterious or damning, depending on your beliefs about AI. If you follow my research, you should already guess what comes next.
A meta-meta-analysis reanalyzed all 74 experimental studies from that initial meta-analysis [1], coding each experimental paradigm for whether it actually allowed the human to learn how the AI behaves over time.
The reanalysis uncovered a critical design flaw across the entire field [2]: almost no studies provided participants with outcome feedback after their decisions. Providing users with "explainable AI" (where the model justifies its reasoning) without outcome feedback was associated with negative human–AI synergy. Humans simply fell for fluent, plausible-sounding AI rationalizations even when they or the model was wrong.
Conversely, when AI explanations were paired with ground-truth outcome feedback, learning occurred. AI explanations do not help humans learn unless the environment forces the human through an active reinforcement-learning loop to calibrate when the machine’s confidence is a mirage.
[1] Or maybe it's a meta-reanalysis…or a re-meta-analysis…or maybe I just never meta-analysis I didn’t like.
[2] Not including me or David Autor.
Takeaway
Put these 3 studies together and the comfortable corporate fantasy of "centaur workflows" starts to look deeply naive. We are discovering a fundamental law of hybrid intelligence: cognitive prosthetics only amplify what’s already there. If your deep, you go deeper, but if you are shallow, you might never go deep again.
When a senior patent attorney uses an LLM, they already possess a dense, high-dimensional internal model of patent law. When the AI generates a draft, the senior lawyer’s brain automatically computes a prediction error between their own internal model and the machine’s output. That friction is a learning signal; it sharpens their schema. The junior lawyer—or the casual news reader—has no such internal model. For them, the AI’s fluent output doesn't trigger a prediction error; it simply replaces the need to build a model at all. Worse, if the AI offers a persuasive explanation without ground-truth feedback, it actively seduces the human into cognitive surrender.
If we want hybrid intelligence that actually scales human capability rather than farming it out until our cortices smooth over like river stones, we have to invert how we design AI tools. Stop building frictionless concierges that hand over finished answers wrapped in flattering explanations. Build adversarial sparring partners that withhold the final step, force the human to commit to a prior, and deliver brutal, unvarnished outcome feedback. If your AI workflow doesn't occasionally give you a headache, it isn't augmenting your intelligence—it’s liquidating it.
SciFi, Fantasy, & Me
The Rouse was definitely a different experience than of China Miéville other novels. It’s not an immersion into wildly different worlds like Perdido Street Station or The City and The City. It’s more of an immersion into flawed people and turns of phrase.
I’ll admit that it was a challenging audiobook: long repeated passages with subtle alterations are hard to parse with just ears. But it also engaged me for 45 hours of ghosty mysteries.
Stage & Screen
- October 19-23, Warsaw: I'm here right now!
- October 24, London Town: A little interview with the BBC!
- October 26, Bonn: It's on in Bonn!
- October 28, Fayetteville, NC: This is a big maybe, but I've never spoken in North Carolina before.
- October 29, San Diego: ...or maybe I'll be at UCSD for a book talk.
- November 19, Bangkok: Finally back in East Asia!
- Anyone in Australia want to piggyback on that trip?
- December 7-8 NYC: The Kennedy Human Rights Center Gala!
- Already next year: Helsinki, Orlando, Purdue, Curicao, Toronto, Monmouth, UMass, & NYC