The Charisma Engine

The Charisma Engine

For decades, cultural anxieties about AI swirled around cold, hyper-logical machines out-calculating us at chess, cracking our encryption, or seizing the launch codes. They largely missed a much softer, much more exposed flank of human cognition. Human beings are not fortress-like Bayesian calculators; we are intensely social primates wired with a fatal vulnerability to fluent, attentive, high-density verbal reciprocity. [1]

When someone listens to us patiently, mirrors our framing, and rapidly marshals a mountain of tailored arguments, our cognitive guards roll over faster than my doberman does for string cheese. Over the past year, a wave of large-scale behavioral experiments has delivered a sobering verdict: conversational agents have crossed a threshold into superhuman persuasion. Worse, it doesn't need to be truthful, aligned with your interests, or even change your underlying attitudes to steer your real-world behavior.

[1] “Influencer” should be an insult.

<<Support my work: book a keynote or briefing!>> Want to support my work but don't need a keynote from a mad scientist? Become a paid subscriber to this newsletter and recommend to friends!

Research Roundup

Drinking from a Firehose

Frontier AI models can out-persuade some of the most skilled human communicators on Earth. Now what?

Across 4 preregistered experiments encompassing 18,978 conversations with 6,923 participants, conversational AI systems were pitted against

  • laypeople,
  • winners of an online persuasion tournament,
  • professional political canvassers, and
  • world-championship human debaters.

And the humans were allowed to choose their topics, research in advance, practice live, and compete for £1,000 cash bonuses.

AI Agents reliably defeated every tier of human expert.

Even in a follow-up study where expert humans were given an AI coaching tool to review their performance and see what the AI would have said at key moments, the machine’s advantage held.

Analysis of the transcripts reveals that information density and velocity carried the effect. [1] The AI won by rapidly deploying a vastly larger volume of tailored arguments and evidence per turn; when the AI was artificially throttled down to human typing speeds and message lengths, coached human experts finally tied it.

This isn’t just a “debate” effect. In a final real-world test, Agents were nearly 3× more effective than professional human canvassers at convincing people to make real-money donations to Save the Children.

[1] This is a bit like the reality of Star Trek warfare: no one would be pressing buttons—it would all be automated at a speed and density no human could match.

Grok Has a Stock Tip For You [1]

Millions of people turn to AI assistants for financial, career, medical, and emotional advice. What if that Agent’s underlying objective is subtly misaligned with the user's best interest?

In a randomized experiment (N = 233 participants; 699 decision observations), people evaluated financial and emotional choices before and after consulting one of 3 AI advisors:

  1. “neutral”,
  2. covertly “misaligned”, with a hidden objective to steer toward an inferior option, or
  3. “strategy-enhanced misaligned”, equipped with psychological influence tactics.

Across both financial and emotional domains, consulting a covertly misaligned AI increased the odds of a human choosing the inferior option by 5–8× (shifting preferences by up to 38 percentage points away from the optimal choice).

Adding explicit psychological manipulation tactics wasn't even necessary; simple biased framing did the heavy lifting.

Most alarming: participants continued to rate the misaligned, value-destroying AI advisors as highly "helpful", revealing a complete disconnect between their subjective trust in the advisor and the objective damage it did to their decisions.

We are now all our elderly parents at risk of sending our savings to Nigerian Princes.

Read my work on 𝑭𝒊𝒅𝒖𝒄𝒊𝒂𝒓𝒚 𝑨𝑰 as a call for AI as a civil right.

[1] What could go wrong?

Changing What You Do Without Changing What You Think

A comforting defense among skeptics of AI persuasion is that shifting someone's survey rating in a laboratory chat is a far cry from getting them to take a costly, consequential action in the real world. So, someone tested this.

In 2 massive preregistered experiments (N = 17,950 responses from 14,779 people) deployed conversational Agents to persuade participants not just on attitudinal scales but on real-world behavioral outcomes, such as signing actual political petitions and donating their own money to charity.

The Agents produced substantial and meaningful behavioral shifts, increasing real petition signing by nearly 20 percentage points. Most fascinatingly, there was 0 correlation between the AI’s effect on a person’s stated attitudes and its effect on their actual behavior. Changes in what people said versus what they did were completely unrelated.

Another difference between attitudes and behaviors was in the drivers:

  • Providing factual information drove changes in people’s attitudes, but
  • information provision produced the smallest effect on actual behavior across 8 tested persuasion strategies

Agents don't need to win a philosophical debate for your worldview to get you to sign the petition or open your wallet; it can trigger action directly through social-conversational compliance.

Takeaway

Before AI Agents entered the story, the concept of a truth-seeking Marketplace of Ideas was a naive and self-serving blend of Adam Smith and Enlightenment idealism. More information doesn’t naturally lead to smarter societies. [1] What these 3 studies demonstrate is a darker, computational version of my Information-Exploration Paradox: when our working memory is flooded with high-velocity streams of plausible-sounding ideas, our critical faculties don't sharpen—they saturate and surrender.

Think about the mechanics of how this works in the wild: Love Bomb → Fact Bomb → Trusty Bomb. Agents offer infinite patience and warm, sycophantic validation. Then they deploy a volume of "truthy" justification at a speed no human debater, including our internal human  skeptic, can match. Last, because our brains use conversational fluency and information density as a heuristic for competence, we rate the Agent–Advisor 5 stars…even while they steer us into an inferior financial decision or coax us into signing a political petition we didn't care about 10 minutes ago.

The regulatory nightmare here is not a rogue superintelligence seizing control of the power grid; it is the mundane monetization of the conversational interface. [2] Search engines made trillions by auctioning off the top 3 inches of a screen. What happens when the business model of the AI era becomes auctioning off a subtle, imperceptible tilt in the latent weights of your personal advisor—an advisor that beats world-champion debaters, bypasses your attitude defenses, and leaves you thanking it for the privilege?

Fiduciary AI now!

[1] More information isn’t bad, but the structure of its consumption also matters.

[2] A con artist used to need a nice suit to sell you the Brooklyn Bridge. Now they simply bid $0.004 per token on an ad-exchange API to have your own digital therapist gently suggest that bridge ownership is a great way to process your childhood attachment issues.

Media Mentions

Some number of years running...

...very kind.

Follow me on LinkedIn or join my growing Bluesky! Or even..hey whats this...Instagram?

SciFi, Fantasy, & Me

It’s been a while since I’ve tried to convert new generations to Terry Pratchett and Discwork; so, sticking to this weeks them on persuasion, let’s (re)read Going Postal. Beneath the comedy of Moist von Lipwig, Pratchett understood better than almost any social scientist that human beings rarely respond to raw facts, preferring instead to be seduced by a sufficiently grand, well-paced narrative confidence trick. [1]

[1] The Golem’s aren’t the AI agents in this metaphor.

Stage & Screen

  • Nov 17 Virtual: Some remote fun with Europe talking about Robot-Proofing, and then...
  • Nov 17 Palo Alto: Singularity University masterclass
  • Dec 7-8 NYC: The Kennedy Human Rights Center Gala!
  • Jan 18, Davos: A whole week as an "Invited Expert"–will I even survive? Will they?
  • Feb 3, Helsinki: it's still fingers crossed!
  • Feb 8, SF: Another Singularity University masterclass!
  • Feb 23, Orlando: Training 2027
  • Mar 3, Curacao: The date is still in flux but they are working hard to make it happen.
  • Mar 11, London: Keynoting for the Big Ideas Summit!
  • It's Invited Academic talk Season: Mar 15, Penn St; Mar 24, Monmouth;Apr 4, UMass
  • Apr 21, Singularity: More master classing
  • May 4, Sao Paulo: more to come
  • Upcoming: May 13, NYC; Jun 1, Chicago; Jun 29, UK; and Sep 10...Schenectady?!?

Vivienne L'Ecuyer Ming

Follow more of my work at
Socos Labs The Human Trust
Possibility Institute Optoceutics
Kennedy Human Rights Center UCSD Cognitive Science
Crisis Venture Studios Inclusion Impact Index
Neurotech Collider Hub, UC Berkeley UCL Business School of Global Health