This week: will the true social superpower please stand up! And, who benefits from AI?

Research Roundup

Creative Complementarity meets Pareto Principle

Another week, another paper showing the “benefits” of LLMs largely occurring to lower-skill workers…but wait…there’s nuance here...and the first research to show something truly different!

Let’s start with the impact of GitHub Copilot on open source development (via regression discontinuity). As with every other paper to date, the benefits of a coding assistant “are greater for individuals with relatively lower ability”. It seems that they spend less time project managing or collaborating, and more time coding alone with a bit more exploration. That is a mixed and messy bag. If higher-skill coders had shown this behavior I’d be very excited. With only lower-skilled coders showing the effect, less planning and collaboration sounds a bit like they are just following orders.

So where are experienced, high-impact individuals showing benefits from AI? When over 1,000 scientists were given randomized access to an AI tool for generating novel materials (think, a material sciences version of AlphaFold), “AI-assisted researchers discover 44% more materials, resulting in a 39% increase in patent filings and a 17% rise in downstream product innovation.” (Note: I’m skeptical of the “downstream” impact effects being measurable so soon. The “first” AI-assisted drug is still only in phase 2 trials.)

Most notably to me: “while the bottom third of scientists see little benefit, the output of top researchers nearly doubles”!!! The authors’ qualitative analysis suggests that “top scientists leverage their domain knowledge to prioritize promising AI suggestions, while others waste significant resources testing false positives.”

Finally, a use case that isn’t about AI just doing your work for you—compelling evidence of creative complementarity: AI’s boost to creativity over routine skills. And what of the scientists who aren’t seeing these benefits, who cannot see the forest through the AI-generated trees? “82% of scientists report reduced satisfaction with their work due to decreased creativity and skill underutilization”.

If only 20% of PhD scientists are reaping the benefits of creative complementarity, imagine how many unhappy workers we might find in the rest of the economy. We need more people to benefit from AI, and that isn’t a technology problem.

The perspective taking superpower

Years ago I published a paper in which I analyzed the writings of MBA students using (for the era) very fancy AI. One surprising finding was that simple word count strongly predicted a student's final grade in the course, almost as much as the content of their essays. Are social skills simply about bullshiting your way through life with an avalanche of words?

When people interact in challenging social scenarios, such as complex problem solving in teams, individuals with “stronger theory of mind demonstrated greater cognitive sensitivity and pragmatic conversational skills”. Interestingly, “only cognitive sensitivity [not conversation skills] subsequently predicted more favorable first impressions”. Being able to understand the thoughts and perspectives of others drives positive impressions.

And when perspective taking is lacking…it seems to drive zero-sum thinking. When people make judgements about commercial transactions, “buyers [are] consistently seen as less likely to benefit from transactions than sellers”. One of the principal drivers of this zero-sum bias is limited perspective taking, “failing to observe that people do not arbitrarily enter exchanges”.

Perspective taking is a superpower; conversational skills are just camouflage.

<<Support my work: book a keynote or briefing!>>

Want to support my work but don't need a keynote from a mad scientist? Become a paid subscriber to this newsletter and recommend to friends!

Weekly Indulgence

During lockdown, my research explored how collaborative documents drove the smartest teams. Today, AI-mediated collaboration can take the smartest teams to the next level. Read more in my guest post for Google.

...and read further when "How to Robot-Proof Your Kids" comes out next year.

Also, I need to start keeping count of when I'm kicked out of a venue by security. To be clear, I mean that I stay so late answering questions and talking with the audience that burly men with tasers chase us all out. +1 for Novo Nordisk.

Stage & Screen

• December 4-5, LA: A talk at RAND in Santa Monica.
  • Maybe more to come!
• December 10, NYC: It's that time of year again: RFK Human Rights's Annual Gala. Another year to support our amazing work in defending journalists and civil rights defenders around the world.
  • (It's sad that I must say this but...we have nothing to do with RFK Jr.)
  • There might also be an interview with...wait for it.
• Are you in NYC this December? Let's meet!

Does your company, university, or conference just happen to be in one of the above locations and want the "best keynote I've ever heard" (shockingly spoken by multiple audiences last year)?

SciFi, Fantasy, & Me

I watched the first season of Arcane with my kids. Here's one’s assessment: “Arcane is a visually striking exploration of the power struggle between Piltover and Zaun, mirrored in the tragic rivalry of Vi and Jinx. Though the character dynamics are strong, the central romance between Vi and Caitlyn is somewhat rushed. The worldbuilding shines in many ways, but the complexities of its interspecies society are largely unexplored. Despite these flaws, Arcane remains a gripping and immersive experience, with Season 2 offering the potential for even greater narrative depth. Despite its flaws, I’ve enjoyed Arcane and am soon to dive into Season 2.”

Vivienne L'Ecuyer Ming