This week's Socos Academy looks at the hardest rung of the socioeconomic ladder: the top.

Mad Science Solves...

This is a huge question. First because it calls up the complimentary question, “Who’s getting replaced by AI?” But even after we’ve decided the borders between substitution and augmentation—there truly are jobs which ought to go away as currently practiced—we are confronted with the question of augmentation. For me this one is just as crucial because a lazy choice today, the very choice we are daydreaming ourselves into, stands to rob us of our future capacity for some efficiency gains today.

The first set of research articles I’m sharing this week consistently reveal that LLMs principally boost the productivity of less experienced, junior workers. In some ways it levels the playing field, sharing hard won knowledge and skills from previous cohorts of workers. From call center workers to professional writers, LLMs boost productivity in lower-skilled workers without all of wasted time learning. But that, of course, is the very problem. If an AI is simply giving them the answers, they will never truly learn to do their job.

This isn’t simply a curmudgeonly bit of gloom from an AI doomer. First, I love to build machine learning models to explore challenging problems. I also regularly use Bard, GPT, and Stable Diffusion to enhance those explorations, but I never allow those tools to do the exploring for me. (For one, they can’t.) Second, there is already a substantial body of research on technology-mediated learning and AI tutors in particular. Rather than being the magic wand that changes every child’s life, along  with the fortunes of humanity, the research reveals that when technology gives learners the answers…they never learn. This is as true of adults using Google searches as kids with fully realized LLM tutors.

There is already a study on professional consultants showing that their work is both faster and higher quality using GPT, but that they are less likely to think critically and produce innovative solutions. An entire generation of scientists, lawyers, doctors, entrepreneurs, artists, programmers, and so much more might well find themselves trapped as perpetual novices.

We need to do better than looking at what tools joice productivity numbers on day 1. Instead, AI-augmented work should be true tutors, always challenging their users not to simply produce something quicker, but to be someone better.

This is one of our main new focus areas at The Human Trust: AI-augmented human development. From the planned relaunch of Muse, helping parents create lift in their childrens’ lives, to new machine learning technologies for maximizing collective intelligence in groups, our work isn’t about removing frictions and incresing productivity. It’s about better people.

Stage & Screen

Hello from LONDON! And not just London, but Birmingham and Aberdeen as well. New places...hurray!

• First, I'm speaking about AI & education at the University of Birmingham on the 15th.
• Then I give the keynote for the FT's Future of AI on the 16th. (Causal Reiforcement Learning (cRL) is part of my answer; More creative labor is the other.) Buy your ticket now!
• Finally, it's up to Scotland for my keynote at Aberdeen Tech Fest on the 17th.

After that...Thanksgiving, and then...New York City...twice! (November 30 - December 7): I'm back in NYC on my endless quest to find the fabled best slice.

• I'm talking "AI, Ethics, and Investments" for RFK Human Rights on the 30th.
• I'll be doing a two remote keynotes from NYC:
  • For Singapore and Manila on the 30th: Building better AI by Investing in People
  • For the UK on the 4th: Developing Excellence in Medical Education.
• It is AI, Design, and Ethnography on the the 4th.
• I'll be cogitating over the future of healthcare with the new ARPA-H on the 5th.
• And I'll be at the RFK Ripple of Hope Gala on the 6th!

I still have open times in NYC. I would love to give a talk just for your organization on any topic: AI, neurotech, education, the Future of Creativity, the Neuroscience of Trust, The Tax on Being Different ...why I'm such a charming weirdo. If you have events, opportunities, or would be interested in hosting a dinner or other event, please reach out to my team below.

<<If you are interested in pursuing an opportunity with Vivienne in or around these locations, please reach out ASAP!>>

Research Roundup

No Augmentation without Innovation

Does AI actually improve productivity? The initial results are in and the answer is yes, but the more important question is for whom?

When over 5,000 “customer support agents” were given an “AI-based conversational assistant” their productivity increased “by 14% on average”. The assistant also improved “customer sentiment” and “employee retention”, while reducing “requests for managerial intervention”. Notably, the AI assistant had the “greatest impact on novice and low-skilled workers” with “minimal impact on experienced and highly skilled workers”.

When ChatGPT was used to help “college-educated professionals” on “mid-level professional writing tasks”, time to complete the task “decreased by 40% and output quality rose by 18%”. Just as with the customer support agents, however, most of the productivity boost went to lower-productivity workers.

In an “experiment with GitHub Copilot”, developers “completed the task 55.8% faster than the control group.” Yet again, inexperienced workers received the biggest boost from AI assistance.

In contrast, AI assistants don’t appear to help radiologists. These doctors systematically “underweight the AI’s information relative to their own”. They just don’t update their beliefs optimally to give the AI feedback. And they ignore this feedback despite GPT-4’s notable success in passing the US Medical LIcencing Board Exam “by over 20 points”.

So we see large gains across a range of domains for novice, low-skilled workers the day they start using AI tools, but little to no benefit on day 1 for experienced professionals. This leaves me wondering, what do these productive gains look like on day 1,000 or 10,000?

These tools are very new, and their current value is about how they most immediately augment work. Experienced workers have a well-honed process that hasn’t involved an LLM or other AI; they have relatively little to gain without completely changing how they work. Inexperienced, lower-skilled workers, in contrast, stand to gain from all of the experience-derived insights transferred to them by the AI.

Because of this immediate gain, companies will look for that quick boost to lower-productivity workers. Already, the hunt for embarrassingly automatable tasks has begun with claims like “80% of the U.S. workforce could have at least 10% of their work tasks affected by the introduction of LLMs, while approximately 19% of workers may see at least 50% of their tasks impacted.” (Since when is “19%” an approximation?) This thinking is why the dominant design aesthetic in AI is substitution. Nearly every example I see is the lazy approach of people assigning a task to an AI and then either accepting or rejecting the product. This is similar to the modular concept of work as alignment and assignment. Whether between people or with AI, this is not augmentative collaboration.

And given decades of research on AI tutors in education, I would expert the quick productivity boosts to early-career  professionals with speed initial job learning but substantially slow long-term career development. When it just tells you what to do, learning stops.

More valuable models of AI collaboration and augmentation are out there. Using LMMs to simulate economic expectations, “measure the passage of time in fiction”, and even “accelerating science with human-aware artificial intelligence”. These uses lean into creative complementarity, the substantial productivity boost to augmenting creativity versus automating the routine. Read more examples in “Robot Proof”.

Augmented Collective Intelligence

What makes me excited about AI? Certainly not the seeming dominant trend of “what sull task can we automate away today?” It’s as inspiring as “an app for dry cleaning” or “nail salon scheduling on the blockchain”. What can transform us?

Two recent papers use AI to lift collective intelligence. In “Automating hybrid collective intelligence in open-ended medical diagnostics” leverages “semantic knowledge graphs”, NLP, and “SNOMED CT medical ontology” to pool unstructured diagnostic notes for groups of doctors and dramatically improve diagnostic accuracy. “While single diagnosticians achieved 46% accuracy, pooling the decisions of 10 diagnosticians increased this to 76%,” and this lift spanned “medical specialties” and doctor’s levels of experience. My model of augment collective intelligence dynamically weighs the contributions of collaborators, not just summing their contributions but also identifying with the minority opinion should drive innovation.

Augmented collective intelligence also needs to know who should collaborate and dynamically form teams to maximize collective intelligence. I wrote about this “match-maker” role in “Innovating Innovation”; so, I was thrilled to find a new implementation of the concept in “Scaffolding cooperation in human groups with deep reinforcement learning”. In the experiment, people played a “group cooperation game” for “for real monetary stakes” while an AI combining a DNN and a simulation model made “recommendations to create or break connections between group members”. This AI “social planner” nearly doubled corporations rates for augmented groups (77.7%) versus static networks (42.8%).

Balancing trust and diversity is one of the most challenging parts of developing collective intelligence. Dynamic link creation and deletion is a powerful tool for maximizing innovation and team productivity, and I’m so excited to see the concept implemented here. I also recommended using personality models of team members to improve matching, but the authors do identify a behavior in their “social planner” that I hadn’t considered. It encourages prosociality in defectors ”by moving them to small highly cooperative neighborhoods” where being an asshole just doesn't pay.

Making us better people—this is what AI can do.