The New Foundations [RR]

Scientists are taking foundation models far beyond "predict the next word". This week we explore foundation models for health, weather, and people's choices.

Predicting Life Trajectories

Foundation models are breaking out of the text box to trace the trajectories of health, climate, and decision making.

A new model, Delphi-2M, adapts the GPT architecture to a far more complex and consequential system: the progression of human disease over a lifetime.

Trained on “0.4 million UK Biobank participants”, Delphi-2M doesn't just “predict the rates of more than 1,000 diseases” with high accuracy; it can generate “synthetic future health trajectories”, offering a glimpse into potential paths.

The real breakthrough, however, isn't just prediction. Explainable AI methods reveal the temporal "story" of disease—how one condition influences another over decades—providing a dynamic map of health instead of a static snapshot.

This represents a powerful new paradigm for precision medicine, though the model's learned biases remind us that even our biological stories are shaped by the societal data we feed them.

A Foundation Model for a Big System

The foundation model paradigm is proving its power far beyond language, now tackling the planet itself.

Aurora is a massive transformer model trained on “more than million hours of diverse geophysical data” to model the Earth's complex systems. It consistently outperforms traditional, computationally-intensive operational models in everything from high-resolution weather to air quality and tropical cyclone tracking.

Aurora both slashes computational costs by orders of magnitude and allows researchers and organizations to fine-tune their own copies. This makes it a powerful step toward expanding and diversifying Earth system science.

This isn't just a better weather report; it's a glimpse into a future where understanding and predicting complex dynamics—climate, culture, lives—becomes an accessible reality.

Perfect People Product

The foundation model paradigm isn't just for modeling the external world of health or climate; it's now being turned inward to model one of the most complex systems of all: us.

A deep neural network trained on the “over 90,000 human decisions across more than 2,400 procedurally generated games…predicts human choices with greater accuracy than leading theories of strategic behavior”.

While that’s impressive, the real breakthrough is the “interpretable behavioural model” derived for the original DNN. It reveals that our ability to be rational, strategic, and guess others' moves is “highly context-dependent”, shifting with the complexity of the "game" we're playing.

You and I aren't the same person from choice to choice. We change with context.

Transforming AI from a mere prediction tool into a partner for theory-building demonstrates the power of hybrid collective intelligence.

Vivienne L'Ecuyer Ming