AI can predict how you’ll respond to a survey. But that’s not the same as understanding you

Social scientists use experiments to understand why people change their minds or behaviour, and large language models like GPT-4 are now predicting the outcomes of many social science experiments with surprising accuracy.
A study led by Harvard researcher Ashwini Ashokkumar found a strong correlation between real experiment results and LLM predictions, suggesting LLMs might capture meaningful patterns in text-based human behaviour.
However, a warning has been issued that predicting human responses does not equate to understanding human behaviour, and “synthetic respondents” or “silicon samples” aren’t a direct substitute for actual human research.
Scholars warn that AI systems can create “illusions of understanding,” which look insightful but may reflect sophisticated pattern-matching instead of genuine insight.
While LLM-generated forecasts might assist in running cost-effective pilot studies before launching expensive experiments, combining LLM predictions with human data usually yields more accurate results.
There’s a risk of misusing AI’s predictive capabilities, especially in optimising harmful persuasion, and the reliance on proprietary language models limits the ability of other researchers to verify or repeat findings.
The overarching message is that while AI prediction tools like LLMs might be valuable for social science, genuine understanding and the insights drawn from real human experiences are still irreplaceable.

You may have missed