AI generated scientific studies with AI simulated human participants new frontier in social sciences

Do we even need humans to study human behavior anymore or can we just simulate human response using AI? This is a question we soon must ask ourselves as AI technology takes new leaps forward. While opening up for research otherwise not possible, some suggest this is a dangerous path to walk.

The Study

Can AI and LLMs replace humans in social science studies to overcome limitations and biases inherent in traditional studies?

This is the question posed by the researchers, to which the answer is “kind of”. The researchers successfully managed to run the experiment with LLMs responsible for every step in the progress, introducing a new approach to testing scientific hypotheses.

They ran several different scenarios: a negotiation, bail hearing, job interview, and an auction. In all of the scenarios the LLM played all of the roles, of the tested, tester, analyzer, etc.

Implications

If we allow the system to act as both the test person and the ones running the tests, can we still claim to have tested our theories on humans?

While AIs and LLMs have become hauntingly good at impersonating humans, they are still not… human. Until we get AGI that can run experiments on us by itself, the AIs we create are simply as good as the data it’s tested on.

With that said, the LLMs of tomorrow (and to some extent of today) will simply have so much information about us to accurately predict our behavior better than we can. This will allow for incredibly easy and cheap access to automatically generated predictions and data on subjects that are hard or expensive to study right now.

The paper gives interesting insights into how social science research could be automated and improved in the near future, hopefully leading to many more interesting studies of ourselves as humans!

2024-04-26 11:07