Helpful AI Chatbots Weaken Human Behavior Simulation, Study Finds
TL;DR. A large-scale study reveals that training AI models for helpfulness reduces their capacity to accurately simulate human behavior, worsening with each generation. - Foundation models predict human responses better than their fine-tuned chatbot counterparts. - The effect persists across model families including Qwen, Llama, and OLMo architectures. - Researchers found even demographic profiling does not significantly improve individual prediction accuracy.
- Training LLMs for helpfulness paradoxically weakens their ability to simulate human behavior.
- Base models consistently outperform their post-trained, instruction-tuned variants in predicting human actions.
- This deficit worsens with each new generation of 'helpful' AI models.
- A large dataset (Psych-201) with 208,000 participants and 26 million responses informed the study.