DESCRIPTION
Beyond the Hype: The Promises and Perils of Large Language Models
LLMs like ChatGPT are becoming essential tools in our daily lives. From generating creative content to answering complex questions, their applications are vast and transformative. Understanding how they work and how to interact with them effectively is therefore crucial.
Despite their impressive capabilities, LLMs have significant shortcomings. A major issue is hallucination, where they confidently present false information as fact. Furthermore, they can perpetuate biases present in their training data, leading to skewed or discriminatory outputs. Crucially, LLMs lack genuine understanding of the world. They operate on statistical probabilities to predict the next word, not on a foundation of critical thinking or common sense.
Given these limitations, studying how humans interact with LLMs is vital. Users often exhibit an 'automation bias,' blindly trusting LLM outputs even when they are incorrect. Our research focuses on the psychological factors behind this, such as the level of trust, perceived credibility, and the tendency to view an LLM as a human-like conversational partner. Understanding these factors is key to designing better interfaces that encourage users to critically and consciously engage with LLMs.
This project investigates how various factors influence trust and task performance in interacting with LLMs.
Research topics include:
- The influence of specific factors, such as anthropomorphism, on trust in a LLM;
- The effectiveness of LLMs in supporting creativity or critical thinking;
- The persuasive abilities of LLMs in influencing users.
Keywords
LLMs, trust, influencing strategies, decision support
Method
Experimental research
Information
Please contact Lynn Weiher (l.weiher@utwente.nl) when you are interested in this assignment. The assignment is open to two students.
Literature
- de Visser, E. J., Monfort, S. S., McKendrick, R., Smith, M. A. B., McKnight, P. E., Krueger, F., & Parasuraman, R. (2016). Almost human: Anthropomorphism increases trust resilience in cognitive agents. Journal of Experimental Psychology: Applied, 22(3), 331–349. https://doi.org/10.1037/xap0000092
- Kumar, H., Vincentius, J., Jordan, E., & Anderson, A. (2025, April). Human creativity in the age of llms: Randomized experiments on divergent and convergent thinking. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (pp. 1-18). 2410.03703