Human Factors

SUPERVISOR: MARLISE WESTERHOF, MSC

Enabling users to efficiently navigate through websites and intuitively gather information is a crucial element of user experience. Difficulties in accessing information does not only result in frustration for the user, but can also lead to far more serious consequences, e.g. when facing difficulties accessing health-related information on governmental websites. Thus, the common user’s mental model has to be considered when designing how information is structured on a website. If the navigation structure of a website matches the mental model of users, it helps users to find what they are looking for.

Mental models are abstract, inner representations that people have in their mind regarding things from the external world, such as objects or systems. In other words: a mental model is an explanation of how an individual thinks something works. Card sorting is a method to elicit mental models of specific domains, and is frequently applied for organizing the structure of informational websites in a way that it suits the user's expectations.

In this bachelor thesis assignment, you will apply the method card sorting to elicit user's mental model of a specific domain (e.g. E-government websites such as GGD/Health Services or municipalities). Based on the obtained insights, you will formulate design recommendations for a user-centred information structure for the chosen domain.

SUPERVISOR: JULE LANDWEHR MSC, SECOND SUPERVISOR: DR. SIMONE BORSCI

Background

Conversational agents, such as chatbots and voice interfaces, can be used for multiple purposes e.g., support customer experience with services. These new tools are growing and more and more integrated into systems such as websites, social networks, cars. Smart and AI-based conversational agents are shaping the future of human-computer interaction however little is known about how to assess people reaction and satisfaction after the use of these systems.

Goals

You will build upon previous work done on a new scale to assess satisfaction with chatbots. Your experimental work will focus on the evaluation of the satisfaction scale using card sorting to further streamline the validity of the scale. You will perform a card sorting using the items of the scale to collect data.

The current validated version of the scale is composed by 11 items, and it is available in Dutch, Spanish, German and English, and Italian (the last one is still under validation).

The target is to involve a large number of participants working (potentially) in a team to evaluate the validity of the satisfaction scale and possibly get some new insights into additional factors and items.

Requirements

You should be aware of the statistical methods regarding factorial analysis and for inferential statistics, and be able to use R for such purposes.

Previous work

Borsci, S., Malizia, A., Schmettow, M., Van Der Velde, F., Tariverdiyeva, G., Balaji, D., & Chamberlain, A. (2021). The Chatbot Usability Scale: the Design and Pilot of a Usability Scale for Interaction with AI-Based Conversational Agents. Personal and Ubiquitous Computing, 1-25.

Bos, M. A. (2021). Testing a scale for perceived usability and user satisfaction in chatbots: Testing the BotScale (Master's thesis, University of Twente).

Kerwien Lopez, S. M. (2021). Confirmatory Factor Analysis of a new Satisfaction Scale for conversational agents and the role of decision-making styles (Bachelor's thesis, University of Twente).

Kerwien Lopez, S. M. (2021). Confirmatory Factor Analysis of a new Satisfaction Scale for conversational agents and the role of decision-making styles (Bachelor's thesis, University of Twente).

Beerlage-de Jong, N., Kip, H., & Kelders, S. M. (2020). Evaluation of the perceived persuasiveness questionnaire: User-centered card-sort study. Journal of Medical Internet Research, 22(10). https://doi.org/10.2196/20404

Agarwal, N. K. (2011). Verifying survey items for construct validity: A two-stage sorting procedure for questionnaire design in Information Behavior Research. Proceedings of the American Society for Information Science and Technology, 48(1), 1–8. https://doi.org/10.1002/meet.2011.14504801166