Enhancing Agent-Based Models with Artificial Intelligence for Complex Decision Making
Shaheen Abdulkareem is a PhD student in the Department of Governance and Technology for Sustainability (CSTM). Her supervisor is prof.dr. T. Filatova from the Faculty of Behavioural, Management and Social sciences (BMS) and her co-supervisors are dr. P.W.M. Augustijn from the Faculty of Geo-Information Science and Earth Observation and dr. Y.T. Mustafa from the University of Dohuk.
This doctoral thesis was undertaken in collaboration with the University of Dohuk.
Complexity in human behaviour can play a crucial role in socio-environmental processes like disease diffusion. An example of such complex behaviour is risk perception, and behavioural change due to perceived risk. Computational models, and in particular Agent-based models (ABMs), have evolved as tools for simulating complex real-world processes.
ABMs consist of interactive adaptive entities and provide natural environment for simulating behavioural changes. ABMs often use naive deterministic algorithms, which are rule-based, to simulate change in agents’ behaviour. While agents in ABMs are sometimes endowed with memory, the actual learning in machine learning style is rarely implemented. The endogenous switching of expectations formation strategies using learning algorithms is underdeveloped in ABMs.
The goal of my PhD research is to systematically test the effects of implementing social and environmental intelligence on the dynamics and emergent outcomes of a spatial ABM. Spatial ABMs often use spatial data (GIS data) to construct real geographic environments in which agents are situated. Agents need to take changes in the spatial environment into account and adjust their behaviour accordingly. Understanding the learning processes of agents in the spatial ABM can assist in developing better strategies in problem-solving and coordination mechanisms.
Learning algorithms allow for richer agents’ architecture for operationalization of more realistic learning decisions beyond a simplistic treatment of agents’ cognitive and sensory capacities. Chapter two reviews recent ABMs that employ different learning algorithms to create intelligent agents with a focus on spatial ABMs. We provide a systematic structured analysis of 1) the specific operationalization of agent’s decision-making for various tasks, and treatment of spatial environment in the design of learning algorithms in terms of integrating social and spatial factors, and group and/or individual learning, 2) the reasons that motivate researchers to use learning algorithms in their models, and 3) the growth rate of integrating learning algorithms within ABMs. This chapter highlights the trends in the current practice of learning algorithms used to enhance ABMs.
Chapter three presents an innovative approach to extend agent-based disease models by capturing behavioural aspects of decision-making in a risky context using machine learning techniques. We illustrate it with a case of cholera in Kumasi, Ghana, accounting for spatial and social risk factors that affect intelligent behaviour and corresponding disease incidents. The results of computational experiments comparing intelligent with zero-intelligent representations of agents are discussed. We present a spatial disease ABM with agents’ behavior grounded in Protection Motivation Theory (PMT). Spatial and temporal patterns of disease diffusion among zero-intelligent agents are compared to those produced by a population of intelligent agents. Two Bayesian Networks (BNs), designed and coded using R, are further integrated with the NetLogo-based Cholera ABM. The first is a one-tier BN1 (only risk perception), the second is a two-tier BN2 (risk and coping behavior). Our results emphasize that learning allows a population of heterogeneous and spatially distributed agents to perceive risk and acquire and share knowledge via a social network about the effectiveness of various disease protection actions. This is especially relevant when studying cumulative impacts of behavioural changes and possible intervention strategies. As agents learn about the effectiveness of preventive measures in addition to learning to recognize risks, the society as a whole makes healthier and more cost-effective choices.
There is a difference between ABMs with pure social intelligence based on information exchange among agents and ABMs with integrated spatial intelligence. Spatial intelligence refers to the fact that agents sense their environment, perform a judgement on its dynamically-changing conditions, and adjust their behaviour based on this judgement. When spatial intelligence is used in ABMs, it often facilitates navigation (human or animal) or adaptation to land cover change. Fewer implementations are available for assessing risky situations engaging agents’ risk perception. In chapter three, agents evaluate changes in visual pollution (floating waste) in a river combined with personal information and media attention on cholera to decide which water source to use.
The work of chapter four is a continuation of the study presented in chapter three. It focuses on validating the spatial intelligence by collecting data on people’s risk perception for cholera via two online surveys: MOOC survey (Geohealth online course) and Google survey (an online survey). While most of the questions were identical in the two surveys, there was one difference. In the MOOC survey, participants chose to use or not to use river water for drinking through judging about its quality by the visual appearance (pictures shown). The Google survey collected information on the influence of individual risk factors on the willingness to use the river water using only textual description of the water quality situation. The risk perception of participants is questioned based on one factor and a combination of factors. Results from the survey confirm the fact that people judge quality of water visually, but also show the strong influence of media on risk perception.
Learning algorithms steer agent decisions in ABMs, serving as a vehicle for implementing behaviour changes during simulation runs. However, when training an ML algorithm, obtaining large sets of micro-level human behaviour data is often problematic. Information on human behaviour is often collected via surveys of relatively small sample sizes. Chapter five presents a methodology for training a learning algorithm to guide agent behaviour using a limited survey data sample. We apply different implementation strategies using survey data and BNs. By being grounded in probabilistic directed graphical models, BNs stand out among other learning algorithms in that they can be based on expert knowledge and/or known datasets. This chapter presents four alternative implementations of data-driven BNs to support agent decisions in cholera ABM. We differentiate between training BNs before, or during the simulation runs, using only survey data or a combination of survey data and expert knowledge. The four different implementations are then illustrated using the cholera ABM. The results indicate that a balance between expert knowledge and survey data provides the best control over the learning process of the agents and produces the most realistic agent behaviour.
Adaptive behaviour of agents is contingent on how well they learn about changes in disease risks and about coping options, individually or in interactions with others. The impact of different types of social learning compared to individual learning is an underexplored domain in disease modelling, and in agent-based models of socio-environmental systems in general. Chapter six pursues a quantitative test on the influence of agents’ ability to learn – individually or in a group – on the disease dynamics. Our experiments illustrate that individual intelligent judgements about disease risks and the selection of disease coping actions are outperformed by social intelligence (individually or leader-based). While the majority vote performs poorly here. Importantly, the choice of a particular type of individual or group learning in agents-based models should account for the nature and cultural norms of the society, for which epidemics prevention strategies are being tested.
