Responsible Prediction under Critical Uncertainty: An Epistemic Analysis of Neuroprognostic Innovation Practices After Cardiac Arrest
Due to the COVID-19 crisis the PhD defence of Mayli Mertens will take place (partly) online.
The PhD defence can be followed by a live stream.
Mayli Mertens is a PhD student in the research group Philosphy. Her supervisor is prof.dr. M. Boenink from the Faculty of Behavioural, Management and Social Sciences (BMS).
The means for prognostication of patients in coma after cardiac arrest are limited, and the outcome is always uncertain. This uncertainty poses a moral dilemma for family and professionals: discontinuing life-sustaining treatment risks that a patient with good neurological prospects dies, while continuing life-sustaining treatment risks that a patient with unacceptably poor neurological prospects survives. Innovation of prognostication is therefore driven by the desire to reduce the uncertainty of the prognosis, and thus alleviate dilemma. This thesis analyzes whether and, if so, how new technology, and the use of continuous electroencephalogram (cEEG) specifically, can be a responsible innovation of prognosis for patients in post-anoxic coma after cardiac arrest.
In order to determine under which conditions cEEG can be a responsible innovation in prognostication for patients in a coma after cardiac arrest, Mayli investigated the concerns of those involved (relatives, healthcare professionals, and ex-patients) in current prognostic practice. She also explored to what extent and how cEEG could address these concerns, and whether new problems could arise. Based on that exploration, Mayli formulated points that required attention for responsible innovation and made suggestions for how to promote responsible development of cEEG. Many of the concerns are not specific to cEEG, but also relevant to other innovations in neuroprognostics.
Chapter 1 sets Mayli’s research against the background of recent scientific discussions about 'responsible innovation', or 'responsible research and innovation' (RI). Mayli justifies her approach to RI and explains why (and to what extent) it differs from the existing approaches. In those existing approaches, three assumptions are recurring: (1) Emerging technologies must be assessed for their radical novelty and unpredictability. (2) An early assessment is necessary to influence the innovation trajectory. (3) Anticipating the unknown is necessary to prepare for the unpredictable. Mayli argues that these three assumptions do not hold for what she calls liminal innovation practices: innovations defined by a continuous back and forth between experiment and implementation, between research and practice, and between past and future innovations. First of all, (1) technologies that are central to liminal innovation practices have different characteristics than those typically attributed to emerging technologies. Moreover (2) the liminal innovation trajectory is different, so that continuous evaluation and design is still possible long after implementation. Finally (3) these differences require a reorientation in RI approach. Instead of anticipating the unknown and uncertain, responsible liminal innovation should return to observation of the known and predictable.
Accordingly, Chapters 2, 3 and 4 are based on extensive empirical research of liminal innovation practices; prognostic practices in ICUs where cEEG monitoring was performed in the context of scientific research. Mayli observed two such ICUs in the Netherlands and followed the care of more than twenty patients in coma after cardiac arrest. For four of those patients (of whom the prognosis and outcome differed), she interviewed five people involved, including one to three relatives, two or three care providers, and where possible the patient themselves. In addition, she also conducted observations and interviews in a liminal innovation practice in an ICU in the United States, and in various locations in Israel. This comparative fieldwork helped put the Dutch practice in perspective.
In Chapter 2, Mayli uses her observations and interviews to analyze in more detail the clinical dilemma she described in the introduction. She lays out how those involved experience this dilemma, and explain what concerns and questions it raises for them. In practice, prognosis is followed by the choice to (a) discontinue treatment, (b) continue, or (c) advance/delay that decision. This choice appears to be based on a careful consideration of possible harm to the patient, the patient's family and society as a whole. However, Mayli’s empirical research shows that all of these considerations depend not only on the reliability of the predicted outcome, but also on whether that predicted outcome correctly reflects the disadvantages and advantages at stake. Her analysis of stakeholders' concerns and how they weigh them in decision-making shows that avoiding potential harm to a given patient requires answers to two very different questions: (1) what outcome can be achieved and (2) will that result is acceptable? However, efforts to reduce neuroprognostic uncertainty have focused solely on answering the first question.
In chapter 3, Mayli links the two questions to discussions about medical futility and goes deeper into how those involved in clinical practice try to answer the second question: which outcome (which quality of life) is acceptable? The decision whether to discontinue life-prolonging treatment of a patient in coma is usually motivated by the consideration that the treatment is ‘medically pointless’. In the medical and ethical literature on medical futility, a distinction is made between physiological and normative (or quantitative versus qualitative) futility. Uncertainty regarding physiological futility is widely recognized and neuroprognostic tests have been developed to reduce that uncertainty. However, empirical evidence shows that judgments about normative futility are often just as uncertain: there is uncertainty about how the outcome will be experienced and evaluated by the patient. Moreover, it is uncertain how environmental, contextual, and institutional factors can influence that experience and evaluation. Mayli therefore argues that any serious attempt to reduce neuroprognostic uncertainty should clearly distinguish between: (1) uncertainty whether the proposed treatment can physiologically achieve the desired effect (physiological uncertainty) and (2) uncertainty whether the physiological effects of the treatment constitute an acceptable outcome (normative uncertainty). She concludes that the latter uncertainty can render normative futility judgments so indeterminate that it is more accurate to say that one must predict normative considerations rather than identify them. Thus, Mayli argues that neuroprognostic practice requires not one, but two predictions. Different parties may also provide different expertise to inform such divergent forecasts.
As mentioned, new prognostic tests such as cEEG are being developed with the ambition to reduce the clinical dilemma in decision-making about the treatment of patients in coma after cardiac arrest. In Chapter 4, Mayli investigates how plausible it is that the use of cEEG monitoring will make it easier to decide whether or not to discontinue life-prolonging treatment. To do this, she maps out the effects of cEEG as it becomes part of a social-technical network in an intensive care unit. The analysis focuses on (1) the way test results are constructed, (2) the kind of decision support these results provide, and (3) how cEEG affects communication between professionals and family members. Mayli argues that in some cases cEEG can indeed remove or reduce uncertainty, but that the technology also raises new concerns. Moreover, she points out that the actual effects of cEEG in practice depend on how cEEG is designed and implemented in practice. For a responsible implementation, in the further development of cEEG, for example, more attention should be paid to the reliability of the interpretations of the EEG signal and to the influence of the user interface on the interpretation and communication of outcomes. Furthermore, cEEG also creates a third, 'grey' outcome category: EEG patterns that are neither clearly favorable nor clearly unfavorable. More research into patients with this outcome is desirable. A number of recommendations are not specific to cEEG, but relevant to all neuroprognostic innovation. For example, Mayli recommends being more transparent about the evaluative nature of outcome measures and separating the evaluative assessment of possible outcomes from the physiological outcome measures as much as possible.
Chapter 5 focuses on the question of what the preferences of Dutch citizens are with regard to dealing with physiological and normative uncertainty in neuroprognostics. What prognostic information would they like to receive, how should decisions about whether or not to discontinue life-prolonging treatment be made, and what do they consider an acceptable quality of life after coma? Under the leadership of colleagues from the Health Technology Assessment department and in collaboration with colleagues from the Clinical Neurophysiology, a survey was developed that was distributed among a representative sample of 500 Dutch adults. The results of this study indicate that the current way in which prognostic information is used to predict what is physiologically and normatively futile is not fully in line with the preferences of the respondents. With regard to physiological futility, half of the study population considers discontinuation of treatment in patients in a coma after cardiac arrest unacceptable based on uncertain physiological prognostic information. With regard to normative futility, more than half of the study population prefers to remain alive in conditions which are considered a 'poor' outcome and are thus grounds for discontinuation of treatment. This indicates that the way in which prognostic information is currently used in Dutch ICUs should be reconsidered if we are to ensure the responsible use of new prognostic tests.
In the last two chapters, Mayli focuses on the uncertainties associated with self-fulfilling prophecies, i.e. the phenomenon that predictions produce their own truth. The potential for self-fulfilling prophecies (SFPs) in medicine is widely recognized as claims about the effect of the treatment or about the expected condition of the patient can have a self-fulfilling effect. In neuroprognostication, poor prognosis often leads to a decision to discontinue treatment which leads to the death of the patient, thereby confirming the poor outcome. The initial prediction has thus come true but it is uncertain whether this would have happened anyway, or whether this is solely the result of the discontinuation of treatment. In the medical and bioethics literature on SFPs, interest so far has focused on self-fulfilling predictions that alter the treatment’s effect or patient’s outcome.
Chapter 6 presents an alternative analysis of SFPs in medicine. Central to this is the argument that SFPs are always problematic, regardless of whether they change the outcome or not. Because SFPs are always true, the quality of the prediction is often not even questioned. But even if one were to try, one cannot check retrospectively, counterfactually, to what extent the basis of the prediction was correct. This is because it is difficult to distinguish between ‘transformative SFPs' that alter the outcome and 'operative SFPs' that do not. Based on this analysis, Mayli makes two points. First of all, since they always involve their own fulfillment, SFPs do not produce reliable feedback. Not being able to determine whether the basis of the prediction was correct or flawed means that one cannot learn from failure, nor from perceived success. This is the central epistemic flaw of SFPs. Second, Mayli argues that this has implications with regard to responsibility of researchers and decision-makers. Her point is not so much that SFPs leave room for forecasters and decision-makers to avoid responsibility for the consequences of their prediction; there may be good reasons to still use the prediction as a basis for decision-making. More problematically, because of SFPs, forecasters and decision-makers avoid responsibility for their failure to learn.
In chapter 7, Mayli investigates the implications of the analysis from chapter 6 for SFPs in neuroprognostics. Adequate evaluation of neuroprognostic practices requires, first, properly identifying all SFPs, including those that do not alter patient outcome. Such an evaluation means that one no longer focuses solely on false-positive test results, but on the influence of all positive test results, true or false. Second, it requires identifying two different SFPs within neuroprognostics; one due to physiological uncertainty and one due to normative uncertainty. A prediction that the physiological outcome will be poor means that the outcome is actually poor, regardless of whether it would otherwise be. A normative prediction that the physiological outcome will be poorly experienced increases the likelihood that it will be poorly experienced. Conversely, a prediction that the outcome may be perceived as good will increase the likelihood that it will indeed be perceived as good. Finally, analysis of the effects of physiological and normative SFPs in both acute and chronic care at two interrelated levels is necessary; first with regard to the medical outcomes for individual patients, and second with regard to the consequences for all members of the patient group. Mayli argues that, apart from sometimes resulting in undesirable outcomes, SFPs hinder learning. This makes decision-makers prone to repeat the same mistakes, both with regard to physiological and normative predictions of medical futility. Automated prediction through machine learning techniques can further exacerbate these problems. Mayli shows how SFPs as a result of physiological and normative predictions jeopardize the quality of medical practices, and significantly complicate innovation and policy. Finally, she proposes interventions to identify misprognosis and break the resulting feedback loops in research.
In the conclusion, Mayli reflects on the findings of this study and their broader relevance. She does this by providing concrete answers to the research questions. As motivation for her main question 'how to responsibly innovate neuroprognostics for patients in post-anoxic coma after cardiac arrest', she first briefly outlines the problems that arise in practice. Because treatment decisions based on prognostic information are hindered by uncertainty, work is underway to innovate neuroprognostics. In order to analyze what effect cEEG-based prognosis could have on current practice, Mayli provides an overview of the existing concerns of those involved in the care of patients in post-anoxic coma in the Netherlands. This shows that those involved find it difficult to make decisions based on uncertain predictions about the neurological outcome. Moreover, there is also uncertainty about how that outcome will be experienced by the patient. Clear communication is essential in this challenging process. After the analysis, Mayli summarizes her advice on how best to develop and implement forecasting based on cEEG, so that it takes the existing concerns and possible undesirable effects into account as much as possible. Much attention should be paid to who interprets the cEEG signals and how, and how these interpretations are communicated. Finally, she elaborates on the broader relevance of her findings. First, she shows how her insights on liminal innovation practices are not only relevant for innovation in clinical settings in general, but also for innovations in other domains. Then she shows why her analysis of self-fulfilling prophecies is also relevant for practical and automated predictions in general. A final overarching remark concerns the general importance of persistent uncertainty in forecasting and how different types of uncertainty can play a role. She concludes that refraining from making predictions about health and (risk of) disease can sometimes offer a wise alternative. Finally, Mayli indicates which of her recommendations apply not only to cEEG, but to neuroprognostic innovation in general. These are mainly epistemic issues. First of all, it is necessary to separate physiological and normative uncertainty in both research and care. It is crucial to no longer express outcomes in evaluative terms (e.g. favourable/unfavourable or good/bad), but as much as possible in descriptive terms (e.g. CPC or GOS categories). Second, self-fulfilling prophecies in predictions of both physiological and normative futility must be seriously addressed. Mayli emphasizes the importance of learning and the assessment of such learning as the cornerstone of responsible innovation.