simulation modeling to optimize personalized oncology
Koen Degeling is a PhD student in the research group Health Technology and Services Research (HTSR). His supervisor is prof.dr. M.J. IJzerman from the faculty of Behavioural, Management and Social sciences (BMS).
Many novel cancer therapies and biomarkers have been developed and adopted over the last two decades. Treatment landscapes in metastatic cancers now typically comprise different targeted and immunotherapies across multiple treatment lines, with several predictive and prognostic biomarkers available to stratify patients into subpopulations to maximize clinical efficacy and patient health outcomes. These advances have paved the way for personalized oncology, aiming to identify the best treatment for each patient individually based on specific patient and disease characteristics. Although these developments have contributed to improved health outcomes, they are also associated with considerable cost. These cost are not only monetary in terms of excessive cancer drug prices, for example, but also in terms of increased complexity regarding the selection and combination of therapies and biomarkers across multiple lines of treatment. Patient-level simulation modeling methods have potential to address these challenges, as simulation models are able to estimate the health and economic impact of different treatment strategies based on specific patient and disease characteristics. Despite their clear potential, several questions regarding the use of patient-level simulation modeling methods haven been raised. For example, in what scenarios should patient-level simulation methods be applied? How should these methods be implemented? And how can challenges regarding their use be addressed?
Aiming to make a substantial contribution to answering these questions, the objectives of this thesis were to review the current status of health economic modeling in personalized medicine (Part 1), provide guidance regarding the implementation of patient-level simulation modeling methods (Part 2), address challenges regarding the computational burden of simulation models (Part 3), and illustrate the potential of using simulation models to optimize personalized oncology pathways (Part 4). Although these aspects are applicable to all patient-level simulation modeling methods in general, focus in this thesis was on discrete event simulation.
As a starting point, Part 1 (Chapters 2, 3, and 4) of this thesis provided an overview of the current status of simulation modeling in personalized medicine, including comparisons of different simulation modeling methods.
The review presented in Chapter 2 showed that published health economic simulation models that were classified as personalized, only represent the complex dynamics of current clinical pathways to a limited extent. The use of patient-level simulation modeling methods to model these pathways also showed to be limited, as only 6 out of 31 identified models were developed using patient-level methods. A lack of incentives and guidance for developing personalized simulation models, as well as the complexity of implementing and understanding patient-level simulation models, were amongst reasons discussed to explain these observations.
Chapter 3 compared discrete event simulation to cohort-level discrete-time state-transition modeling in a case study on metastatic colorectal cancer treatment. This case study estimated the cost-effectiveness of capecitabine and bevacizumab maintenance treatment for patients with stable disease or better after six cycles of capecitabine, oxaliplatin, and bevacizumab induction therapy, compared to an observation strategy. Although the discrete event simulation was found to provide slightly more accurate time-to-event predictions, incremental cost-effectiveness ratios were similar: €172,443 and €168,383 per quality adjusted life year gained for the state-transition model and discrete event simulation, respectively. This study additionally found that the discrete event simulation represented a patient flow more naturally, as the parametric distributions used to populate this model were less prone to irregularities and more straightforward to interpret compared to the time-dependent transition probabilities used to populate the state-transition model.
Chapter 4 illustrated how discrete event simulation and timed automata, a modeling paradigm new to the field of health economics, can go beyond traditional health economic simulation modeling methods and transparently model the health and economic impact of preference-sensitive treatment decisions. This case study investigated the potential impact of using circulating tumor cell analysis for informing treatment switching decisions in metastatic castration-resistant prostate cancer, compared to current practice in which prostate-specific antigen and bone scans are used as response markers. It was estimated that first- and second-line overtreatment could be reduced by 6.99 and 7.02 weeks by applying circulating tumor cells as response marker, for the timed automata and discrete event simulation model respectively. At a willingness to pay of €50,000 per quality adjusted life year gained, this translated to estimated net monetary benefits of €1033 and €1104, respectively. Implementing interactions was more straightforward in the timed automata model, which benefitted from its compositional structure. However, there were several software specific issues to this modeling technique and, hence, discrete event simulation was considered to be the preferred method.
Aiming to address challenges regarding the implementation of patient-level simulation modeling methods, Part 2 (Chapters 5, 6, and 7) provided methodological and practical guidance for the application of these methods.
The study presented in Chapter 5 illustrated the importance of appropriately reflecting parameter uncertainty (i.e., second-order uncertainty) when parametric distributions are used to reflect stochastic uncertainty (i.e., first-order uncertainty) in patient-level simulation models. This study compared two approaches to reflect second-order uncertainty in distribution parameters based on a simulation study and a case study. An approach based on non-parametric bootstrapping was found to be preferred over an approach using multivariate Normal distributions. An important benefit of the non-parametric bootstrapping approach was that it maintains correlations not only between distribution parameters, but also between different distributions estimated based on the same dataset. Results from this study also highlighted that neglecting to adequately account for parameter uncertainty in patient-level simulation models, will result in underestimation of the total amount of uncertainty surrounding simulation outcomes and may result in cost-effectiveness point-estimates to be biased.
Chapters 6 and 7 provided methodological and practical guidance for implementing competing risks in discrete event simulations, by illustrating and comparing different approaches to do so in a simulation study and a case study. Both studies found substantial differences between modeling approaches, both with regard to the approaches’ performance as well as in health economic outcomes and the uncertainty surrounding these outcomes. More specifically, Chapter 6 investigated the implementation of competing risks in discrete event simulation models based on uncensored individual patient time-to-event data. This study showed that the preferred modeling approach is to sample the time-to-event first from a single multimodal distribution representing all competing events, and to subsequently sample the corresponding event using a (multinomial) logistic regression model. However, a more straightforward (to implement) modeling approach that uses event-specific probabilities and distributions to sample an event first and the time-to-event second, showed similar performance when sufficient observations were available. Chapter 7 demonstrated and compared modeling approaches based on censored time-to-event data. This study showed that the presence of censoring resulted in too much complexity to make a general recommendation, as different approaches showed to be preferred depending on the level of censoring and overlap between competing time-to-event distributions.
Part 3 of this thesis (Chapters 8, 9, and 10) focused on the use of metamodeling methods to address challenges regarding the computational burden of simulating individual patients, which has been an important barrier to the application of patient-level modeling methods.
Chapter 8 presented a review on the use of metamodeling methods in health economics, which only found 13 previous applications. The study showed that computational issues are applicable to both patient-level and cohort-level simulation models, as 7 out of 13 studies used cohort-level methods to develop the original health economic simulation model. This confirms that not only the model type itself, but also the analysis performed determines whether runtime issues may occur. Although the number of applications in health economics was found to be limited, the applications that were identified highlighted the potential of metamodeling methods to perform computationally demanding analyses, such as probabilistic sensitivity analyses and value of information analyses.
Chapter 9 presented a structured overview of metamodeling methods for use in health economics, including considerations regarding their selection and implementation, to provide health economic modelers with essential tools for applying these methods. All identified methods were structured in a comprehensive six-step process: 1) identification of suitable techniques, 2) data simulation, 3) metamodel fitting, 4) performance assessment, 5) analysis execution, and 6) results verification. This process does not only guides modelers in the selection and application of metamodeling methods, it also provides guidance on what should be communicated about when reporting metamodeling studies.
Chapter 10 presented a case study on colorectal cancer screening in the Netherlands, which further illustrated the potential of using metamodeling methods in health economics. In this study, a metamodel was used to address computational challenges with the validated Adenoma and Serrated pathway to Colorectal CAncer (ASCCA) model, to allow optimization of the Dutch colorectal cancer screening program, while accounting for colonoscopy capacity constraints. Results from this study suggested that for a maximum of 550 colonoscopies lifelong per 1,000 individuals, which reflects current colonoscopy capacity in the Netherlands, the optimal screening strategy is to perform 21 biennial screening rounds, starting at age 33 years and using a fecal immunochemical test cut-off of 150 ng/ml. This strategy is expected to yield 0.086 life years and savings of €361 per individual compared to a scenario without screening. Although these results suggested screening should start at an earlier age compared to current practice, which is to start screening at age 55, strategies involving screening at relatively young ages were extrapolations beyond the data used to populate the ASCCA model and they should, therefore, be interpreted with care.
Part 4 (Chapter 11) answered the remaining question regarding the potential of simulation models to evaluate and optimize personalized oncology pathways.
Chapter 4 already illustrated that patient-level simulation methods can be used to model and evaluate biomarker-based, preference-sensitive treatment decisions. Additionally, Chapter 10 illustrated how simulation modeling methods combined with metamodeling methods can be used to optimize screening strategies while accounting for resource constraints. To further illustrate their potential, Chapter 11 applied simulation modeling methods based on real-world registry data to inform treatment targeting decisions in metastatic colorectal cancer. This study showed that health outcomes of patients could be substantially improved by selecting the treatment most likely to be effective in terms of progression-free survival, based on specific patient and disease characteristics. By targeting a different treatment for 219 patients, which represented 25% of the complete cohort, median progression-free survival for these patients was estimated to improve significantly from 175 days to 269 days. For the complete patient cohort, this resulted in an estimated improved median first-line progression-free survival of 288 days compared to 265 days as observed from the data. Although multivariable (survival) models were used to account for selection bias, these promising results obtained based on observational data should be interpreted with care.
This thesis has illustrated the potential of patient-level simulation modeling methods to address challenges regarding the economic burden and complexity of personalized oncology. It contributed methodological and practical guidance on the application of these methods, providing modelers with essential tools to adequately evaluate and optimize personalized oncology pathways. The case studies performed, provided insights that would have been very time and resource consuming to obtain by performing (additional) randomized studies, if feasible at all. Although it was illustrated that simulation methods can be used to identify strategies that improve health outcomes while making efficient use of scarce health care resources, further multidisciplinary research is needed to translate this conceptual and methodological work into practice. These efforts may identify for what type of research questions and under which circumstances patient-level simulation modeling methods are most valuable, how confidence in simulation models can be established, and which outcomes are key in optimizing personalized oncology pathways.
