HomeNewsCritical assessment of prediction models needed

Critical assessment of prediction models needed

Thousands of prediction models are created each year, but only a fraction of these are applied in clinical practice. UT PhD candidate Tom Hueting examined which criteria prediction models must meet and how they are applied in breast and prostate cancer.

Clinical prediction models are statistical tools that can be used to help diagnose or predict a disease. For example, a healthcare professional can use them to predict lymph node metastases in patients with prostate cancer based on patient and tumour characteristics. Prediction models therefore have much to offer the healthcare sector. Hueting worked on a new artificial intelligence model to provide a personalised prognosis of the risk of breast cancer recurrence. Earlier research revealed that the number of follow-up examinations for breast cancer could be reduced by 9,000 per year if a prediction model is used to assess the risk of recurrence of the disease.

From 922 to 34 models

It takes time and effort to develop a clinical prediction model. During a systematic review, Hueting noticed that between 2010 and 2020, over 900 prediction models had been developed for breast cancer alone, but a large majority of them had a high risk of bias. Hueting validated models from that study against data from the Netherlands Cancer Registry. Of the 87 models he validated, 34 performed well enough to be applied to a larger patient group.

Six challenges

How come so few models can actually be applied in practice? Hueting identifies six challenges in his PhD thesis. He says that a good prediction model must be accessible, transparent, generalisable, maintained, and interpretable by both healthcare professionals and patients, and provide demonstrable added value. Only when all these requirements are met can a model be used in practice.

INFLUENCE: predicting breast cancer recurrence

One of the models Hueting specifically looked at is the INFLUENCE model. This model predicts the risk of recurrence after breast cancer treatment, and is used by the physician and the patient to plan a personalised follow-up route. In previous versions, researchers mainly looked at recurrence of breast cancer in the same breast, but did not consider the recurrence of contralateral breast cancer (breast cancer that recurs in the other breast). The new updated model does accurately predict this.

In addition, the model predicts the extent to which the breast cancer could spread to other organs. The online INFLUENCE model has subsequently been certified as a medical instrument. The practical application of the INFLUENCE model is currently being investigated in the context of a decision aid by the Santeon hospital group.

Lymph node models in prostate cancer

In addition to INFLUENCE, Hueting investigated sixteen models that predict the risk of prostate cancer spreading to the lymph nodes. In this study, he looked at their application in Dutch hospitals, and concluded that two models do this best: the Briganti nomogram (created in 2012) and the MSKCC web calculator. He also analysed the cost-effectiveness of the application of these two models.

Additional information

Tom Hueting is a PhD candidate attached to the Health Technology & Services Research (HTSR) department of the Faculty of Behavioural, Management and Social Sciences and TechMed Centre. His PhD thesis is entitled Developing, validating, and evaluating clinical prediction models in breast and prostate cancer and is available online

K.W. Wesselink MSc (Kees)
Science Communication Officer (available Mon-Fri)