Seminar: How to assess and clinical prediction model, [1] introduction

Part 1 Case discussion 

3-4 people in each group and discuss for 15 minutes

You are the academic committee of your hospital. A tech company comes to your boss, the director of the hospital, to promote a prediction model that can predict diabetes. The director asks you to listen and give her your opinion.

What would be your initial thoughts about this?

This excellent model is based on a large size, representative cohort study of 1300 participants in Taiwan, using age, sex, blood pressure, BMI, waist circumference, blood pressure, fasting blood sugar, and blood lipid profile, it can predict 10-year risk of having diabetes with a C-statistic of 0.7. Importantly, adding blood pressure and blood lipid profile substantially improved the performance. They told you C-statistic is like an area under the curve (AUC), and is a widely accepted standard that is used to assess clinical prediction model, and normally it is considered acceptable if C-statistic is above 0.7, and excellent if above 0.8. 

The model also calibrates well, which is another important aspect of a clinical prediction model. 

Separate models are made for male and female, and can be used to predict anyone with an age between 30 and 70 years old. 

The model is developed using the most advanced machine learning algorithm that maximises predictability and validity. It has been developed from a randomly selected half of the dataset and validated in another independent half of the dataset. An advanced machine learning model has been used to find the best combinations of factors that can make the best prediction. The similar kind of model has been used to predict consumer behaviour and targeted advertisement.

Part 2 Topics overview

The cookbooks of a risk stratification model

The role of a clinical prediction model

Performance indicators and their problems

Pathway thinking and benchmarking methods 

Workshop: basic number skills as a clinician