Prediction vs fortune-telling: Methodological considerations to avoid the latter
Stillbirth is over-represented in lower and lower-middle-income countries, spurring research into prediction models. Naturally, predicting rare outcomes is as tricky as finding a needle in a haystack… during a snowstorm. Stillbirth prediction is particularly challenging because only part of the population is represented - unrecognised pregnancies and miscarriages are typically excluded - and the consequences of such selection are not well understood. How can the researcher avoid fortune-telling? Firstly, identify whether the model is for explanation versus prediction – as they are different. Next, acquire a sufficiently large representative sample. That should be obvious. Then, internally and externally validate the results while reporting standard prediction metrics (sensitivity, specificity, positive and negative predictive values, AUC, accuracy). Finally, do not overstate results. Do not simply report accuracy. It is easy to maximise accuracy of stillbirth prediction. To do so, simply predict all births as live births. Of course this results in a useless prediction tool. Do not ignore specificity. Specificity is the ability to predict the livebirths and is important because there are many more livebirths than stillbirths and future screening may be expensive. Do not ignore positive predictive values. Of those who test positive, the proportion that are actually stillborn is important as the test could increase psychological distress among patients and introduce additional iatrogenic morbidities from over-treatment. These issues are discussed in relation to a study on stillbirth prediction in India. See reference below for more information.
References