#WeAreTogether: 'Uncertainty in pandemic forecasts must be communicated'
30 June 2020
Models guiding policies to contain the COVID-19 pandemic must communicate the uncertainty in their projections or risk bringing about ineffective measures, experts have warned.
In a comment piece published in Nature, 22 experts from several disciplines argue that models that discounting uncertainties might undermine social trust in science and convey a false sense of predictability, thus leading to potentially harmful consequences if used to guide policies in uncertain situations.
Dr Samuele Lo Piano, a post-doctoral Built Environment researcher at the University of Reading, said: "The uncertainty of available information should always be taken into consideration to ensure action is effective in the real world as well as in simulations. Incorporating possible variations in simulations means they produce a wider range of predictions, highlighting more clearly where they are uncertain."
The Nature piece references a recent study co-authored by Dr Lo Piano, which found that models predicting the amount of land that will need irrigating in future under climate change were more realistic when they incorporated uncertainty.
The authors have proposed a set of practices to increase the transparency and capacity of models to provide useful insights for policy-making. These guidelines combine statistical approaches with social norms that should be observed right from model design to the effective implementation of model-based policies.
Among their recommendations, the scientists argue that:
- models should make their assumptions explicit
- systematically account for uncertainties
- avoid overconfidence in projections
- acknowledge model limitations
They also ask modellers to acknowledge ignorance, a much needed virtue to prevent model end-users from making excessively adventurous claims on issues that might not have a clear cut fix.
Professor Andrea Saltelli, first author at the Center for the Sciences and the Humanities at the University of Bergen, said: "Modellers, decision makers and citizens need to establish new social norms such that modellers are not permitted to project more certainty than their models deserve, and politicians are not allowed to offload accountability to models of their choosing."