Experts from several disciplines have raised concerns about the use of seemingly precise models to guide policies in matters as complex as the current Covid-19 pandemic. 22 scholars wrote down five principles on how to responsibly use scientific models. This comment was published in Nature on 24 June 2020.
They argue that models that disregard uncertainties undermine society's trust in science and convey a false sense of predictability. This leads to potentially harmful consequences to society if the model-based conclusions are used to guide policies in the real, uncertain world.
Dynamics
As Ismael Rafols, senior researcher at the Centre for Science and Technology Studies (CWTS) of Leiden University, puts it: "Once a number takes centre-stage with a prevailing narrative, other possible estimates can disappear from view. This may lead to problematic consequences as other options are marginalized. These dynamics can be observed in many issues from risk assessment to research evaluation".
Increase of transparency and suitability
The authors of the comment have proposed a set of principles to increase the transparency and suitability of models to provide useful insights for policy-making. These guidelines combine statistical approaches with social norms that should be observed throughout the process that goes from model design to the effective implementation of model-based policies.
The group argues that models should make their assumptions explicit, systematically account for uncertainties, tame model hubris and acknowledge model limitations. Multiple views on the issue of interest should be integrated, and keep stakeholders involved, to do justice to the complexity of the situation at hand. They also ask modelers to acknowledge ignorance, a much needed virtue to prevent end-users of the model from making excessively adventurous claims on issues that might not have a clear cut fix.
New social norms needed
First author Professor Andrea Saltelli, Center for the Sciences and the Humanities, (University of Bergen): "Modellers, decision makers and citizens need to establish new social norms such that modellers are not permitted to project more certainty than their models deserve, and politicians are not allowed to offload accountability to models of their choosing."
In the Responsible Evaluation thematic hub, CWTS brings together its expertise in the responsible use of peer review and scientometric indicators to evaluate scholarly research. The hub provides a basis for public discussion, education, and advise about responsible evaluation, targeted in particular at academic leaders, research managers, science policy makers, funding agencies, and publishers.
