Too often, policy recommendations in climate-related research are either an afterthought or stray too far into advocacy, a major new analysis has found.
Researchers led by the University of Cambridge conducted a systematic review of over 3000 scientific papers focused on climate change mitigation, and found that recommendations of how to turn science into policy were either non-existent, failed to account for uncertainty, or were 'wish lists' disconnected from the study's findings.
In a period when trust in institutions is at historic lows and scientists are under increasing pressure to develop solutions for net zero at speed, the researchers say that making policy recommendations realistic, relevant and transparent is vital to help improve trust in science, while also making it more likely that robust science can be turned into effective climate and energy policy.
The team say that although developing meaningful policy recommendations is a vital part of the research process, many scientists are unfamiliar with exactly how policy is developed, which can lead them to make unrealistic recommendations. To improve the quality of these recommendations, the team say that training needs to be made available, and funders should include policy reporting as part of the funding process.
The researchers have also developed a tool that aims to help scientists when developing policy recommendations, so that any recommendations are realistic, tangible and tightly linked to the original research. Their results are reported in the journal npj environmental social sciences.
The study is part AI for Net Zero project, funded by UK Research and Innovation (UKRI), which is looking at how to use AI solutions to help with reaching Net Zero, and attempts to make research results more accessible to the complex world of making policy.
"Our first step in making research more accessible to policy makers was to understand how researchers in engineering and climate science make policy recommendations now," said lead author Dr Vangelis Danopoulos from Cambridge's Statistical Laboratory. "And we found that while the science is robust, the policy recommendations – how the science gets implemented – were too often treated like an afterthought."
Danopoulos and his colleagues created a tool, called Evidence Communication Rules for Policy (ECR-P) to assess the quality of policy recommendations in scientific papers published since 2019 in the areas of green energy and transportation. From an initial larger group of over 3000 publications, they then conducted a more thorough analysis of 23 papers.
The analysis identified three main types of problem:
- Levels of uncertainty were rarely disclosed or highlighted
- Advocacy language (e.g. "must be forbidden") would creep in
- Recommendations were disconnected from the study's actual findings (the 'wish list')
Danopoulos said that being clear about uncertainty in particular, is vital to help improve trust. "Being clear about the uncertainties, being clear about the trade-offs is so important if we're going to bring our science to the people, industries and governments who will be charged with turning that science into decisions," he said. "Highlighting what we don't know is just as important as highlighting what we do know."
One of the many challenges in turning science into policy is that while scientists are deep experts in a specific field, many of the people charged with designing policy have much broader remits. If these individuals do not have a fully accurate idea of what the risks and uncertainties are for a particular recommendation, then any resulting policy will be built on dubious foundations.
Danopoulos also said that scientists must be careful about including emotive language in their research. "A lot of the recommendations that fell into the 'wish list' camp seemed to be written from a point of view of whatever was topical at that point in time," he said. "As researchers, we have to be so careful about how we express our findings and how we make our recommendations."
As part of their analysis, the researchers also identified high-quality examples of policy recommendations. One paper about how to roll out electric vehicles in the UK treated policy recommendations as an outcome from the start, and not an optional extra, which meant the recommendations were of high quality.
Danopoulos said that their ECR-P tool could be one way to help researchers develop better policy recommendations, but that this is a multi-faceted problem that needs scientists, funders, institutions and policymakers working together.
"This work wasn't intended as a way to scold scientists, but rather to highlight that actually turning good science into good policy is a difficult and complex process, and it's one that many scientists simply haven't had training in," said Danopoulos. "Anything we can do to support scientists in that process will ultimately mean better science, and better policy to support the transition to clean energy."
The research was supported in part by the Engineering and Physical Sciences Research Council (EPSRC), part of UK Research and Innovation (UKRI).
