An international group of researchers has developed a new tool which can help identify problematic randomised controlled trials (RCTs), including fraudulent studies, where there are serious concerns about trustworthiness.
The final version of the tool, called INSPECT-SR, is now published on the pre-print server medRxiV.
It was developed by a worldwide collaboration of more than 150 integrity and health research experts, led by Dr Jack Wilkinson from The University of Manchester
Funded by the National Institute for Health and Care Research (NIHR), it was developed in collaboration with the University of Colorado Anschutz Medical Campus and Cochrane, a not-for-profit organisation which is the world's leading publisher of health systematic reviews.
Some of the studies are subject to critical but honest errors, but many appear to be fraudulent.
Concerns are growing over the increasing numbers of problematic high-level summaries of the research evidence from randomised controlled trials , known as systematic reviews.
In 2023 alone, over 10,000 research papers issued globally were retracted by journals according to an analysis by Nature, many of which used evidence from problematic RCTs.
Dr Wilkinson warns problematic RCTs can result in medical research potentially being compromised, drug development hindered and promising academic research jeopardised.
INSPECT-SR is designed to root out problematic RCTs which publish faked or manipulated data or have Inadvertently made critical errors.
Some, written for a fee by outfits known as "paper mills", are entirely fabricated.
The tool guides users through a series of 21 checks, grouped into 4 domains:
- Post publication notices which express concern and retractions.
- conduct, governance, and transparency
- text and figures
- data discrepancies and statistical errors.
One of the most well-known examples of problematic RCT research was around claims the drug Ivermectin, hailed as a miracle drug that would save the lives of people with severe COVID-19.
However, some of the trials used to make the Ivermectin claims appear to have been fabricated, according health authorities in the US, UK and EU. Subsequent high-quality trials suggested little or no benefit.
In another example , the National Institute for Health and Care Excellence (NICE) reversed recommendation for a device called a fetal pillow, developed to assist caesarean sections, following the retraction of three clinical studies supporting it.
According to an article in BJOG: An International Journal of Obstetrics & Gynaecology, a trustworthiness assessment may have prevented the use of the evidence in the NICE guideline, as it contained statistical anomalies.
And trustworthiness concerns were also identified in a group of trials around the use of CBT and exercise to combat spinal pain. The trials had substantial impacts on clinical practice guidelines. Several have now been retracted.
Dr Wilkinson said: "When a systematic review is carried out, it includes all randomised controlled trials on a given topic.
"But historically, there has been no way to identify fraudulent or otherwise problematic RCTs, meaning that these studies are inadvertently included in systematic reviews.
"This is a big problem, as systematic reviews are very influential - they inform health guidelines for example.
"Most fraudulent RCTs are produced by individual researchers rather than commercial paper mills, but with the advent of AI I fear this is likely to become more of a problem in the future."
He added: "Academic papers are often assessed for quality before they are published. But reviewers do not ask the more fundamental question of whether the evidence they are reading is even genuine..
"But we anticipate that INSPECT-SR will become the standard for assessing trustworthiness of RCTs, especially as it has been created withCochrane for use in their systematic reviews of health interventions.
"However, it's important to stress that our tool is not merely a test for fraud and misconduct- though clearly many problematic studies are examples of that.
"It also tests for critical errors which is why our priority is to determine if a clinical trial should be used to guide healthcare decisions.
"Work is ongoing to develop more automated systems - perhaps using AI- to assist with this process. In the future, we hope to expand our work to detect problems in other forms of research studies, not just clinical trials."
- The paper INSPECT-SR: a tool for assessing trustworthiness of 1 randomised controlled trials is available on the print server medRxiV https://doi.org/10.1101/2025.09.03.25334905
