Scientists from prestigious universities and large research groups are significantly more likely to have their research published in general, top-tier scientific journals. Meanwhile, authors based in China and those studying politics, economics, gender or other social issues face a significant disadvantage.
These are a few takeaways from a sweeping new University of Colorado Boulder-led study, published July 15 in the journal Science Advances.
The analysis of more than 110,000 manuscript submissions over five years offers the most comprehensive glimpse yet at who makes the cut, and who doesn't, during the rigorous editorial and peer review process at elite science journals.
"These journals play a special role in the science ecosystem, shaping what ideas get attention, inform society and shape policy," said senior author Aaron Clauset, a professor of computer science and researcher with CU Boulder's BioFrontiers Institute . "Yet very few studies have been able to open the black box and take a look at their review process."
The findings raise an important question: To what degree are important scientific discoveries going unnoticed because their authors lack certain characteristics?
By calling out potential biases and making their dataset available to the public, the authors also hope to help journals improve their processes.
"With the rise of AI slop and the ability to mass produce junk science, the role of established gatekeepers like these elite journals becomes more important," said first author Sam Zhang, an assistant professor of mathematics and statistics at University of Vermont who earned his doctorate in applied math from CU Boulder.
Probing the 'science of science'
For a decade, Clauset has studied what he calls "the science of science," scouring data to paint a clear picture of who enters (and leaves) academia, who gets tenure and whose ideas tend to spread.
"There is no science without scientists, so it's important to know who has the opportunity to be 'in the room' so to speak, and who does not, to make discoveries," said Clauset.
For the new study, Clauset and Zhang were joined by Dan Larremore, associate professor of computer science at CU, and Nick Laberge, then a doctoral candidate there. They analyzed anonymized manuscript records for 110,303 submissions to Science and Science Advances between 2016 and 2020.
Unlike topical journals, general interest journals cover everything from health to space to politics and are regularly covered by the mainstream media. They strongly influence careers, with grants and tenure often tied to publishing there.
The competition is stiff. In 2023 alone, Science received more than 15,000 submissions.
As a result, such journals have a two-tiered review process: In-house editors make a first pass and then enlist outside subject experts to make suggestions and then accept or reject.
Over five years, the study found, Science accepted just 6.1% of submissions; Science Advances accepted 10.5%. About 83% of submissions to Science and nearly 75% to Science Advances were screened and rejected by in-house editors prior to undergoing peer review. It also found that:
- Papers with 10 or more authors are three times as likely to be published than papers with 1–5 authors.
- Manuscripts with corresponding authors from the most prestigious institutions had an 11.6% acceptance rate, compared to 3.4% for those from the least prestigious institutions.
- Studies with authors from China were more than three times less likely to be published than papers with U.S.- or Canada-based authors. Notably, authors with Chinese names, but based in the U.S., also faced a disadvantage.
- Even pre-COVID, research related to viruses, infection, RNA and immune response had a six-fold greater likelihood of being accepted than research on politics, economics, gender and the societal dimensions of health.
- Work by female corresponding authors was slightly less likely to be accepted. Had this trend not been there, Science would have published about 23 more papers by women per year.
"The fact that we find only a very marginal association with gender is in some sense reassuring," said Clauset. "But the number is still not zero."
Are editors and reviewers biased?
The authors stress that the findings do not prove that journal editors or reviewers are inherently biased. Instead, they shine a light on questions they and other researchers should explore.
For instance, Clauset notes, the prestige association was glaring throughout the editorial review and peer review process. Are reviewers inherently biased toward papers that come from Harvard or Stanford, or are those papers truly of better quality?
Are journals somehow biased against Chinese-based researchers, or do Chinese government incentives and other factors prompt China-based authors to submit an inordinately high number of papers?
In the spirit of transparency, the American Association for the Advancement of Science (the journals' parent organization), volunteered to share the data, as long as the team developed a way to anonymize it.
"The current evidence suggests, overall, these journals seem to be doing pretty well at just treating the science as the science. But this is not a totally clean bill of health," said Clauset. "In the current environment where the integrity of science is being attacked in bad faith, we need to be thinking about whether the processes at these journals live up to the ideals the scientific community has set for them."
