Perceptions from the hematology community
A report commissioned by the American Society of Hematology. Prepared by Clarke & Esposito.
Anyone who spends any time in the world of academic and professional publishing hears a lot about how peer review is “broken.” Such claims are often followed by recipes for “fixes,” which range from the modest (double-blind review), to the unrealistic (investigating authors for undeclared conflicts of interest), to the extreme (replacing journals with preprints). But these claims are typically opinions (or sales pitches) based on anecdote and generalization. And so when the American Society of Hematology (ASH) approached us about conducting a broad survey gauging peer review attitudes and perspectives across the hematology community, we were enthusiastic.
Working closely with ASH and the independent research firm ReadEx, we developed a survey instrument and invited just over 25,000 individuals to participate. We received responses back from 8%, or 1,944 researchers. Although this research focused on one specific field (hematology), many elements may be generalizable to other medical specialties, as well as to STM publishing more generally. What they told us was illuminating. Far from a “broken” system, we found that researchers overwhelmingly hold positive views regarding journal peer review and consider it an essential and helpful aspect of scientific communication. 80% of respondents agreed with the statement “Scientific communication is greatly helped by peer review.” 88% disagreed with the statement “Peer review is unnecessary.” Asked how peer review improves the science of papers, respondents agreed that peer review resulted in “major points [made] more clear” (69%), “data presentation improved” (66%), “overstatement corrected” (64%), and “scientific errors identified” (59%). Only 10% of respondents indicated dissatisfaction with the peer review process used by journals in their field.
Respondents also affirmed a preference for selective journals, strongly agreeing that a function of peer review should be to determine originality (88%) and importance (81%) — a view at odds with the proliferation of “sound science” peer review journals and with the vision for scholarly communication exposed by Plan S.
When asked about peer review at different stages in the scientific communication process, journal-based peer review clearly remains the gold standard. When asked “To what extent do you agree that peer review at the following stages improves the quality of papers,” 82% of respondents answered affirmatively with regard to journals as compared with 42% for preprints and 40% for post-publication comments.
That said, while researchers greatly value journal peer review, they indicated there is room for improvement. Respondents highlighted a few areas where peer review is falling short. While 82% of respondents said that peer review should detect fraud, only 35% said peer review was actually good at doing this. Similarly, whereas 72% of respondents indicated that peer review should catch plagiarism, only 35% said this was presently happening effectively.
A surprising finding was the degree to which respondents were open to changes to standard peer review processes. 76% of respondents indicated a preference for double-blind peer review, as compared with only 34% for single-blind review, which remains far more common. There was also strong support (75%) for more collaborative processes whereby the editor discusses the paper with each reviewer and comes to some degree of consensus before sharing reviews with the author. Support for making peer review reports public was less strong (42% for vs 27% against), but given how few journals make peer review reports public, this was still a surprising finding.
ASH has generously agreed to make the survey findings public in the form of this report.
Download Report (PDF)
