Peer review is meant to prevent scientific misconduct: but it has its own problems

In 2023, Annals of the academic journal Operations Research retracted the entire special issue as the peer review process was compromised.

The incident focused on broader concerns about the peer review process in modern science. It showed that processes aimed at capturing research problems before publication could be in itself wrong.

And when it is done, it creates a huge ripple effect that undermines the integrity of scientific research.

So, how is peer review intended to work? Why do you sometimes fail? And what can you do to improve it?

The evolving process

Peer reviews are known to have occurred in the mid-20th century as the demand for specialized research continued to rise following the end of World War II. This was peer reviewed mainly by the editors of the Society and University Press Press, which he learned in contrast to the 18th and 19th centuries.

Today, peer reviews are primarily conducted by external peer reviewers who are asked by journal editors to conduct a review of manuscripts focusing on the quality and value of their research.

They are chosen from a pool of reviewers according to their discipline and their area of ​​expertise. Their tasks include ensuring that the paper is relevant to the purpose and scope of the journal receiving the paper, review the relevant literature, check the methodology, determine the importance of the findings, highlight areas omitted in the paper, and propose changes to improve the overall paper.

Traditional forms of peer review occur before the paper is published. Both the reviewer and the author remain anonymous.

Different areas take slightly different approaches to the review process. For example, double-blind peer reviews are preferred in the humanities. This involves two external peer reviewers reviewing the paper and sending the review to the editor where they will process the paper. The author responds to the recommendations of the editor and reviewer.

Based on editorial approval, this paper will proceed to publication.

Contrast this approach with open peer reviews that may occur before and after the publication of an article. Advocates of this approach say it promotes transparency and accountability.

Issues with current systems

Due to problems with the peer review process, the chronicle example of operations research withdraws the entire special issue. Springer Nature retracted a total of 2,923 papers from its large journal portfolio in 2024, citing issues of research and academic integrity.

A year ago, the Journal of Electronic Imaging retracted nearly 80 papers following an investigation into peer review fraud.

Such actions highlight many challenges to current peer review systems.

For example, academic workloads and institutional pressures on academics reduce the time they can spend as external peer reviewers due to the institutional pressure to produce more and more publications. It also prevents you from agreeing to become a peer reviewer in the first place.

This leads to what is called peer reviewer fatigue. In other words, reviewers do not have the ability to do any further reviews at this time.

Journal editors can prove that this reason is given. Reviewers who create high-quality manuscript reviews often receive more requests than they can respond from Journal Editors due to the time coefficients and their workloads, as well as the workload and institutional commitments mentioned above.

There is also the possibility of operating the peer review process. This can include fake peer review issues. This is the process in which the author is asked to suggest a reviewer and a fake email address and a fake peer review is submitted. There are indications that artificial intelligence is making this problem worse.

Plunder journals with questionable publication practices, such as charging fees for publishing articles, also publish low-quality articles that have not undergone a strict peer-review process.

In a guest post on the Academic Integrity website Retraction Watch, education researcher Richard Phelps denounced journal editors for not reviewing literature reviews of articles for accuracy. This post criticized the claims of rejection from researchers regarding the lack of previous research on this topic, and lower quality literature reviews have been reviewed more widely.

Enhance the process

There are ways that journal editors can enhance the peer review process for journals in relation to the quality of reviewers pools and the quality of reviews received.

Journals can regularly review current reviewers pools and broaden the pool by writing directly to the authors of recently published papers. You can also create a personal approach for researchers in this field to do reviews or add them to your reviews list.

The journal can also review the current guidelines of the reviewer to ensure that there is a consistent set of criteria that the reviewer can use to evaluate the paper and explain the reasons for the evaluation across key elements of the manuscript.

A “strength-based approach” for reviews can be encouraged. This is where the gap between the paper’s strengths and the paper’s gap makes the feedback more “developing” and less focused on what’s wrong with the paper.

From my experience as a journal editor, the author also finds it helpful to receive reviewer comments along with a general summary from editors that highlight the key issues raised by reviewers.

This article will be republished from the conversation under a Creative Commons license. Please read the original article.