Shruti Kumar (name changed) is a professor at a medical research institute, working on diagnosing a neglected tropical disease that infects almost a million new people every year. Prof. Kumar said that with the increasing number of requests by scientific publishers to peer review research manuscripts in recent years, the process takes so much of her time that she has consciously decided to accept only requests that relate to a specific sub-discipline of her area of specialisation.
Indeed, as STEM journals rapidly expand globally and the pressure to publish more papers grows, the pool of qualified experts to review the papers is becoming increasingly strained, and top academic publishers are turning not to their peers but to artificial intelligence (AI) for help.
Nearly 15,000 years
Prof. Kumar said AI could help detect plagiarism: “Some publishers do percentage plagiarism checks before sending manuscripts to reviewers, saying for example, 40% plagiarism was detected with AI, and this is useful information for the reviewer and it cuts their time doing the same thing.”
In 2020, peer reviewers worldwide dedicated approximately 130 million hours, equivalent to nearly 15,000 years, to the review process, according to Deeksha Gupta, director, global strategy for society programmes at the American Chemical Society. This poses an immense burden to reviewers who must balance reviewing responsibilities with their own academic and research commitments.
But “the peer-reviewed journal system has been unable to adapt to and provide a suitable system that can leverage AI technologies while controlling for downsides,” per a recent paper published in The Innovation, co-authored by structural chemist Gautam Desiraju from the Indian Institute of Science, Bengaluru. The rate of annual data generation is outstripping that of the annually published number of academic articles, the paper added.
No replacement for humans
The global scientific community needs to experiment with alternatives to the peer-reviewed journal process “lest scientific productivity falls due to errors or oversight induced by AI-generated knowledge becoming accepted scientific findings,” the authors added.
“While AI cannot replace human reviewers or make final editorial decisions, it can play a valuable supporting role,” Dr. Gupta said. “AI-integrated tools can assist in accurately matching manuscripts with appropriate subject-matter experts and provide preliminary assessments during the prescreening stage. This ensures that only submissions of sufficient quality and relevance proceed to full peer review, thereby reducing unnecessary workload for reviewers.”
AI can meaningfully assist in peer review but only when leveraged responsibly for bounded tasks, Shane Rydquist, associate vice-president, delivery and solutions, Cactus Communications Ltd., said. AI can aid researchers’ workflows, allowing them to manage routine tasks such as literature searches and data organisation, identify subtle patterns in complex data, and surface unexpected connections between distant fields that a human might never encounter, he added.
Indeed, AI’s role should be to augment human expertise, for example in checking plagiarism and integrity, as AI excels at detecting text similarity, image manipulation, and data fabrication patterns, Dr. Rydquist said. It can also help with screening, assessing submission quality, formatting compliance, aligning scope, analysing expertise, identifying potential reviewers based on their publication history, detecting biases by flagging potentially problematic or biased language, and identifying conflict-of-interest patterns.
Dr. Rydquist added that “the key is augmentation, not replacement”.
Amplification risk
That said, one still needs human judgement in “evaluating conceptual novelty and significance; assessing methodological soundness in context; making nuanced judgements about appropriateness for a journal’s audience; and providing constructive feedback that advances science,” he added.
From a publisher’s perspective, integrating such AI systems is still in its developmental phase, according to Dr. Gupta: “Rigorous testing and validation are essential before these tools can be deployed at scale.”
But one concern raised in The Innovation paper is the risk of amplification of a mistake creeping into a machine-made summary, which could lead future authors to cite and proliferate the “fundamentally incorrect or misinterpreted science.”
This “will inevitably then reflect non-replicable papers, mostly without providing any clear indication of the same,” the authors wrote.
‘Not the only reliable source’
There are also biases in AI models that may be difficult to understand and control, like those stemming from choices of inclusion or exclusion in datasets, from assumptions in the algorithmic process, and from socioeconomic factors embedded in the operative institutions developing AI.
“Designing a system to counter this is a difficult, continuous undertaking for which the present peer-reviewed journal system seems ill-equipped,” the authors added.
“We’ve already come across cases where people have used generative AI to inadvertently propagate false citations, for example, instances where large language models (LLMs) sometimes generate plausible-sounding but non-existent references, which can create misleading chains of evidence,” Dr. Rydquist said.
When using any generative AI platform or tool, people tend to overlook that it may miss subtle technical errors that a human expert would catch immediately, Dr. Rydquist continued: “And LLMs have a tendency to over-represent highly-cited but potentially flawed work while underweighting newer, corrective research. This is why human oversight remains indispensable. Without critical evaluation, AI can quickly accelerate misinformation.”
A December 18 paper in Science suggested that despite both the excitement and concern about using AI in academia, “empirical evidence remains fragmented” and the impact of LLMs is not fully understood. The paper showed that LLMs have begun to “reshape scientific production” and that the importance of English fluency will fall behind “but the importance of robust quality-assessment frameworks and deep methodological scrutiny is paramount”.
“AI in its current phase of development is certainly not going to be the only reliable source to refer to and make decisions,” Dr. Gupta said.
‘Distinctly human territory’
With the evolution of intelligent machines, the intelligence of human beings, especially the experts and scientists, is going to be more valuable and we need to be cautious and careful on how to smartly employ these tools without biasing our concepts and fundamentals, she added.
A simple yet effective strategy to minimise errors in AI-driven data synthesis is to avoid relying on a single model, Dr. Gupta explained. Using a combination of models can provide more balanced and accurate results:
“Equally important is ensuring that the source datasets are obtained from authentic and credible databases.”
A few technical safeguards can further minimise AI-related errors, Dr. Rydquist added. For example, always validate AI-generated citations and data summaries against primary sources, he said.
So does AI help or hinder creativity? For one, AI can only make incremental discoveries. It can’t however achieve fundamental discoveries for generating truly original hypotheses as humans can.
In fact AI may be inadvertently constraining creativity and lateral thinking, according to Dr. Rydquist: “Wrestling with a problem deeply often generates insights, but AI shortcuts may deprive scientists of this generative friction.”
An important distinction, however, is that while AI is great at solving well-defined problems within established frameworks, “true creativity often involves reframing the problem itself or questioning fundamental assumptions. This remains distinctly human territory,” he added.
“The challenge lies not just in embracing technological progress,” Dr. Gupta said, “but in preserving the human spirit that fuels true innovation.”
T.V. Padma is a science journalist based in New Delhi.
