By Matthew Ismail, Editor in Chief of the Charleston Briefings and a Charleston Conference Director

“The first person we ought to serve is the author…The author should have as many options as possible.” Elliot Lumb on ATG the Podcast

Peer review is an essential part of the scientific publishing process. Though we sometimes hear some open access or preprinting activists say that peer review related to journals is unnecessary, and that articles should all just be published on open servers so that the community of scientists can decide what is important…I think most of us still prefer to have an organized system of prepublication expert vetting of articles making scientific claims before they’re loosed upon the world. Which also accounts for the reservations many people have about preprint servers.

Publisher-organized peer review as currently practiced, however, is certainly open to criticism. Traditional peer review is a rather too narrow process in which journal editors depend upon networks of trusted colleagues to act as peer reviewers, thus making the system potentially too exclusive and unrepresentative. Since the system is based on these narrow networks of reviewers, if an article is rejected by one editor, when the author submits it again at another publisher, the peer review process starts all over again. This makes the process slow, repetitive, and expensive (even wasteful)  in addition to being overly narrow and exclusive.

And these problems of closed networks and great expense are made worse by the reproducibility crisis, with the corresponding and widespread problem that, even when an article has been judged by peer review to have used sound research methods and to have drawn reasonable and interesting conclusions, other researchers can’t make the experiment work again. This certainly makes it obvious that peer review may be vetting the soundness of the research methods and the interest of the conclusions drawn from the research–but peer review certainly does not verify the results or make them reproducible.

And these sorts of reservations about traditional peer review is where Elliot Lumb and his company, PeerRef, come in. Elliot finished a PhD in Chemistry at the University of Nottingham in 2018 and then decided to go into publishing, beginning his career as a commissioning editor at Frontiers, the open access science publisher. He worked initially as a commissioning editor, but became increasingly interested in the business side of the company, moving into work involving strategy and planning issues and seeking to make intradepartmental processes more efficient. (Lumb, 2023)

As Elliot found himself more involved in matters related to innovation and open access business strategy at Frontiers, he also found that the inefficiency of the system of peer review kept coming up. Not only was peer review opaque and wasteful, as discussed above, but a paper would also be judged by narrow criteria at each journal, so an editor might reject a paper because its emphasis was not quite right for them or because they had published something vaguely similar the previous year. Such journal-specific concerns were not related to the quality of the research, yet if the article was rejected, the peer review process had to start over again at the next journal to which the author submitted. 

With a growing interest in entrepreneurship allied to such publishing concerns while at Frontiers, Elliot eventually came up with the idea for PeerRef. 

Now, when Elliot started PeerRef, he had some pretty uncompromising ideas concerning traditional publishers: they are exploitative, unnecessary, wasteful, and exclusivist. Open access would liberate research from these traditional publishing giants, and the new system of preprinting, which often presented itself as a free and open alternative to traditional publishing, would free publishing even from publishers! And PeerRef would operate in this space of openness and freedom.

Elliot’s initial idea for PeerRef was, he admits in hindsight, a bit naive, though many of its elements remain crucial to him. PeerRef, he hoped, would help the OA movement to bring the world of traditional publishing to its knees by establishing free and open alternatives to working with publishers. PeerRef would organize peer review with a broad and inclusive cadre of peer reviewers, rather than relying on the network of  particular editors.  These peer reviewers–a peer is simply a similarly qualified person who studies the same thing as you do–would then be available to review preprints that were posted on open servers. After the process of peer review, PeerRef would publish the review for all to see. If, at this point, the author chose to work with a publisher, that was their choice. But PeerRef and the preprint servers would render any subsequent journal publication redundant, and authors and funders would increasingly see that a peer reviewed preprint was actually a published article–minus the exploitation.

Yet, after a few months of talking to researchers as he launched PeerRef–rather than just OA advocates–Elliot began to see that there was more to traditional publishing than just organizing peer review and taking profits. Indeed, the more he talked to researchers the more he realized that the value proposition of traditional publishers was not in organizing peer review at all.

For Elliot, publishers mostly add value for the authors and readers after publication. Researchers made clear to him that they valued the work that publishers do in building community, curating research for readers, and making published works more accessible and discoverable. Publishers have experts in SEO (“Google is like your journal’s homepage,” he says) and accessibility and help the author to reach specific audiences, such as policymakers or clinical researchers.  (Lumb, 2023) All of this is actually crucial for researchers.

After his initial enthusiasm for disruption, Elliot then pivoted and acknowledged that, by creating a journal-independent system of open peer review with PeerRef, he wasn’t so much displacing traditional publishers as giving them an opportunity to streamline their processes by leaving peer review to others and focusing on what they do best. Thus, instead of trying to disrupt traditional publishing, Elliot sought to give researchers the best of both worlds: speedy and independent peer review and effective post-publication support. 

“The first person we want to serve is the author,” Elliot says. “We ought to give the author as many opportunities as possible.” (Lumb, 2023) If an author posts a preprint, has it reviewed, and satisfies their ambition for that piece of research, that option exists. If they want to post a preprint and have it reviewed and then submit it to a journal, then that is also fine. And the beauty of the PeerRef system would be that the initial review could travel with the paper, obviating the need for the review process to start over again. 

PeerRef, therefore, would establish an independent network of qualified reviewers, peers whose competence can be evaluated  by the quality of their published reviews, whether they are known to the editor or not. This broadened network would bring a more inclusive approach to recruiting reviewers and would take some pressure off of those who are currently doing most of the reviewing work. This seems like a very good idea.

What is the business model of this system of journal-independent open peer review? Elliot says that there are two obvious models: one in which the publisher pays when they use PeerRef, and another in which the researcher pays for the peer review out of their grant, like an APC. All the funders would need to do is to add a line on their grant forms for peer review. Elliot thinks that the APC-like model is more sustainable.

Thus, in this new model, instead of publishers doing everything in the publishing process themselves, there would be a variety of stakeholders in the process who would have different roles. Authors, funders, universities, peer reviewers, preprint servers, publishers–all would have a specialized role in the process. This gives authors more flexibility, provides the opportunity for more peer reviewers to participate in the publishing process, and would allow publishers to streamline their operations and focus on what they do best.

Elliot also sees an opportunity to extend the advantage of an enlarged pool of peer reviewers through his partnership with Sven Fund’s new company, ReviewerCredits. (Fund, 2023) ReviewerCredits allows reviewers to receive acknowledgement for the usually anonymous work of peer reviewing articles, as well as allowing them to claim rewards for their work such as APC credits. For Elliot, these changes to peer review would be win-win for all involved.

Elliot remains committed to open access and to transformation of the publishing world, but with a new respect for the contributions of traditional publishers. He doesn’t see particularly rapid growth of PeerRef in the near future, simply because publishing moves slowly, but in the meantime he is seeking more funding, developing the PeerRef brand, and building partnerships with more preprint servers and publishers. Elliot also wants to work with others in his new domain of open peer review to be sure that they have consistent standards for services such as DOIs.

After this initial period of establishing open peer review, which includes a period of educating stakeholders about the advantages of the system, Elliot predicts that PeerRef will take off as a partner in a transformed peer review process. And this does seem like a good direction for peer review, in general.

References

Fund, Sven, “A Conversation with Sven Fund, Managing Director, ReviewerCredits, and Senior Director, Knowledge Unlatched,” ATGthePodcast 191, Feb 26, 2023.

Lumb, Elliot, “A Conversation with Elliott Lumb, Founder and CEO, PeerRef,” ATGthePodcast 191, Jan 10, 2023.