By Donald T. Hawkins, Freelance Columnist and Editor

The Charleston In Between conference occurs approximately halfway between two Charleston Library Conferences in the first week of November; its purpose is to conduct an in-depth exploration of important developments in the information industry for which there would not be time at the main conference. The 2023 In Between conference occurred virtually on September 6-7, and examined peer review. It was organized and moderated by Cris Ferguson, Dean of Libraries at Murray State University.

Opening Keynote

Brian Nosek, Co-Founder and Executive Director, Center for Open Science, said that the current model of peer review (which he called the Old Model) is a gatekeeper to a reward. The process is opaque and produces just a glimpse in time after the work has been completed. The decision process is binary and leads to a decision to accept or reject the paper. If it is accepted, the paper gets a credibility stamp. The challenges of this model include predatory journals, paper mills, fake AI papers, fraud, and selective reporting. Trustworthy research is hard work; untrustworthy research uses superficial signals of hard work.

The new model views peer review as a reward. It is transparent and occurs over the lifecycle of the paper, reacts to the marketplace, and is versioned (like knowledge, literature can be versioned). Transparency in peer review makes bad practice more inconvenient and therefore detectable. 

Here are some indicators of trustworthiness in the marketplace;

The new model expands the conception and gets rewards to the researchers, so it is hard to do fraudulent work. 

Shifting peer review from gatekeeper to reward has these effects:

• Exposes practices that make research trustworthy,
• Reduces the capability of dysfunctional markets to thrive, 
• Increases the emphasis on rigor and the quality of research, and
• Allows researchers to recognize good practice.

Stakeholder Perspectives Panel

Joris Rossum, a member of NISO’s Peer Review Committee, said that open science means transparency.  Peer review has been a black box; we must open it up and become more transparent about models that have been used. Readers need openness and transparency. 

Peer review has become a NISO Standard, and we must communicate consistently about it. Standard terminology steps include:

• Identify transparency: the extent to which identities of reviewers are made visible to each other during the review process,
• Reviewers interacting with each other: Direct interaction with each other and exchange of information during peer review,
• Review information published, and
• Commenting after publication, which can be open to anyone or only for editors, publishers, and selected individuals.

The next steps are to implement this process beginning with journals, then articles, and then other research objects such as books or software.

Kelly Smith, a member of the Editorial Board of Serials Review, discussed a librarian’s perspective on peer review. Its benefits include:

• Early access to emerging research,
• Contributing to the scholarly community,
• Networking and scholarly opportunities,
• Increasing cognitive skills, and
• Building a CV.

To become a reviewer, one can apply to journal editors, or simply let one’s desire be known through networking.

Courtney McAllister, Associate Editor of Serials Librarian and Serials Review presented a journal editor’s perspective on peer review. The journals she works with are published by Taylor & Francis and use double anonymized peer review. The biggest challenges in getting reviewers are finding people with the time to do reviews, aligning their expertise with a broad field of content and the shifts and churn in librarianship caused by peer review not being taught in library schools. An editor must standardize feedback, budget time for the process, and reach out to people newer in the profession. The goal is to avoid burning out reviewers, providing good feedback to authors, and publishing as much quality content as possible.

The future includes further refinement of peer review prompts, continued outreach, information sessions for prospective reviewers, and more dynamic engagement with editorial boards. The main issue is defining what sustainable peer review looks like.

What’s New in Peer Review

This was a lightning session in which 5 panelists discussed developments in peer review in 10-minute talks.

• Laura Feetham-Walker, IOP Publishing: IOP’s co-review process strives for equal recognition and is a supportive experience for all. IOP allows two or more people to collaborate on a review. A trial provided favorable results when multiple reviewers collaborated; the quality of co-reviewed articles is very high. The junior co-reviewer submits the article to the journal Editor so they get recognition for their work and are entered into IOP’s database of potential reviewers.
• Daniel Dotson from Ohio State University studied errors in articles and how they can lead to retractions. He found that only about 0.92% of articles published between 2012 and 2021 had a document type of “erratum”. About half of the articles with errata are in medicine; many of the journals have high impact factors and are top journals in their area. Retractions are very rare. Only about 1% of the errata were cited more than 10 times; about 6% of the retracted articles were cited over 50 times. 
• Lindsay Morton, Senior Manager at PLOS and Co-Chair of the Peer Review Week Steering Committee said that Peer Review Week (September 25 to 29 this year) is an annual event celebrating the value that peer review brings to scholarly communication and exploring how it is evolving. This year’s theme is “Peer Review and the Future of Publishing”; participation is open to anyone with an interest in peer review. The events of the week can be found here. AI tools have become widespread and will increase access to natural language processing and help reviewers become more effective. Researchers are working in new and more effective ways such as larger projects and dispersed teams, which raises questions about the best way to assess research.
• Sven Fund, Managing Director, Reviewer Credits noted that the problem of peer review is not only technological but social. It is the single largest challenge in academic publishing and affects reviewers, publishers, authors, and funders. Challenges for reviewers include insufficient matching of articles to appropriate reviewers, little incentive to be a reviewer because peer review is invisible, lack of rewards, almost no quality control in reviews, and spamming. Peer review must become more efficient to make it attractive. Ethical rewarding is vital. Some incentives are provided by publishers and should be an investment into the publishing system. About 40% of reviewers have not been trained, which is unique to publishing. 
• Abeni Wickham from SciFree spoke about automating peer review workflows. Pain points: the process is affected by unstructured emails among participants, 15 million hours lost reviewing rejected articles, the need for transparency in the process, and the inequitable distribution of researchers (who you know vs. what you know). Pain points can be turned into gains by a semi-automation of peer review, allowing funders and university administrators to access reviews, automating keyword generation, and giving reviewers the tags assigned to the document by the author. All versions of reviews should be published. These steps will make peer review a service. 

Transformative Agreements at IOP Publishing

Emma Bartovsky, Senior Transformative Agreement Success Manager, IOP Publishing, (a wholly owned company owned by IOP that publishes journals, conference proceedings, websites, books, and magazines), noted that IOP is dedicated to an open future; last year over 40% of their publications were open. IOP now has transformative agreements with over 900 institutions in 33 countries. They believe in making access to physics research universal and improving the discoverability and impact of research; transformative agreements are the most effective way to achieve this. Here are some key features:

Transformative agreements are driving the shift towards OA. OA benefits for authors include

Inclusivity is achieved by double blind peer review. 

Open Peer Review Panel

Jessica Polka, Executive Director, ASAPbio began this panel discussion by asking how preprints are changing peer review. Preprints enable rapid dissemination and open a new space for collaboration. Nearly 600,000 life science preprints exist today. Some servers such as BioRxive make reviewers’ comments available. Public feedback is an important remedy for misinformation and can lead to retractions. As this chart shows, preprint reviewing has dramatically increased in the last several years.

Eric Shares, Engineering and Collection Analysis Librarian at Iowa State University, reads open peer reviews to make his own reviews better. Many people have never received training in writing reviews. Open peer reviews allow authors to thank a reviewer for a contribution that improves their article. Contributors can read what editors say and see degrees of decision. Readers can discover more background on development of an article and gain a better understanding of nuances. The overall goal is to make reviewing more visible, valuable, and available.

Wolfgang Kaltenbrunner, Senior Researcher at the Centre for Science & Technology Studies, Leiden University in the Netherlands said that there are 4 schools of thought on improving peer review. Here are the main issues of each.

1. Quality and Reproducibility School: Reviewer training and reliability, statistical peer review.
2. Equity and Inclusion School: Diversity of reviewers and editorial boards, double blind peer reviews.
3. Democracy and Transparency School: Reviewer accountability, open and post-publication peer review, preprint peer reviews.
4. Efficiency and Incentives School: Pressure on reviewers and the reviewing system, journal-independent peer review, reviewer recognition.

Open peer review reduces bias, but most peer reviewing is uncommented which is a large problem. Preprints should be distinguished from published literature. We need better ways of establishing trust.

Closing Keynote

Dr. Daniela Saderi, Co-Founder and Director, PREreview and Dr. Antoinette Foster, Director of Community Transformation at Oregon Health & Science University collaborated on the closing keynote address entitled “Beneath the Surface: Shattering Illusions and Embracing Transformation in Peer Review”. Dr. Saderi noted that the publication system is made by people. Our understanding of the world guides our research; scientists are supposed to be objective.

Dr. Foster said that before we solve problems, we must understand them, understand who we are, and how we relate to the past. She addressed the following 3 points:

1. How we got here. Systems of oppression include racism and colonialism.  

Racism gives an unjust amount of resources, rights, and power to white people while denying them to people of color. Colonialism means the policy or practice of acquiring full or partial political control over another country.


Racial projects cannot exist without colonialism. “Race” was created in 1453 by Prince Henry, and it spread to include systematic colonization of North America in 1492 and the establishment of the transatlantic slave trade in 1526.  Racism and slavery cannot happen without land.  Systems of racism include who we are with ourselves and others. They can be personal, interpersonal, institutional, or structural. Many publications were focused on white supremacy. Science and scientific communication have become the new legitimizing agents.

2. Systemic oppression shows up in publishing. Race and genetic ancestry are not interchangeable. There is no genetic basis for race.

3. An essential process for change. The process of collective change occurs in 4 stages.

• Our values shape our decisions and are a reflection of who we are. We choose our own values. 
• What we practice on a small scale is reflected on a larger scale.
• Individuals shape the collective.
• Collectively we shape the change.

Dr. Sideri closed the session by posing the following questions. 

Donald T. Hawkins is a conference blogger and information industry freelance writer. He blogs and writes about conferences for Information Today, Inc. (ITI) and The Charleston Information Group, LLC (publisher of Against The Grain). He maintains the Conference Calendar on the ITI website He contributed a chapter to the book Special Libraries: A Survival Guide (ABC-Clio, 2013) and is the Editor of Personal Archiving: Preserving Our Digital Heritage (Information Today, 2013) and Co-Editor of Public Knowledge: Access and Benefits (Information Today, 2016). He holds a Ph.D. degree from the University of California, Berkeley and has worked in the online information industry for over 50 years.