Home 9 Blog Posts 9 Don’s Conference Notes: Charleston In Between 2024

Don’s Conference Notes: Charleston In Between 2024

by | Apr 3, 2024 | 0 comments

#

By Donald T. Hawkins (Editor and Conference Blogger)

The Charleston In Between conferences began in 2021 to provide an in-depth examination of subjects of current interest at a level of detail not possible at the annual Charleston Conferences. The 2024 conference consisted of half-day sessions on March 19 and 20 and attracted 100 online attendees. The subject this year was integrity in research and publishing.

Why Integrity in Research and Publishing Matters

Mohammad Hosseini

Mohammad Hosseini, Assistant Professor, Northwestern University, Department of Preventive Medicine, provided an excellent basic introduction to integrity and defined it as being honest and having strong moral principles, and moral “uprightness”.  “Integrity” has many synonyms, including honesty, probity, rectitude, honor, good character, ethics, morals, virtue, righteousness, morality, decency, fairness, scrupulousness, sincerity, truthfulness, and trustworthiness.  The book Integrity and Virtues of Reason has an in-depth treatment of the concept of integrity. It is important to understand the core of this concept: wholeheartedness, autonomy and identity, and standing for something. It is about what culture and environment we are in; autonomy, identity, community and cannot exist in a vacuum. Hosseini likened the concept to a Swiss Army Knife.

Research integrity means the attitude and habits of researchers to conduct their research according to appropriate ethical, legal, professional frameworks, obligations, and standards.   It is needed to preserve society’s trust in science and get funding for research. We expect scientists to be accountable to taxpayers because they are funded by government grants. The concepts of ethical research are mandatory and not negotiable. Open science will make research transparent, but researchers may feel that they are constantly being watched. Some of them have not thought about the part research plays in public policy and in the lives of millions of people. Research integrity also applies to institutions: some universities have integrity offices. Members of a community should hold their institutions accountable. 

Publication integrity extends from writing the article until publication. We want research and publication to be trusted. Communication and relationships are different concepts involved in publication integrity. Volunteers in publication (reviewers, etc.) have different viewpoints and are concerned with different workflows. Research integrity and publication integrity are both necessary to maintain a healthy environment in science. They are not new and are distributed around the world. Everyone must know how integrity is important in their own environment. Research is a difficult job because of all the forces on it.

Institutions, Libraries, and Their Role in Publishing Integrity

Curtis Brundy, Associate University Librarian for Scholarly Communications and Collections, Iowa State University, described publishing integrity from a library perspective. The typical librarian’s view used to be that libraries do not have a big role in integrity, but the paper mill crisis has changed our perspectives. Librarians should be active in preserving publishing integrity. The state of the world in 2024 is divided, and trust in US institutions is headed for historic lows. Science is far from a perfect instrument of knowledge; it is just the best we have. Anything that undermines trust in science should be considered as a grave threat. 

Last year was a bumper year for retractions (1l,000 were published by Hindawi), as this graph shows.

Retractions occur because many researchers are evaluated on the number of papers they publish, not their quality. More articles published means more APCs and more revenue for publishers. 

Although librarians have a role in addressing the paper mill crisis, publishers have also recognized the damage that has been done. For example, Frontiers is planning to lay off 600 people this year, the Institute of Physics (IOP) is strengthening critical teams and expanding its integrity efforts, and Wiley/Hindawi is discontinuing the Hindawi brand. Wiley has recognized the threat and has built an AI detection tool. 

Publishers are working together to help address the crisis. Retraction Watch is an essential resource as well as the online journal club PubPeer, and a volunteer group of independent sleuths has actively advocated for increasing efforts to detect fraudulent articles. NISO has developed recommended practices around retractions. 

A new initiative, ERROR, is a bounty program that rewards researchers with an Amazon gift card when they report a problem with an article that leads to its retraction. It has helped to stop people who think they can get away with publishing false results. 

We need libraries to take a role in prevention and cleanup; here are some possible things they could do:

APC models are emphasizing quantity over quality and pushing publishers to high volume approaches, so libraries should work towards non-APC OA business models. Here are some roles for publishers.

Libraries should establish publishing integrity requirements. Do we need audit requirements? What happens to APC retraction donations? Maybe APCs will fund bounty programs?? Publishers should not keep APC revenues; that is like getting rewarded for bad behavior! What are our expectations for reporting and transparency questions? Libraries should join the Committee On Publication Ethics (COPE) and UNITED2ACT which is working to address the challenge of paper mills. Libraries have a role in detecting fake papers; they are well equipped to work as sleuths and can take a role in NISO’s activities. We all should care about the paper mill crisis. 

How to Define Integrity Standards in Publishing with Integrity

Nandita Quaderi, SVP & Editor-in-Chief, Web of Science at Clarivate, echoed several of the issues that Brundy raised and said that misuse of bibliometrics is driving fraudulent behavior. Measures have become a goal in themselves; people are chasing more citations to their articles because of pressures to publish. 

The Web of Science provides 3 levels of curation: its collections, data, and metrics, using an in-house team. This approach protects research integrity because the data and metrics are trusted. Its core collection is subjected to the following 4-step publisher-neutral editorial evaluation. 

Indexed journals filing to meet quality criteria are deleted from the core collection.  

An AI Surveillance Tool assists in-house editors to identify journals of concern. The tool does not remove the need for in-house editors to make decisions about journals. Authors are not penalized for retracting articles, and publishers are not penalized for sharing their data with Clarivate or for retracting articles. Deleted journals must wait for 2 years before they can be resubmitted.  The top priority is to ensure that the Web of Science contains only trustworthy journals. 

A new policy has been implemented for the Journal Citation Indicator (JCI). It has been expanded to the Emerging Sources Citation Index (ESCI) and Arts and Humanities Citation Index (AHCI), so the JCI now provides a single journal-level metric that can be easily interpreted and compared across disciplines. The Journal Impact Factor (JIF) is thus an indicator of trustworthiness and impact. All journals in the core collection now have a JIF. As a result of these changes, almost 9,000 journals from more than 3,000 publishers have received a JIF for the first time.

A Small Publisher’s View of Research Integrity

John Chen, Director of Publication Development, Tech Science Press, began his presentation with a history of the company. It was founded in 1997 in the US. The editorial office moved to Nanjing, China in 2017. It now publishes 30 journals in mechanics, computer science, materials sciences, etc. The journals are fully OA and peer reviewed. Although the Press organizes conferences, journals are their main business. 

The core issue in research integrity is paper mills, which are a major problem because they are always changing, and there are many types of them.  The major types are:

  • Traditional articles produced by real companies with agents that are easy to identify,
  • Research teams producing special issues, and new forms of literature with citation manipulation, and
  • Problematic journal publishers sharing profits from predatory journals with researchers.

Paper mills are not necessarily producing fake papers. Companies producing them are real companies. They cater to authors who are desperate to publish. Characteristics of articles published by paper mills are:

  • Template writing style
  • Methodology is not reproducible, and equations are not always correct.
  • Bulk submission
  • Research level does not match the type of institution.  Author’s affiliation changes.

How they select targets; characteristics of journals:

  • Large volume publication,
  • Broad and general scope,
  • Fast response and fast peer review time,
  • Loopholes in workflow: inadequate review processes,
  • Internal resources (inside people), and
  • Low editorial standards.

Strategies:

  • Flood journals with many submissions to overwhelm publisher’s workforce,
  • Probability of acceptance: Find journals of low quality in different areas and test them for weak points,
  • Loopholes: change authorship when a paper goes to revisions, and
  • Cater to preferences of journal publishers.

Disadvantages of small publishers: limitations on resources, budgets, visibility, and credibility. They have low APCs, resulting in low income. They are not famous, so they tend to be invisible to others in the business. Paper mills are more likely to fight with them and often threaten to destroy them if they do not cooperate.

Advantages of small publishers: they are flexible, adaptive, and responsive, so they can embrace new technology and make changes quickly if necessary. They seek global collaboration and strive for progress and optimization, which is one result of not being famous. Tech Sciences makes changes directly on their own platform, tracks data from Retraction Watch, and manages special issues using a separate module. 

Small publishes have an opportunity to voice their opinions and receive support. To build trust, Press representatives went to book fairs to meet colleagues, joined the Society for Scholarly Publishing (SSP) and the policy committee of the Association of Learned and Professional Society Publishers (ALPSP) to learn more from experts in industry, and formed collaborations with FigCheck, Morressier, and Clear Skies. Nevertheless, they are vulnerable and always need support.

A Large Publisher’s Perspective on Research Integrity

Michael Streeter, Director of Research Integrity and Publishing, Wiley, discovered the guiding principles of publishing integrity:

  • Publishing integrity is integral to the work of publishers, especially in today’s increasingly open environment.
  • Maintaining trust is central and requires a collaborative effort.
  • Innovation in publishing is essential to identify integrity issues early in the process.

Here are the characteristics of Wiley as a large interdisciplinary publisher:

There is now a greater interest by the public in research integrity, which has resulted in high profile investigations and research on paper mills.  For example, the president of Stanford University was accused of scientific fraud and misconduct in his research laboratory and forced to resign.

Generative AI is challenging; we must ensure that ethical values are in place.

We need scalable solutions to be responsive to pre-existing and emerging challenges and uphold our responsibility to maintain the scholarly record.  Policies are only as good as we are able to implement them. Wiley’s research integrity organization has 26 staff members who tackle and respond to concerns when they become known. It serves as a direct point of contact to receive concerns from peer reviewers or interested readers and conducts broad reviews of additional articles using industry resources to give viability. An editorial code of conduct and best practice guidelines on research integrity and publishing ethics were recently launched. 

Editors have an important role in the management of journals. Retractions are managed at scale; articles reaching a certain threshold of issues are retracted, and retraction statements provide a rationale for each one. Wiley has published a white paper on how data is shared responsibly, and how readers are alerted.  

Collaboration with external partners such as STM Integrity Hub and United2Act Against Paper Mills is critical for such actions as duplicate submission detection. Individual sleuths are also raising concerns with publishers. Wiley is testing its own tools to support integrity investigations.  Priorities for collaboration include rewards and incentives, infrastructure development, and training and capacity building.

The Human Factor

Martin Delahunty, Company Director, Inspiring STEM Consulting, noted that there is a range of human stakeholders in publishing, and there is an overwhelming feeling of a crisis. Whistleblowers and sleuths are active; one of the natural inclinations in addressing the challenges is to turn to technology, but ultimately human factors must play a role in maintaining research integrity. We must understand the motivations for researchers to commit fraud. The “Fraud Triangle” shows us 3 factors coming together that can lead to fraud: motivation, opportunity, and rationalization.

Fraud Triangle

Cultural differences can also significantly affect research integrity; for example, a study of 24,000 graduate students from multiple countries found that the “peer cheating effect” is stronger in certain cultures and plays a role in students’ academic cheating. Effective strategies to promote academic integrity are needed. 

The role of personal values in maintaining research integrity must also be acknowledged. Researchers who hold strong ethical values are unlikely to engage in practices such as data manipulation, fabrication or plagiarism and are more likely to report their results transparently and accurately. The pressure to publish or peer pressure may cause researchers to engage in future scientific misconduct, so it is critical to cultivate an environment that encourages and values ethical conduct. There is a continuum from honest mistakes to fraud.

Mentoring significantly influences research integrity; an experienced advisor can guide early career researchers on the importance of ethical conduct. The University of Michigan has a very successful mentoring program, the Research Administration Mentoring Program (RAMP) in which workshops focus on enhancing professional growth.

Researchers play a critical role as peer reviewers. Publishers feel that peer review is a defining characteristic of a modern academic journal, but in contrast, reviewers feel overwhelmed, undermined, and undervalued. Many of them are declining peer review requests; frequently they feel that articles to be reviewed are outside their interests or expertise. Journal editors are saying that one of the hardest parts of their job is to find qualified reviewers. Publishers need to create more meaningful incentives and provide training in reviewing.

It is encouraging to see some universities and research organizations providing research integrity training. Culture is created by the behaviors that we tolerate; if we tolerate bad behavior, people will think they can get away with it. Change starts at the top; leaders must support resourcing for changes and be willing themselves to change.

Here are some recommendations for all stakeholders: 

  • Foster organizational cultures valuing honesty, transparency, and accountability,
  • Support mentorship programs for early career researchers,
  • Provide much more training on peer review,
  • Provide training and education on research integrity,
  • Promote open science practices such as data sharing and replication, and
  • Invest and rely more on people than technology.

Can Technology Help with Research Integrity? Why Didn’t It So Far?

Phill Jones, Lead, Digital Technology, More Brains Cooperative Consultancy, asked why we are worried about research integrity. Alarms have been raised and are increasing in frequency and urgency. An opinion article in PLOS Medicine entitled “Why Most Published Research Findings Are False” was published in 2004 and resulted in extensive discussion.  Sometimes bad research has serious consequences, but not all research integrity concerns are malicious. Websites and blogs (some now defunct) raised awareness, but quality and fairness of them has been variable. PubPeer is a step in the right direction.  There has been a growing threat from industrial research fraud. Predatory journals were not taken seriously at first because we did not have a view of the characteristics of a predatory publisher. Paper mills are a more scalable problem, but they are more difficult to detect because there so many articles. Paper mills target weak journals, and targeted journals can have up to half of their articles generated by paper mills. 

STM Solutions has created the Integrity Hub which is a cloud environment to share tools and data that can detect computer generated manuscripts and duplicate submissions. Obviously, paper mills do not want to get caught by plagiarism detectors so they use sentence fragments and substitute commonly used words with others taken from a thesaurus. Clear Skies is a similar system that identifies patterns in text and detects research fraud.

We are currently seeing the beginnings of a technological arms race. The next thing is AI which uses a range of semantic dependences and large language models (LLMs), although currently there is no way to detect that something was written by an LLM. Open source is easy to deploy and cheaper, and there are many models of it. LLMs have reduced the cost of generating plausible nonsense! We cannot just throw technology at the problem; it is not a magic bullet. If we build a better mousetrap, the mouse will get cleverer!

Research fraud is a socio-technological problem. The real problem is that many problems cannot be reproduced; therefore they are not reliable. Even researchers think there is a problem. We must adjust incentives to drive accountability, trust, and integrity. Scholarship is a process, not a single event. Accountability is a result of transparency. Some publishers ask researchers tell them about their research in exchange for a promise to publish the article regardless of the outcome. Openness demonstrates integrity: electronic lab notebooks produce a digital paper trail and document a research project. A Research Activity Identifier (RAID) is an open identifier for projects that permits tracking and sharing. It will aid in transparency and interoperability in research management.

Conclusions:

  • Technology plays an important role in research integrity.
  • Technology alone is a double-edged sword; it enables bad actors as much as it enables us to fight them.
  • Research integrity is a socio-economic problem.
  • Technology solutions that enable transparency and accountability are the answer.

10 minute Power Pitch/Lightning Round

Neil Christensen: Sales Director, Morressier, a neutral independent platform of integrity, diversity, and independence, can look at analytics and search results, check for paper mill articles, and help authors correct honest mistakes.  Users determine the checks they want enabled. The platform consists of managers of abstracts, proceedings, integrity, and journals.

Gareth Dyke, Academic Director, ReviewerCredits, said that it is an expert peer review network to improve peer review and find, recognize, and reward peer reviewers. Peer review will never again work the way it used to; it is open to abuse or even laziness. Reviewers dislike being subjected to direct marketing as a result being a reviewer.  Many reviewers have never received any training and are not rewarded for reviewing; 50% of all reviews are done by10% of the reviewers. The hardest job for publishers is finding qualified reviewers. This diagram shows attitudes of reviewers toward incentives

ReviewerCredits provides 5 key services: find reviewers, identify and verify reviewers, reward reviewers with credit points and monetary incentives, certify reviewers and recognize their extensive efforts, and provide access by publishers and editors to a network of researchers ready to review.

John Willinsky, Founder, Public Knowledge Project (PKP), described a Publication Facts label similar to nutrition facts labels on groceries, as shown here:  

This label makes it easy to determine how one journal compares to another. PKP is focusing on OA. We have a responsibility to contribute to the public knowledge and specify what distinguishes research as a source of information. The label was designed in consultation with about 100 users. Editors are not involved in generating the numbers on the label. Articles mentioning it have a drop down menu allowing readers to see it. 

Elliott Lumb, Co-Founder of Research Signals noted that it provides signals to researchers indicating the trustworthiness of articles. Research integrity issues are reducing trust in research and are an existential threat for publishers.  Research Signals helps researchers, journal editors, and research integrity professionals identify and prevent publication fraud by enabling them to gauge the legitimacy of a research article through its metadata. Metadata is hard for fraudsters to get away from. It is a key source of information and improves over time. The Research Signals network can integrate different inputs, analyze individual published articles, identify problems in each journal, and prevent articles from getting published as appropriate. It is transparent and easy to use.  Here are some features of the system.

Donald T. Hawkins is a conference blogger and information industry freelance writer. He blogs and writes about conferences for Information Today, Inc. (ITI) and The Charleston Information Group, LLC (publisher of Against The  Grain). He maintains the Conference Calendar on the ITI website. He contributed a chapter to the book Special Libraries: A Survival Guide (ABC-Clio, 2013) and is the Editor of Personal Archiving: Preserving Our Digital Heritage (Information Today, 2013) and Co-Editor of Public Knowledge: Access and Benefits (Information Today, 2016). He holds a Ph.D. degree from the University of California, Berkeley and has worked in the online information industry for over 50 years.

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

LATEST NEWS

Tea Time With Katina And Leah

5-17-24 This article is about how marketing of Misty of Chicoteague became a model of a veritable marketing production master class. The book was first published in 1947 and was a perennial bestseller. It is a semi-true story  of how a boy and a girl  came to own a...

Federal Library and Information Network (FEDLINK) Announces Winners of 2023 Federal Libraries/Information Centers of the Year and 2023 Federal Library Technician of the Year (via Library of Congress) – Plus More ATG News & Announcements for 5/16/24

Image via Pixabay Federal Library and Information Network (FEDLINK) Announces Winners of 2023 Federal Libraries/Information Centers of the Year and 2023 Federal Library Technician of the Year (via Library of Congress) "The Federal Library and Information Network...

SUBSCRIBE TO OUR PODCAST

Share This