v29 #5 Little Red Herrings — Peering into Peer Review

by | Dec 11, 2017 | 0 comments

by Mark Y. Herring  (Dean of Library Services, Dacus Library, Winthrop University)

For the entirety of my career, academic peer review has been the gold standard.  It’s easy to see why because the name says it all.  Whether we’re talking about peer review in the sciences or the humanities, in visual and performing arts, or the social sciences, it always meant the same thing: a process by which something proposed, such as research or publication, underwent a review by those competent in the field to judge its merits.  Thus, those in the social sciences passed judgment on those writing or researching in that area.  Physicians judged the work of other physicians’ research or publications, and humanities scholars on those working in that discipline.  

While peer review worked well for many years, inherent in the calculus was, of course, people, other scholars.  We like to think that scholars, physicians, ministers and politicians — well, the first three anyway — are above reproach and will do the right thing.  However, all of us have lived long enough to know that isn’t the case.  When people are involved, despite their best intentions, the train of good reason will go off the rails.

We live in a time, now, however, where that trains appear to be derailing more often than not.  The irrepressible Scholarly Kitchen had daily posts on peer review (especially its transparency, or lack thereof) for Peer Review Week (http://bit.ly/2gZMDOL).  Particulary well done was a panel discussion on peer review’s past, present, and future (http://bit.ly/2x3T0tM).  Not to be outdone, College and Research Libraries began its recent issue with a guest editorial on who reviews the peer reviewers (http://bit.ly/2y08RIa).  Almost monthly, if not weekly, peer review comes up for discussion and often under a cloud.  

While it has always had its pitfalls, it is subject to “…friends review[ing] the work of each other in an unjust manner [and] undermin[ing] scientific integrity… constitut[ing] a perversion of ethics of science” [Gunsteren, 2015, http://bit.ly/2xbARL1].  But it’s more than even this.  The process in which one scratches the back of another, and both reap the benefits, whether from promotion, tenure, or advancement in some manner, continues apace and shows little signs of slowing down.  “With all its merits,” writes Ashutosh Jogalekar in a 2013 Scientific American piece, “the traditional model of anonymous peer review clearly has flaws; reviewers under the convenient cloak of anonymity can use the system to settle scores, old boys’ clubs can conspire to prevent research from seeing the light of day, and established orthodox reviewers and editors can potentially squelch speculative, groundbreaking work.  In the world of open science and science blogging, all these flaws can be — and have been — potentially addressed” [http://bit.ly/2eLjVAH].

If one is a bit doubtful about these charges, all one needs to do is stroll over to Retraction Watch [http://retractionwatch.com/] and look on in horror.  Every day, the custodians of all things right and true in scientific research are being watched after with carking care.  The results are so overwhelming that anyone who subscribes to the feed would be hard-pressed ever to want to write again.  And bear in mind that many of these published journals went through some form of peer review.   In journals of questionable merit to journals of gold merit, sham, lies, plagiarism, falsified graphs, charts, data and more are brought before the reader in all their inglorious detail.

Some of the blame for this state of affairs, beyond the mere fact that flawed human beings are, well, flawed, is the pressure on everyone to produce articles, research, grants, and so on.  This does not excuse the misprisions, but it does put them in context.  Furthermore, when promotions or dollars are not at stake, there is the tantalizing hook of fame, most of us forgetting how easily fame can become infamous.

But even when none of these things is present, there still exists in peer review the bias of the reviewer.  Any reviewer can find fault, and I am surely not the first person to point this out.  This is especially true in the case of academics.  Isn’t it our nature to be, if not distrustful, then at the very least, skeptical?  To find fault, even if it’s a handful of merely minor problems — should that kill a good idea, a strong case, or a potentially innovative approach?

This point becomes particularly important in the humanities when a given paper may well not have one right answer or approach, at least not in the case of the sciences when a sure outcome can be anticipated mathematically.  Nevertheless, even accounting for this poses its own problems as we have seen recently in the case of the social sciences when outrageous papers have appeared, having successfully made their way through what would appear to be a rigorous peer review process.  I am thinking here of Alan Sokal’s exposure of gravity as a construct (http://bit.ly/1eVRI3m) some decades ago, and of a more recent, if hilarious misstep, regarding the evolution of a social construct (http://bit.ly/2weyN0A).  

I wish I could say what the answer is.  Peer review appears to be taking a downhill slide, fake news is everywhere, and predatory journals threaten to unravel open access.  Trying to untie this Gordian Knot is not an easy task.

Fortunately, librarians are equipped with modern day Fragarachs, that legendary sword that when placed upon the throat of anyone forced the truth out of them.  

 

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

Share This