By: Nancy K. Herther, writer, consultant and former librarian
It was ten years ago that the Altmetrics Manifesto was published. Today the movement to more open science is growing rapidly. In our recent article series on Wikipedia, we looked at how Wikipedia itself provided an interesting lens to study the public impact and value of research. The ability to gauge the value of research in new and interesting ways has been particularly important during the ongoing COVID crisis. And altmetrics has provided key insights and analysis that have already been essential to facing this health and information crisis.
FROM SCHOLARLY INDEXES TO METRIC ANALYSIS
First, there was leafing through issues of individual journals. Later there were index cards – and print indexes – that formed the basis of scholarly research. By the 1960’s, with the evolution of computers, we saw the first major evolution in scholarly research. As Christine Borgman wrote in her 2007 book, Scholarship in the Digital Age, “During the early 1960’s computers were used to digitize text for the first time; the purpose was to reduce the cost and time required to publish two American abstracting journals, the Index Medicus of the National Library of Medicine and the Scientific and Technical Aerospace Reports of the National Aeronautics and Space Administration (NASA). By the late 1960’s such bodies of digitized alphanumeric information, known as bibliographic and numeric databases, constituted a new type of information resource.”
Gene Garfield and his colleagues ‘invented’ the field of scientometrics or citation analysis back in the 1960’s by creating tools that allowed researchers to mine significant data from basic bibliographic information as a way to gauge impact, value and trends from the corpus of scholarly publishing. Things have never since been the same. Citation analysis, Journal Impact Factors and other applications soon followed as computers grew in sophistication and speed.
“Thanks to its online availability, the flow, dissemination, and interaction of research can now be tracked and analyzed beyond what was traditionally accepted as the signifiers of prestige and impact,” explains an excellent Florida Atlantic University Library guide. “Altmetrics are defined as metrics and qualitative data that can be used in addition to traditional impact factors that describe a work’s impact. Due to variations of what influences impact and prestige among disciplines, many fields are using altmetrics as another or additional way to demonstrate the impact.”
COVID: THE PERFECT INFORMATION STORM DURING GLOBAL CRISIS
Today, technology is creating opportunities so quickly that keeping up on the various means of research analysis is just as difficult as keeping up on the research itself. And the current global COVID crisis is exacerbating this growth of information, intensifying the need for ready access to any potentially critical research in the face of this cataclysm.
Given the unprecedented number of preprints, government and institutional reports, and research data, the ability to find key information has become a critical issue in itself. One solution has been the development of COVID research portals to, at least, make the global scientific community aware of key research and data as it is created – even before formal review or publication. This includes the Institute of Translational Health Sciences’ COVID-19 Research Portal, the World Health Organization’s Global Research Database, the European Open Science Cloud Initiative’s European COVID-19 Data Platform, The independent European COVID-19 Data Portal, and the U.S. Centers for Disease Control and Prevention’s listings of COVID-19 websites.
And, it’s not just researchers seeking reliable information: As a public health crisis, people across the globe are trying to understand this novel disease and the best options they have to avoid and survive the pandemic. The best option we all have today is through the internet. “The latest figures show that the number of global internet users has been steadily increasing over the past decade or so,” notes a recent Oberlo report, “and has even more than doubled from the 2.035 billion registered at the start of the decade in 2010. That marks an 8.2 percent average year-over-year growth over this nine-year period…In Europe and the Americas, 82.5 percent and 77.2 percent of the population respectively are internet users. Social media statistics show that nearly all (99 percent) social media users use them to access social networks.”
The sources of available information on research varies from personal stories and advice to datasets to governmental requirements. Consider, for example the variety of COVID coverage available through popular sources of global communication, reporting and commentary today:
- Media outlets across the globe – newspapers, magazines to podcasts and televised/radio programming and more.
- Discussions and mentions posted on social media platforms like Facebook, Twitter, Pinterest, YouTube, What’sApp, Messenger, WeChat and so many more.
- Recommendations on sites such as Figshare, Wirecutter, the Strategist, TikTok, Pinterest, etc.
- Post-Peer review sites like F1000Research, PubPeer and others – even maintained by publishers themselves.
- Comment functions on sites like preLights and Review Commons.
In past times, formal communication of scholarly/research information was well-contained; today communication is constantly evolving and changing. Information itself is finding new channels and ‘containers,’ which is complicating traditional communication patterns and efforts to follow, access and categorize information and data as it is created, debated and used.
Given the importance of information that now captures “the online attention surrounding scholarly content,” new options and resources are needed. The Altmetric.com website itself explains their role as “a complement and alternative to typical bibliometric indicators. Instead of statistically analyzing citation counts and associations between publications, altmetrics measures, captures, mentions, and other types of interactions on the web to demonstrate interest in various works.”
THE CURRENT STATE OF ALTMETRICS
In announcing the initial release of the Altmetrics Manifesto ten years ago, it was stated that “the growth of new, online scholarly tools allows us to make new filters; these altmetrics reflect the broad, rapid impact of scholarship in this burgeoning ecosystem….Citation counting measures are useful, but not sufficient. Metrics like the h-index are even slower than peer-review: a work’s first citation can take years. Citation measures are narrow; influential work may remain uncited. These metrics are narrow; they neglect impact outside the academy, and also ignore the context and reasons for citation. The Journal Impact Factor, which measures journals’ average citations per article, is often incorrectly used to assess the impact of individual articles.” No one could have guessed the key importance that altmetrics is playing in this pandemic.
In honor of the tenth anniversary of the Altmetrics Manifesto, the Altmetric company has published The State of Altmetrics, which explores a decade of innovation and growth in the field. “The pace of research may have accelerated over the past,” Altmetric’s CEO Kathy Christian wrote in the report. “While 2020 has been frenetic on many fronts, it has been amazing to see how creativity and ingenuity can flow from a very difficult situation. It has come not just from researchers themselves, but also from those across the wider community: universities, governments, publishers, and businesses – all rapidly adapting the technologies and processes they use to manage research throughout its lifecycle.”
Digital scholarly platforms like Mendeley, BioMedCentral or F1000 are increasingly integrated in scholarly writing practices and communication strategies across the disciplines.
A key assessment of Altmetrics in a 2020 article in the American Journal of Sports Medicine noted that the “various potential benefits of using altmetrics as opposed to traditional bibliometrics have been proposed: altmetrics allow for the assessment of publication impact beyond academia and the evaluation of different types of less commonly cited scholarly content, such as data sets and software. In addition, altmetrics provide real-time feedback on impact on the order of days to weeks, as opposed to several years, after publication.” And clearly in a pandemic, the need for access to any information that might prove significant is critical.
COVID DATA: ALL THE MORE VITAL…AND PROBLEMATIC
A very interesting preprint analysis by Spanish researchers, posted on biorxiv recently, provides an interesting picture of the impact that COVID is having on research and how access to that research is testing the limits of today’s scholarly publishing system: “Traditional bibliometric databases such as Web of Science or Scopus, which index mainly published journal literature, have become almost instantly obsolete while journals are accelerating to an unprecedented rate their publication track for any COVID-related study. This has led scientists’ attention to unexpected sources such as ad hoc compilations of scientific literature openly accessible and curated by the scientific community.”
COVID-19 has created unprecedented challenges. Research is being shared as carefully – yet as quickly – as possible. The amount of research being published – formally or still in process – has created an information pandemic all its own. As both research and governmental efforts during this global public health crisis continue to develop at speeds that put traditional quality measures in question, researchers are finding key data or information in new places.
Vincent Larivière, Fei Shu and Cassidy R. Sugimoto reflect in a London School of Economics blog posting on the current state of scholarly information in time of such mayhem:
“Less than one third of the cited articles from which the ‘coronavirus articles’ drew information and inspiration were other ‘coronavirus articles’. Even if all articles on the topic of coronaviruses were made available, this would still be insufficient to address the crisis, given the inherently interdisciplinary nature of biomedical research. The knowledge base of science is simply much broader than a single topic. Viewing the literature through the narrow lens of coronavirus articles directly relevant to COVID-19 alone blinds the research effort to other work that could prove crucial. Cures for diseases often come from novel combinations and insights from several areas of research. If the goal of opening research is to advance science and serve society, all research should be open, not just a portion of it.”
THE NEED TO GO EVEN BEYOND TODAY’S ALTMETRICS
In a very interesting assessment by German researchers, titled “The Valuation of Online Science Communication: A Study Into the Scholarly Discourses of Altmetrics and Their Reception,” posted on SSRN last year concluded that “up to now, we only partly know how the performativity of Open Metrics are going to develop, given the proliferation of the discourses. Will writing blogs, for instance, be associated with societal impact? Or is a particular altmetric score going to be identified with a particular value? Does the altmetric score of 56 mean that I have managed to be accountable for my research?”
“Such awareness does not only affect researchers,” the article continues, “but is also performed in the realm of scholarly discourse where we can see a surge of scholarly articles that deal with issues of scrutinization of communication metrics. The fact that these data cannot only be tracked but also integrated in day to day practices does not only become related to the surface of scholarly publishing but also to scholarly self-characterization and self-marketing. More studies are necessary, for instance, to uncover how and why particularly blogs are more strongly referenced and integrated in scholarly publication strategies. User motivation studies for online science communication should receive particular attention.”
Clearly the very existence and use of altmetric attention scores deserves ongoing research to determine the role and impact of visibility as a reliable measure of the impact that research actually achieves. However, for now, it is time to celebrate Altmetrics and the amazing success and acceptance that it has engendered in just a decade.
CELEBRATING TEN YEARS THAT CHANGED RESEARCH METRICS
“The field of altmetrics has flourished due to a diverse ecosystem of commercial and nonprofit innovators and data providers. We need to protect this ecosystem by ensuring altmetrics data remains open for researchers to study, and that data sources remain open for collection and integration into altmetrics services” Altmetric CEO Kathy Christian noted in the 10th anniversary report on Altmetric. “Altmetric is proud to have spent the last decade as a change leader in the wider field of altmetrics.”
In the second part of this series, we hear from Altmetric staff themselves on how they see the company, the movement and the future.
By: Nancy K. Herther, writer, consultant and former librarian with the University of Minnesota Libraries