Home 9 Against the Grain 9 v31#4 Assessing the Success of Library Published Journals

v31#4 Assessing the Success of Library Published Journals

by | Oct 4, 2019 | 0 comments


by Emma Molls  (Publishing Services Librarian, University of Minnesota) 


In 2015, the University of Minnesota (UMN) Libraries launched a publishing program.  Following in the footsteps of earlier library publishing initiatives, the program seeks to serve campus by providing an affordable venue for publishing quality open access journals and other openly accessible formats.  However, given UMN Libraries Publishing’s diverse portfolio and its non-traditional business model, assessing the program’s success required innovative thinking that did not depend on sales figures or disciplinary-specific metrics.  This paper will discuss UMN Libraries Publishing’s development of journal-level goals to assess publication-level success and, in turn, assess the success of a library publishing program.  The paper will present examples of journal goals, templates for editorial boards, and timelines for publishing workflows related to assessment.

We’re Here, Now What?

In many ways, the University of Minnesota Libraries was late to the library publishing party.  University of Michigan Libraries and University of Pittsburgh Library, arguably two of the most robust library publishers, launched publishing initiatives in 2001 and 1999,1 respectively.  Prior to 2015, University of Minnesota Libraries was able to serve campus publishing and publishing-adjacent needs through other existing library services, including the institutional repository (launched in 2007) and digital project assistance offered through the web development department.  After a sharp increase in consultations from on-campus, faculty-led journals, the University of Minnesota (UMN) Libraries launched a publishing program in 2015, built heavily off of the experiences and direction of long-standing library publishing programs at peer institutions.  After four years, UMN Libraries Publishing now offers in-depth publishing consultations and open access journal, monograph, and textbook publishing services to campus-affiliates and scholarly societies.

The University of Minnesota Libraries deliberately and thoughtfully created an open access publishing program, knowing that the program’s success would not, and could not, be measured by the same means as for-profit or self-sustaining publishers.  Likewise, library publishing programs, like other library services, need to meet the current needs of campus, which requires frequent reassessment and flexibility. The innovative approach of library publishing within a campus context, therefore, requires a complex answer to the question: “What does success look like for us?”

The Struggle with Metrics

Across UMN Libraries Publishing’s catalog, we publish journals on three different platforms, Open Journal Systems, WordPress, and bepress Digital Commons, and monographs and textbooks on three different platforms, Manifold, Pressbooks, and DSpace.  Depending on each individual title, some journals publish downloadable PDFs, while others publish full-text HTML. Similarly, depending on the title, monographs and textbooks are available for reading and downloading at the book level, or by chapter-by-chapter.  This diversity of content availability became increasingly problematic in creating internal annual reports used to illustrate the publishing service’s growth and activity to library administration. Even with using a standard like COUNTER,2 when available, is a challenge for measuring library publishing activity because it does not capture nuance across titles, including frequency of publication, number of articles published annually, and size of potential audience.  Although a publisher would like to assume that every person with internet access would be a potential reader of an open access journal article, realistically, depending on subject area and content of individual articles, each journal, and even each article, has a uniquely sized potential audience.  All of these caveats make presenting a straight-forward, usage statistics-centric story of success difficult. While UMN Libraries Publishing still tracks journal-level usage statistics, we are working toward telling our story of success through the success of each journal.  This approach requires that we, as publishers, are in close contact with each journal’s editorial board to identify and meet their objectives and successes. 

To Each Their Own

One of the first UMN Libraries publications, Open Rivers3 (ISSN 2471-190X, published fall 2015), was proposed and developed as a WordPress publication, and therefore required additional conversations and planning around design and website development, especially compared to journals on more templated platforms like bepress Digital Commons.  Led by UMN Publishing Team members with library technology backgrounds, editors and librarians walked through a user design template that asked editors to consider different user segments, mentalities, and key site objectives.  While this template was being prepared primarily for technology needs, it became clear that walking editors through deeper questions related to the publication’s goals needed to be incorporated into the workflow for all publications.  Publishing Team members adapted the user design template to include further questions that were relevant to the non-technology aspects of scholarly publications. These added sections, listed in Table 1, were in large part success-oriented.

After three years of revisions, the former user design template is now referred to as “New Publication Goal Development Questionnaire” and includes the following sections, descriptions, and instructions as listed in Table 1.  The questionnaire is answered by publication leads, typically editors or a steering committee, but frequently facilitated in-person by a member of the Publishing Team.

It became apparent after working with both brand new journals and journals with a longer publication history, that journals would need to have a different measure of success for the short term (one year), the midterm (three years) and the long term (beyond five years).  While it may seem redundant to have separate measures of success for years three and five, this breakdown is specifically linked to indexing and discoverability, which many journals identify as a measure of success. PubMed Central4 and Scopus5 require at least two years of regular publication to be included in the indexes;  however, it may take a journal an additional two years to get through index review.  For brand new titles, goals of year one are typically heavily tied to author and reviewer recruitment and basic editor on-boarding and journal management.  It is important to acknowledge that these goals are not abandoned after years three and five, but should be embedded into the everyday work of the journal editors.

Given the separate measures of success for different years, we are moving toward regular check-in meetings with each editorial team in order to revisit the New Publication Goal Development Questionnaire.  These meetings, especially with mature titles, often result in adding additional yearly goals to the journal. Adding additional goals gives rotating editorial team members a way to leave their “mark” on a journal and often seek to strengthen the journal’s competitiveness in its respective field.  As long as these goals can be tied to the journal’s original business objectives and can be described as a measure of success, there is no problem in expanding the answers to the Goal Development Questionnaire. 

The New Publication Goal Development Questionnaire is a great tool for publishers.  Outside of the initial publication build, the questionnaire is used to help scope additional developments.  For example, if one of our publications has listed “inclusion in PubMed Central” as a measure of success for year five, we know that workflows for XML production must be in place prior to inclusion. 

Their Success, Our Story

UMN Libraries Publishing was launched in order to meet campus needs and serve campus — and part of our service-oriented program is to ensure that each publication is successful by a measure that fits its context.  We believe as an open access publisher, embedded within a public university, we are in a unique position to create a publishing environment that puts academy-led journal priorities first, rather than publisher revenue. 

While we are still figuring out how to best share our story in a quantifiable way, we know what our story is: we succeed when our journals succeed.  Focusing on journal measures of success has implications across an entire publishing program — from proposal application to publication build — and helps shift the focus of publishing back on scholars.  


1.  Library Publishing Coalition Directory Committee.  (2018). Library Publishing Directory 2018. 2016D11D14].  http://www. librarypublishing.org/resources/directory/lpd2017

2.  COUNTER is a standards organization that develops and maintains the standard for counting the use of electronic resources.  The COUNTER Project. https://www.projectcounter.org/about/

3.  Open Rivershttp://editions.lib.umn.edu/openrivers/

4.  National Library of Medicine.  (2019). Journal Selection for MEDLINE Indexing at NLM.  https://www.nlm.nih.gov/lstrc/j_sel_faq.html

5.  Elsevier.  (2019). Content Policy and Selection.  https://www.elsevier.com/solutions/scopus/how-scopus-works/content/content-policy-and-selection


Submit a Comment

Your email address will not be published.

Share This