E-Resource Usage Statistics: Finding the Right Mix
by Cheryl E. Highsmith (Coordinator of Electronic Resources and Serials, Chapman University) <[email protected]>
As Coordinator of Electronic Resources and Serials for the Leatherby Libraries of Chapman University, over the past several years, I have re-envisioned usage statistics gathering and reporting and promoted data-driven decision-making by infusing such principles into the renewal process. Recognizing that e-journals, eBooks, and databases each represent unique types of e-resource content, and that the usage metric employed for each should be selected accordingly, I implemented a genre-specific approach to gathering usage statistics. Prior to this change in emphasis, only the metric of searches had been reported and considered locally across all e-resources.
This approach did not appear to distinguish standardized Counting Online Usage of Networked Electronic Resources (COUNTER) reports from vendor-produced or vendor-provided usage metrics, the latter of which was not routinely compatible for the purposes of comparison with the former. There was no documented attempt to reconcile or provide context and perspective to these disparate bits of data. As a result, comparative analysis was not undertaken at a deeper, meaningful level. Data, whether standardized or non-standardized, for databases — “apples” — and e-journals — “oranges” — were compared as if equivalent, while one eBook collection — the sole “banana”— was thrown into the same “fruit salad.”
Currently, standardized metrics are the preferred metrics and are gathered, where available, across platforms. These sources are collated, examined, and reported on a fiscal year basis:
• database searches [sources: COUNTER Database Report 1 or equivalent]
• e-journal full-text requests or downloads [sources: COUNTER Journal Report 1 or similar]
• eBook sections [source: COUNTER Book Report 2]
Searches are pulled from the Database Report 1 (Total Searches and Sessions by Month and Database), whereas full-text article requests or downloads are pulled from the Journal Report 1 (Number of Successful Full-Text Article Requests by Month and Journal) or similar. The relative merits of available reports are weighed in special cases, such as electing to examine and record usage from the Journal Report 1 (as opposed to the Database Report 1) for PsycARTICLES. The applicable local complexities of eBook usage gathering and data examination will be addressed later.
These operations are conducted with basic spreadsheets — no proprietary vendor tool of any kind is used, other than employing options to have reports sent directly to email accounts. Usage is gathered based on the local fiscal year, which runs June to May. We leverage this information into evidence-based decision making for systematic review concerning renewal or cancellation and upgrading of simultaneous usage restrictions. Data that are not compatible with the COUNTER framework may be gathered and examined separately on a case-by-case basis, but the ability to view comparable data sets over the same time frame is strongly preferred.
As chair of the Electronic Resource Committee (ERC), I schedule and lead the annual review of usage statistics, which is held in the fall to coincide with the annual renewal cycle. Usually dominating the discussion is journal usage data analysis, due to the ongoing conversion of journal titles to online-only format (where feasible), and there is a keen interest in acquiring e-journal backfiles. Recommendations for cancellations are made by this group according to analysis of usage, with the identification of duplicate coverage often an accompanying benefit. Suggestions or requests for new journal subscriptions are solicited from liaison librarians, and final decisions on new or restored subscriptions are made by the ERC. Currently, budgetary support for journal subscriptions is drawn from joint funds without expressed distinctions among subject areas. There is no expectation that the library will make one-time purchases of e-journal backfiles in any given year. Such purchases have been opportunistic, utilizing available end-of-fiscal-year funds, perhaps in response to consortial offers.
The clearly preferred source for making collection management decisions regarding current e-journal subscriptions is the Journal Report 1, but equivalent reports may be compatible and suitable for the purposes of cross-comparison. I fold corresponding print journal title usage into annual reporting and cross-reference this internal use count with any vendor-provided usage data for the equivalent electronic format journal. An expressed collection management goal is to dispense with print format journals (wherever possible) in favor of the greater access and ease of analysis afforded by the online format. Such conversions must allow for IP-authenticated access for the authorized user community, and some existing subscriptions stubbornly refuse to comply with the requirement. For a little over one year, we have made a portfolio of e-journals accessible to our users without mediation via block purchases of pay-per-view (PPV) downloads and now consider usage of all of these titles as if each were subscribed.
The cost-per-use (CPU) is calculated, where discernable, and benchmarked in alignment with the average cost of the standard ILL transaction as promulgated in the literature. Other considerations may include the impact factor, the size of the academic programs served by the title, and the length of time a journal converted from print to online access has had to find an online audience. Where applicable, the reliability of the hosting platform, platform access fees charged by the publisher or host vendor, and the university’s plans for new or expanded academic programs may factor into the decision-making process.
The most in-depth analysis of usage is reserved for renewal decisions for subscribed journal titles or expansion of simultaneous usage. Despite the primary focus on subscribed e-journals or purchased back files, on the horizon is a more systematic examination of the usage of e-journals uniquely accessible via aggregated full-text databases, where full-text access is often subject to content embargoes. Close examination of the raw data reports has proven to yield benefits for e-journals, particularly where such scrutiny has uncovered flaws in vendor reporting, including the identification of subscribed titles for which usage data was not being provided.
We are reasonably satisfied with our ability to derive the CPU metric for individual e-journal titles and, currently, have several platforms hosting unique title lists of varying sizes. In such cases, there is no bundled collection cost, which serves to obscure the cost of these titles, and it is relatively easy to cancel individually-subscribed titles annually.
Deriving the holy grail of CPU for databases has proven more problematic, due, in large part, to the bundling of product pricing and an array of databases that are offered as “free” add-ons when the library subscribes to a key, expensive database. Another scenario entails a database for which access is complimentary, as long as the library annually renews all the other subscribed products from the vendor.
Database usage is examined on a case-by-case basis, but the assumption is that the majority of all databases will be renewed annually. Aside from low usage, other considerations for cancellation are high cost, low use, duplication of content, and sustained or repetitive technical problems preventing reliable access.
The eBook genre is coming into its own. Arguably, it remains in a less standardized state with regard to uniform vendor business models than either databases or e-journals. We subscribe to relatively few eBooks, and CPU, per se, is not currently analyzed. Rather than clustering eBook management decisions annually, collection reviews may be made in response to vendor offers throughout the fiscal year, and usage reports are pulled and examined to support purchasing decisions per platform as needed.
We are still facing the fact that our select eBook vendors have not congregated around a primary metric. We will now be providing a split metric overview usage report, with the preferred COUNTER Book Report 2 (Number of Successful Section Requests by Month and Title) competing with the second gathered metric of the Book Report 1 (Number of Successful Title Requests by Month and Title). Where the COUNTER Book Report 3 (Turnaways by Month and Title) is applicable, these data may inform future expansion of access, when such options are financially feasible. A single platform currently houses a high percentage of eBook titles under simultaneous usage restriction. This collection has now moved to a platform where the Book Report 3 is available for analysis. Usage for scattered eBooks available on database platforms is not routinely examined or reported, as of now.
Continuity of access is a key factor in analyzing usage and is closely tied to judicious use of collection funds. Raw vendor reports are retained and archived for future reference. Each genre has a dedicated overall fiscal year compilation spreadsheet where the primary arrangement is by vendor or publisher. Ideally, usage statistics would be reductive to one all-encompassing metric. But in the interest of granular examination of usage, for the time being, we are pursuing the worthy goal of comparing apples to apples until such time as that elusive ideal of the one-size-fits-all metric becomes a reality. As we continue to accumulate stored data, time series reporting where grand fiscal year totals are entered into master spreadsheets for continuing e-resources, per genre, allows for usage overview and analysis of trends. Reports in this format must account for such variables as the occasional database and e-journal migration, with the resultant potential overlapping transitional usage data. Other factors challenging continuity in reporting include the detailing or documenting of cancellations or cessations, titles changes, and significant product upgrades.
Future trends and events will necessarily dictate a reflection on existing practices and drive procedures. Emerging and expanding services models, such as patron-driven acquisition (PDA), may influence renewals and prove to be a more cost-effective and responsive option than outright subscriptions or purchases. We would actively consider implementation of a proprietary third-party usage gathering or loading tool, pending available funding. We recently launched a discovery service, and after I have the opportunity to review its impact on the recorded usage of electronic resources, I will act on my observations and suggest refinements for in-house usage gathering, reporting, and analysis, accordingly. The now-combined format coverage of the COUNTER Code of Practice for e-Resources: Release 4, with the deadline date for implementation of 31 Dec. 2013, will inform a reexamination of internal practices, a realignment of reporting priorities, as needed, and the anticipated incorporation of new vendor-provided reports into the mix.
Driven by ever-changing vendor options, the e-resource landscape will continue to evolve. A flexible approach in the management of electronic collections will entail being proactive in exploring new options, while reacting analytically to the data content of usage reports. For the immediate future, the “orange,” “apple,” and “banana” representing the three genres remain in the usage statistics mix, but may be joined in the future by new ingredients.