v29 #5 Future Through the Past — QEP Impact & Conclusion

by | Dec 11, 2017 | 0 comments

Column Editor:  Donald Beagle  (Director of Library Services, Belmont Abbey College, 100 Belmont – Mt. Holly Road, Belmont, NC  28012-1802;  Phone: 704-461-6740;  Fax: 704-461-6743)

Column Editor’s Note:  In my previous two “future-through-past” ATG columns (see v.29#2, April 2017, p.52 and v.29#3, June 2017, p.67) about the structure and outcomes of the QEP at Belmont Abbey College, I commented that this final column would summarize the interesting assessment results that flowed from our PILOT project, and its dual-focus structure organized around both Information Literacy and the development of our Learning Commons. — DB

 

The independently-validated test results from SAILS (Standardized Assessment of Information Literacy Skills) contrasted scores from our entering freshmen in 2010 with graduating seniors in 2014.  In our initial QEP Project Proposal, we had speculated (hopefully) that our dual-focus on a) IL instructional support, and b) learning space innovations conducive to collaborative learning and constructivist pedagogies, would yield scoring results at least equal to overall SAILS results from peer 4-year liberal arts colleges.  Our actual freshmen-to-senior improvements went well beyond that.  (As noted in a prior column, the SAILS test structure effectively consolidates six ACRL Information Literacy Competency Standards of 2000 into four SAILS Q&A student scoring outcomes.)

For Standard 1 (2010-2014), “Determines nature and extent of information needed,” our students showed a 9% scoring improvement, as compared with a 5% improvement for their counterparts across all 4-year colleges using SAILS, and a 2% scoring improvement when all university results were included.

For Standard 2 (2010-2014), “Access needed information effectively and efficiently,” our students showed a 10% scoring improvement, as compared with a 3% improvement for their counterparts across all 4-year colleges using SAILS, and a 0% scoring improvement when all university results were included.

For Standards 3/4 (2010-2014), “Evaluates information and its sources critically, and incorporates selected information into his/her knowledge base,” our students showed a 5% scoring improvement, as compared with a 1% improvement for their counterparts across all 4-year colleges using SAILS, and a -3% scoring decline when all university results were included.

For Standards 5/6 (2010-2014), “Understands social, legal, and economic issues surrounding use of information, etc.” our students showed a 14% scoring improvement, as compared with a 10% improvement for their counterparts across all 4-year colleges using SAILS, and a 4% scoring improvement when all university results were included.

These results for the 2010-2014 testing regime were obviously very encouraging, but would they be reinforced or undermined by subsequent results in the next 4-year cycle? In fact, the next 4-year testing regime strongly reinforced the first set of results.  They second cycle again showed freshmen-to-senior IL scoring improvements at Belmont Abbey College significantly better than corresponding outcomes across peer 4-year colleges and also better than results across all SAILS-testing institutions.

For Standard 1 (2011-2015), “Determines nature and extent of information needed,” our students showed a 14% scoring improvement, as compared with a 1% improvement for their counterparts across all 4-year colleges using SAILS, and a 1% scoring improvement when all university results were included.

For Standard 2 (2011-2015), “Access needed information effectively and efficiently,” our students showed a 12% scoring improvement, as compared with a 2% improvement for their counterparts across all 4-year colleges using SAILS, and a 1% scoring improvement when all university results were included.

For Standards 3/4 (2011-2015), “Evaluates information and its sources critically, and incorporates selected information into his/her knowledge base,” our students showed a 9% scoring improvement, as compared with a -2% decline for their counterparts across all 4-year colleges using SAILS, and a -3% scoring decline when all university results were included.

For Standards 5/6 (2011-2015), “Understands social, legal, and economic issues surrounding use of information, etc” our students showed a 9% scoring improvement, as compared with a 6% improvement for their counterparts across all 4-year colleges using SAILS, and a 2% scoring improvement when all university results were included.

Implications & Questions

These independently verifiable results raise one obvious question: can any single factor in our QEP be identified as being primarily responsible for our freshmen-to-senior SAILS test scores showing steeper improvements than corresponding freshmen-to-senior SAILS test scores from peer colleges and from all institutions?

The single factor that most sharply differentiated our Information Literacy QEP from all others we studied in the 2008-2010 proposal formulation period was our dual focus on IL instruction AND the simultaneous implementation of our Learning Commons.  It is, therefore, very tempting to say that this dual focus was responsible for our SAILS testing scores showing superior results to colleges and universities whose IL QEP’s placed sole focus on IL instructional activities.

There is, however, one serious gap in our knowledge about institutions using SAILS: we have no data about which college and university libraries employing the SAILS test during that time period did or did not have spaces identifiable as Information Commons (IC) or Learning Commons (LC).  It is an open question whether a retrospective study of colleges and universities using SAILS from 2009-10 to 2014-15 could uncover data about the presence or absence of IC / LC spaces.  It seems especially unlikely that such a study would find enough institutions whose IC / LC implementations corresponded exactly with the start of an IL QEP to make meaningful comparisons.

It therefore seems unlikely that any future research can reliably replicate the outcomes demonstrated by the IL QEP at Belmont Abbey College for the simple reason that the ACRL IL Competency Standards of 2000 have now, of course, been replaced by the “Framework.”  But it is clear that IC / LC implementation has continued in numerous college and university libraries since 2015, and new testing protocols designed around the “Framework” (including one from Project SAILS) are now available.  It will be a matter of significant interest to see whether future statistical correlations appear between implementation of IC / LC facilities and IL test freshmen-to-senior scoring improvements.  

 

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

Share This