A longitudinal study of the development and evaluation of an information literacy test

Abstract

Purpose

This article seeks to provide an overview of the development, construction and evaluation of an information literacy testing instrument connected with the VOILA! online information literacy tutorials, developed by librarians at Hunter College libraries for students enrolled in the college's first‐year orientation seminar (ORSEMs).

Design/methodology/approach

The VOILA! online information literacy tutorial, developed by librarians at Hunter College, has a test as one of its components. All incoming students registered in the college's orientation seminars (ORSEMs) are required to go through the VOILA! tutorials and take the test. The longitudinal development and evaluation of the test are reviewed, including initial statement of competencies, test creation, test construction (assessment type, question formulation and question type) and test evolution. Validity and reliability are also discussed in detail. Finally, a review of the test results including general statistics and the connection of student performance to test construction is provided.

Findings

Test construction was given a great deal of attention during the development of VOILA!, as librarians wanted an instrument that was valid and reliable. The importance of iterative test design and attention to the details of test construction is stressed in the article, and the methodology used to determine validity and reliability is presented. Those interested in testing information literacy skills must be able to examine the instrument, the results and the design process in a way that contributes meaningfully to the preparation of students to undertake research at the college level.

Research limitations/implications

Librarians will continue to modify the VOILA! test in keeping with evaluation results, and the assessment will continue. Thus the results presented may not fully represent the final evaluative outcome.

Practical implications

This research is useful for information professionals who are interested in test design, construction and evaluation. The longitudinal study presents clear, practical information that can be used as a case study for other libraries interested in this area. The VOILA! test itself is included as an Appendix, and can be used as a model for other libraries to consult.

Originality/value

A literature review revealed a number of articles about testing and information literacy, but very few, if any, articles approached the discussion from a longitudinal test construction standpoint. In this way, the study presents information that is original. The study would be valuable for information professionals who are interested in test design, construction and evaluation.

Citation

Ondrusek, A., Dent, V.F., Bonadie‐Joseph, I. and Williams, C. (2005), "A longitudinal study of the development and evaluation of an information literacy test", Reference Services Review, Vol. 33 No. 4, pp. 388-417. https://doi.org/10.1108/00907320510631544