Evaluating and Measuring the Value, Use and Impact of Digital Collections

Citation

Huysmans, F. (2013), "Evaluating and Measuring the Value, Use and Impact of Digital Collections", Performance Measurement and Metrics, Vol. 14 No. 1, pp. 93-94. https://doi.org/10.1108/14678041311316167

The last decade has witnessed massive investments in the digitization of scholarly and cultural heritage materials worldwide. Both from a preservation and presentation point of view, governments and businesses have devoted large sums of money to scan analogue objects, and store and disclose their digital representations. The digitization process being still in its infancy in view of the five millennia of cultural production (or what remains of it) that preceded it, it is remarkable how much has been “learned by doing” in this short time span. The initial optimism – “once digitized, an object has been saved forever” – quickly gave way to a more realistic view, as both heritage institutions and funders became aware of the costs of now having to preserve both the physical and the digital objects in the future. File and metadata standards as well as software applications appeared to evolve so rapidly that emulation and migration would become the rule rather than the exception.

And that is only the supply side of the picture. What institutions have also learned is that digital supply will not automatically create demand. While funding streams have rapidly dried up in the current economic recession, the question what digitization actually yields in terms of use, value and impact becomes all the more pressing. With Evaluating and Measuring the Value, Use and Impact of Digital Collections, a group of authors largely of British origin has tried to fill the need for institutions struggling with the showing‐value‐for‐money question. This had led to a concise volume of 11 chapters divided up in three parts:

1. (1)

   digital transformations in libraries, museums and archives;

2. (2)

   understanding and measuring the use, impact and value of digital collections; and

3. (3)

   enhancing the future impact and value of digital collections.

The first part, with separate chapters on libraries, museums and archives going digital, is useful in introducing the actual theme of the book. One cannot evade the impression, however, that the authors did little to elaborate on the individual cases they are familiar with. However, informative the individual case studies they present may be, little evidence is given of how representative these cases are for other types of libraries, museums and archives. The reader knows or can at least imagine how diverse national and public libraries, art and natural museums, and national and private archives are. This leaves the first part of the volume lacking in relevance for large parts of the fields to be covered.

The second part starts with the observation that “[…] while it is undoubtedly a cliché, there is some truth in the argument that until relatively recently digital resources have often been created with a ‘build it and they will come’ mentality” (p. 63). When “they” don’t come, or they do but not in numbers impressive enough to the funders, the services built are in severe danger of disappearing as quickly as they have been created. Ben Showers of JISC (UK) discusses why and how use and impact assessment should be part of the digitization strategy of institutions, more specifically of the life cycle of any of their products and services, to prevent them from becoming data cemeteries. As a spin‐off of a number of impact evaluation studies, JISC has offered a toolkit institutions can gear to their specific information needs and then feeding it back to the community. The next chapter by Dobreva, O’Dwyer and Konstantelos focuses on user needs, putting user experiences in accessing the digitized content in the forefront of impact assessment. Theirs is an instructive framework for practitioners in institutions involved in digitization programs. It is followed by a chapter on measuring scholarly information seeking behaviour illustrating one particular evaluation study in great detail, but without attempting to draw wider lessons for other institutions. A nice but rather impressionistic piece by Tanner, resulting in five modes of value people derive from digitized resources – option, prestige, education, existence and bequest value – concludes this section.

The third and last part offers three instructive chapters on using ICT tools and methods in arts and humanities research; on how to build an infrastructure that keeps research data accessible over long periods of time, and on the sustainability (keeping resources instantly usable) as opposed to mere preservation (without emulation/migration efforts being made).

Overseeing the whole volume, it must be said that it does not live up to its ambition as described in six ambitious questions in the introductory chapter by the editor. The collection of chapters would have been much more valuable had the contributors agreed on some common framework connecting concepts like “value”, “impact” and “outcome”, let alone “use”. Conceptual rigour is, admittedly, not easily attained in more “qualitative” matters of impact, but the volume would definitively have gained in usefulness for many readers had the authors departed from the same concepts. For readers not working in the UK or the Anglo‐Saxon world, it is irritating at times that all cases stem from the English‐speaking world, and that it is taken for granted that readers will be familiar with policy‐making peculiarities as well as acronyms and names of the mainly British institutions. Add to that the too widely cast framework of the book, resulting in considerably reducing the internal variety of the respective library, museum and archive sectors, then it is clear that a revised edition of the book would benefit considerably from tackling these shortcomings.