Search results

The purpose of this paper is to report a range of innovative approaches, methods and tools for evaluating the impact of public libraries and supporting the development of library leaders drawn from projects supported by the Global Libraries initiative of the Bill and Melinda Gates Foundation.

A range of approaches, tools and methods are described by specialist staff who were directly involved with these innovations.

Some comments are provided on the effectiveness of these innovations.

All of these innovations have been tried out in public library settings, usually over a period of years.

All of the selected innovations could be adapted and used by library leaders or impact specialists.

Several of the innovations are specifically designed to engage library staff and users in evaluation processes.

The impact planning and assessment programme of GL was the largest sustained international public library evaluation programme so far attempted. This paper reports on the more innovative evaluation activities undertaken through this programme.

This paper aims to explain how the Global Libraries (GL) Initiative of the Bill and Melinda Gates Foundation plans for and assesses the impact of its programs on libraries and communities as part of their commitment to improving lives by increasing public access to computers and the internet in public libraries.

GL encourages libraries to design programs and services that address local and national priorities. Impact assessment processes assist libraries in collecting, analyzing and reporting on evidence to show: whether activities are conducted effectively to enable learning and improve processes; whether the program makes a difference to users and communities; and what impact evidence to use in advocacy for continued support and funding. GL's process and philosophy are described and case studies from two grantee programs in two countries are provided.

The Polish‐American Freedom Foundation (PAFF) adopted innovative methods developed for them by the Information Society Development Foundation to determine baseline library usage in rural public libraries in Poland. Rather than counting borrowers, which was the approach facilitated by the available data, they conducted random sampling in target areas. This allowed them to count users who do not borrow books, and to gauge scale of repeat usage and user demographics. In so doing, the team found that libraries have a substantially higher reach than book‐lending data alone suggested. The International Research and Exchanges Board (IREX) assessed community information needs and determined that education, health, and employment are the issues about which Romanians want more information. Libraries in Romania will design services based on a careful understanding of those needs.

The research findings reported are drawn from systematic quantitative and qualitative evidence gathering undertaken as part of the impact‐planning and assessment programs being implemented in the two countries.

The paper presents a current view of the GL approach to impact assessment and service sustainability as well as two early reports, one on the results of baseline evidence collection and the other on community needs assessment in the two participating countries.

The purpose of this paper is to report on performance measurement and impact assessment progress made in 14 countries as part of the Global Libraries initiative, starting with the early country grants in Mexico and Chile. For the mature grants in Bulgaria, Botswana, Poland, Romania, Ukraine and Viet Nam which were recently completed or are approaching completion, the nature of the country program is outlined, before the impact assessment work is described and some recent results and conclusions are reported. A similar approach is adopted with pilot and new grants in Colombia, Indonesia, South Africa, Turkey and Lithuania.

The country reports are presented as a series of case studies, in some cases supplementing those in an earlier special issue of this journal.

Where appropriate, recent country-specific survey findings are reported.

This paper shares Global Libraries IPA learning at country level with people in other countries who may be contemplating public library evaluation at regional, national or local level or who are interested in performance measurement and impact evaluation.

These cases studies reflect concentrated impact assessment and performance measurement work at country level across a range of countries over more than 12 years.

The purpose of this paper is to describe a range of innovative (for public library performance measurement and impact assessment) methods and tools developed by country teams as part of the Global Libraries (GL) initiative. Short reports are provided on: a return on investment study, a simplified data processing system for library managers and an online reporting system for public libraries in Ukraine; a study of the public image of Polish libraries in print mass media, two approaches to sustainability of performance measurement and impact assessment in Romania, through tools to conduct pop-up surveys and use of agricultural subsidies support data, assessments of changes in public library managers’ planning efforts in Poland and of their perceptions of libraries and their own role, using Modified Delphi forecasting, in Lithuania, two ways of focussing on the world of public library users by engaging non-profit organizations in library research in Poland and conducting impact studies in virtual environments in Chile.

A range of methods and tools and their uses are described.

No specific research findings are reported.

All of these tools and methods have been (or are being) trialed in national public library contexts; some have been developed over several years.

Useful for people in other (non-GL) countries who may be contemplating public library evaluation at regional, national or local level or who are interested in performance measurement and impact evaluation.

This paper is part of a GL effort to share what participants have learnt about impact planning and assessment in public libraries with the wider international libraries community.

The impact planning and assessment program of GL has been the largest sustained international public library evaluation program so far attempted. This paper reports on the more innovative evaluation activities undertaken at country level through this program.

The purpose of this paper is to emphasize some of the qualitative results from the data collection throughout the second year of the Biblionet Program in Romania.

Concentrated here are the conclusions drawn from reports of the County Coordinators, online questionnaires, quantitative and qualitative research reports, and information from colleagues in the Biblionet team.

The results and findings have been grouped under the four key objectives of the Biblionet Programme: the citizens' information needs are better fulfilled with the help of public libraries; librarians provide better services to communities; for the National Association of Librarians and Public Libraries in Romania (ANBPR) to improve its capacity to provide services to members; and public authorities to increase their support for library development.

A wide range of improvement examples are provided that can be easily transferred to other countries.

This paper aims to describe the main impact and performance measures employed by the Global Libraries – Biblionet Romania programme (GL‐Biblionet) and reports on the pilot application of a pop‐up survey designed to provide focused user‐feedback to inform the development of the programme.

GL Biblionet aims to ensure that community informational needs are increasingly met at public libraries, and that these in turn provide better services to users and the community. An efficient performance management system of public library computing service is required for these purposes, including an adequate data/feedback collection system, and a flexible mechanism to analyse and report the findings. A pop‐up survey mechanism was developed and tested as a key part of the user feedback element of impact and performance measurement.

The pilot implementation of the pop‐up survey was broadly successful in providing useful data but the survey results need to be contextualised and triangulated with other impact and performance measurement data to assess their level of reliability. More nuanced questions about users' self‐assessments of competence in using ICT need to be developed.

Although only at the pilot stage, the GL‐Biblionet approach to pop‐up survey development offers a potentially useful tool for long‐term use by public libraries in Romania and for other countries faced with similar public library development and user feedback issues.

This paper reports on a novel and potentially valuable application of pop‐up survey technology for national systems of public libraries and for local or regional application.

As more countries have joined the Global Libraries (GL) Initiative, the IPA Specialists have gradually evolved into a cohesive team that finds opportunities to meet fairly regularly. Their discussions about the practicalities and challenges of conducting IPA at the country level have drawn attention to a number of issues that are likely to be relevant to other IPA specialists working in similar roles in other countries and to library service managers who are interested in impact evaluation. This paper aims to summarise some of the collective experiences.

Several of the IPA Specialists have “adopted” an issue. Some describe their own country approach to the topic; others have been more analytical in their presentations.

Opinions and findings about four topics are described. They are: sharing learning within the country and sharing beyond that country; ensuring that IPA evidence reflects government and international priorities, and finally combining qualitative and quantitative evidence‐forming strategic collaborations and telling stories.

The value of the paper lies in informing other IPA specialists in similar roles in other countries, and library service managers who are interested in impact evaluation.

The purpose of this paper is to address an important but neglected aspect of evidence-based management, how to apply actionable data to strategic decision making to support the organization in reaching appropriate decisions. This process involves connecting advocacy work with active measures to collect and interpret impact data, which are the real ingredient to help understand the big picture and make advocacy more efficient.

The paper presents two complementary models of intervention to better use relevant data to help management decisions, within the program team and beyond the program team. In both cases the paper attempts to identify the key factors in understanding outcome evidence in order to have better communication and help the decision-taking process.

Customizing evaluation results in order to present the information in an accessible form for various stakeholders appears to be a key factor in delivering the right message and having a successful advocacy campaign.

This paper reports on two complementary approaches to persuade stakeholders (program managers, or stakeholders external to the program team) to take action based on specific impact or management data delivery. The idea is potentially appropriate for any program, or project, where advocacy processes are needed to determine appropriate actions. Processing and presenting data in an actionable way is a key success factor to determine the expected management decision or successful advocacy step.

The purpose of this paper is to report on the findings from the pilot stage evaluation of the Global Libraries – Biblionet Romania project. The project involves introducing public access ICT via public libraries in Romania and the pilot evaluation was designed to ensure the adoption of an appropriate impact planning and assessment program for the main project.

Since the start of the Global Libraries – Biblionet Romania project, planning and managing for results have been the guiding philosophy. The team was concerned to find the right balance between what is too much for librarians in terms of data collection, and what precisely to promote as useful metrics with which the librarians are comfortable and which do not create too much extra work. A step‐by‐step approach has been used to involve librarians in decisions, facilitate consensus, and double‐check agreements, understandings and results. This involved phases of engagement with users in the pilot project areas, librarians, and the national professional body.

The pilot has revealed some of the key characteristics of library patrons as public access ICT users, some of the major factors in implementing public access computing through libraries and important elements of the capacity for change in the national professional body, all of which will inform the design and planning of the main project.

Librarians in the 12 pilot sites had different levels of expertise and interest in data collection and different levels of understanding of the relationship between impact assessment and service development. The paper expresses how one can quantify the benefits of using computer and internet in public libraries. Handled skilfully, this approach can be a strong advocacy tool to communicating with public authorities and other local or national stakeholders. Although this was a pilot project, it was substantial in scale and provides useful insights into all facets of a major implementation program as well as important assessment data on Romanian public libraries.

The pilot project has already provided original evidence of the significance and value of public libraries in Romania. (The main project that will be designed on the basis of this evidence will in due course add a major and continuing further dimension to this process of assessment.)