Home | Performance Measures and 
Evaluative Tools (Summary)
Full Text: Page 1 | Page 2 | Page 3 | Page 4

GOOD PRACTICE GUIDELINES 

Public library statistics
Statistics are valuable for improving standards or gaining financial support. Public libraries in Europe mostly have good statistics but issues include:

• Timeliness [they are often published “late”].

• Completeness [they may not be grossed up, making time trends impossible].

• International standardisation [they may not respect international standard definitions].

• Presentation and access [they may be poorly presented and hard to find].

• Fitness for purpose [they may not cover topics needed by stakeholders – electronic services is a current issue].

Public library performance measures
Performance measures help managers to condense statistical data into meaningful indicators of actual performance – the term ‘indicator’ may be preferred as it correctly suggests that judgement is needed in interpreting the results. However, to be useful indicators must be robust and meaningful. They must be related exactly to the service’s objectives. If used to compare one library with another, great care is needed to ensure comparability. Finally, the effort expended to collect datasets for the calculation of performance measures must be in proportion to the likely benefits from improved decision-making.

New statistical measures and other tools for evaluating electronic services 
Work has taken place, in Europe and elsewhere, to develop indicators for the provision of IT-based services. Some of these measures are relatively crude but may nevertheless be useful: examples would be the number of publicly-accessible PCs installed in public library branches and hits on public-facing web sites.

More sophisticated approaches have been explored in a number of projects, mostly with an academic library focus. The EC Telematics Programme EQUINOX Project was designed to cover all sectors. In the USA, the main work has been undertaken by McClure and Bertot.

Benchmarking [metric and process] 
Benchmarking essentially means comparison with a view to improvement. Types of benchmarking include:

• Metric [statistical comparison using either existing published statistics or figures collected specially for the purpose]

• Process [where libraries investigate how others achieve their results]

Benchmarks can be set against other public libraries or other kinds of organisations for generic functions such as purchasing or personnel management.

There are examples of benchmarking at national level in the UK and in Germany (see Links) Both models could easily be adapted for use in other countries.

Measures of value and impact
Measuring value and impact is extremely complex. The issue is best considered as part of a continuum of possible measures:

Input > Process > Output > Outcome > Impact

• Inputs include all the resources that the organisation uses in order to produce whatever service or product it wants to sell (using ‘sell’ in a wide-ranging sense). They include finance, buildings, raw materials, machinery, staff, and increasingly recognised as an important input, information. Measurement usually consists of counting, though increasingly emphasis is shifting to measures of quality. It cannot be assumed, however, that a better-resourced library is necessarily a better library.

• Processes are whatever is done to inputs to create something new. When we measure processes it is usually to see how well they are working. So a library may measure processing times for new books as a way of seeing if the process of making stock available quickly is working as well as it should.

• Outputs are the things that the organisation produces. Examples are ‘book issues’, ‘reference questions answered’ and ‘seats occupied by readers’. Very often measurement consists of counting the number of outputs produced. However, the quality of provision will be as important as quantity.

• Outcomes are the short to medium term results of applying outputs. The number of borrowed books actually read might be an example of an outcome of book issuing.

• Impact is the effect of the outcomes on the environment – again, using that term very broadly to include people and society at large – and is usually long-term. An example here might be the growth in literacy as a result of reading library books.

The main difficulty in measuring impact is that very many different factors contribute, so isolating one (such as the library contribution) is very difficult. Usually the best way forward is to ask customers for their views and/or to observe their behaviour.

Standardised questionnaires for end-users
Questionnaires are a market-research tool of great potential value to public libraries. They may be regarded as relatively high cost and there are skills in question selection, selection of samples and processing which small public libraries may find challenging. By using a standardised questionnaire, costs are reduced and value added. 

A range of standardised questionnaires in use nationally in the UK is PLUS [Public Library User Survey]. Not only do libraries save effort by using the same tool, they put the results into a common database to allow inter-library comparisons and time series [benchmarking]. Results are accepted as performance indicators for the UK Public Library Standards. There are now four questionnaires covering basic library use, use by children, views of non-users and use of electronic services.

Standards
There are two relevant international standards:

• ISO 2789:1991 Information and documentation – International library statistics. This covers how to count libraries, librarians, books, cassettes, discs, etc. for inclusion in statistics related to library provision. 

• ISO 11620:1998 Information and documentation - Library performance indicators. This specifies a set of 29 indicators grouped in the following areas: a) user satisfaction; b) public services; and c) technical services.

Both are currently [2002] undergoing revision. They are both valuable but they are not well enough known and not strong on electronic measures or value and impact.

Measuring the use of electronic resources
The use of licensed electronic information resources will continue to expand and in some cases become the sole or dominant means of access to content. The electronic environment, as manifested by the World Wide Web, provides an opportunity to improve the measurement of the use of these resources. 

There are lots of tools available to analyse web logs. A typical example is Analog. The site also provides online tutorials. It is recommended that those contemplating using statistics from web servers and the like should familiarise themselves with some of the issues concerning interpretation of such data e.g. “The Value of Web Statistics”: “Why Web statistics are (worse than) meaningless”.

New statistical measures and other tools for evaluating electronic services
Developing National Data Collection Models for Public Library Network Statistics and Performance Measures
The study (2000-01) addressed ways to design and implement a national data collection, analysis, and reporting system for public library network statistics and performance measures that is both accurate and reports data in a timely fashion. It built upon previous work (see next item) that produced a core set of network statistics and performance measures for public libraries (see below). The project directors are J.C. Bertot and C.R. McClure.

Developing National Library Network Statistics & Performance Measures
This study developed a core set of national statistics and performance measures that librarians, researchers, and policy makers can use to describe public library and library based state-wide network use of the Internet and Web based services and resources. 

Measures for Electronic Resources (E-metrics)
The E-metrics project aims to explore the feasibility of collecting data on the usage of electronic resources.

Urban Libraries Council (USA) Impacts of the Internet on Public Library Use. The research study consisted of a national random telephone survey of 3,097 adults conducted during the spring of 2000. It was conducted in either English or Spanish. 

It is possible for libraries to compare themselves with other peer libraries, using the Public Library Peer Comparison Tool or the Academic Libraries version.

Home | Performance Measures and 
Evaluative Tools (Summary)
Full Text: Page 1 | Page 2 | Page 3 | Page 4