Measuring Academic Research: How to Undertake a Bibliometric Study

Citation

Haddow, G. (2010), "Measuring Academic Research: How to Undertake a Bibliometric Study", Library Management, Vol. 31 No. 4/5, pp. 370-372. https://doi.org/10.1108/01435121011046452

Bibliometric research is a complex area to comprehend and to undertake, so a book that promises “step‐by‐step explanations about how to carry out a bibliometric study” is welcome indeed. Sadly, for the students and novices approaching bibliometrics for the first time, Ana Andres' Measuring Academic Research: How to Undertake a Bibliometric Study falls short of meeting this promise. Readers anticipating a “how to” as suggested in the title, in which one might expect information about the intricacies of identifying data for a bibliometric study, discussion about what that data means when it has been extracted, as well as details about the assumptions and limitations researchers face when working in the field, will need to consult an alternative source. To be fair, the author states in the first sentence of the introduction that the aim of the book is “to illustrate the main analyses applied in bibliometric studies” and this is very much its focus. Hair‐splitting perhaps, but the book is a “how to” conduct analyses, rather than a “how to” undertake bibliometric studies.

Measuring Academic Research is well presented with extensive use of tables and figures to illustrate how data are organised and analysed to conduct a range of bibliometric analyses, beginning with Lotka's law and moving through to the present day with the Eigenfactor. The core content is divided into two parts over six chapters; four chapters deal with analyses that require non‐citation data and two later chapters discuss citation‐based analyses. A further three chapters, an introduction and two concluding chapters, provide the context and discourse around the development, application, and some weaknesses of bibliometric research. In describing the analyses used in bibliometrics the author makes good use of examples from existing research and devises her own examples to demonstrate the process.

Andres has identified the best known and most frequently used bibliometric indicators to explain and discuss, including: Bradford's law, the Journal Impact Factor, cited and citing half‐life, the h‐index, and the g‐index. Each indicator is discussed in terms of its theoretical and practical application, the data required, and the equation or process required. As such, the book brings together a range of indicators that might be considered by researchers to study individual, journal, group, and national patterns of publication, productivity and impact. The explanations of the equations and their component parts, as well as the examples provided to illustrate how to conduct the analyses (presented in a step‐by‐step fashion) is useful in assisting the reader to understand the process and appreciate the indicators' application. A number of indicators receive particular attention, one assumes due to their prominence in terms of longevity and/or extent of discussion in the literature, although this is not stated, with the Journal Impact Factor and h‐index accounting for almost a third of the book's content.

While the explanations and instructions provided about the analyses would be of value to readers embarking on a bibliometric study for the first time and needing background information, many of the citation‐based analyses are now performed by the major citation databases. It could be argued that much of the book's content is redundant, especially as the author refers almost solely to Web of Knowledge as the data source. This is a major short‐coming of the work. In the last six years, Elsevier has rapidly moved into a seriously competitive position as an alternative source of citation data – an issue that is discussed only in the final short chapter of the book. Further, it is compounded by the lack of information, other than three pages, which discuss characteristics of databases to consider before conducting a bibliometric study, about sources of non‐citation data.

Andres' work succeeds as a collection of the main bibliometric analyses available today, but it is undecided in terms of its audience. At times the book presents extremely complex equations and concepts, while other issues, such as the difficulties inherent in actually conducting a bibliometric study, are given scant attention. Professional librarians will find a number of reasonably emphatic statements, such as the suggestion that libraries “decide which journals to purchase … based on their impact factor” (p. 121), that they may feel compelled to challenge, and the author gives examples of search strategies that are inadequate to the search task. Those familiar with bibliometric research will be aware that many of the analyses are automated functions in the source databases and will also have a better appreciation of the history, philosophical underpinnings, and limitations of bibliometric research than the author has provided. On the other hand, researchers and students new to the field might be deterred by the lack of information about locating data for bibliometric studies, and their introduction to the field will comprise a range of elaborate equations, some components of which are not fully explained. This last criticism can be extended to a more general concern about the book, which relates in part to its clarity and consistency in expression, and also to the debatable interpretations of previous bibliometric research and discussion.