Developing a model to measure the effectiveness of research units

Abstract

Research and development (R&D) effectiveness has traditionally been measured in quantitative terms using measures such as the number of published papers (in journals, conference proceedings, etc.); patents; technologies successfully transferred or the external cash flow secured by a R&D organization. These are at times coupled with qualitative indicators such as the impact factor of the journals in which the papers are published and science citation index. However, all these measures often fail to adequately evaluate the effectiveness of research units (RUs) that carry out technological innovation projects. The present study examines this important conceptual question. The study attempts to develop a subjective measure of effectiveness of RU functioning within the laboratories under the Council of Scientific and Industrial Research (CSIR), India, in terms of peer review at four levels – self assessment, head of the RU, external scientific, and external administrative evaluators, using data from 236 RUs working in different laboratories functioning under CSIR. The measurement model component of structural equation modeling methodology using the LISREL 7.16 program has been adopted as the primary methodology for the study. In the measurement model presented, the observed indicators of the different effectiveness measure concepts (R&D effectiveness, user‐oriented effectiveness, administrative effectiveness and recognition) are assumed to possess causal relations with one another. Since subjective measures have not only weak measurement properties, but are also influenced by systematic and random measurement errors, the model's reliability and construct validity – both in terms of convergent validity and discriminate validity – have been ascertained.

Citation

Roy, S., Nagpaul, P.S. and Mohapatra, P.K.J. (2003), "Developing a model to measure the effectiveness of research units", International Journal of Operations & Production Management, Vol. 23 No. 12, pp. 1514-1531. https://doi.org/10.1108/01443570310506740