Designing public web information systems with quality in mind: Public reporting of hospital performance data

Abstract

Purpose

This paper aims to describe evolution of a new public information web site, through evaluation‐refinement prototyping cycles.

Design/methodology/approach

An expanding range of participants is being engaged in formal evaluations as the site design evolves. The Flesch‐Kincaid Grade Level Score is applied to assess ease of reading in the wording used; the National Quality Forum guideline statements are applied to determine whether the prototyping design process is meeting performance expectations; and then Nielsen's heuristics are applied to evaluate ease of use of the latest prototype.

Findings

The page wordings started at a high reading grade level to be technically correct, with a strategy to progressively reduce levels without losing meaning. Reading level was reduced to an average of two and as much as six grades through editing between the third and fourth‐generation prototypes. None of the National Quality Forum principles were found missing from the development process. The prototype web site was ranked at the middle compared to official public web sites of seven other States' healthcare‐associated infection programs, some of which had been open to the public for more than a year. Many of the heuristic violations that weighed against the prototype were described as being minor and easily fixed. Collaboration between a State health department and a university to advance this evaluation‐refinement process was valuable to both parties, enhancing the ability to produce a new public information web site that is more likely to meet the needs of its intended audience.

Practical implications

In response to increasing expectations of transparency and accountability, a growing number of public web sites are displaying hospital performance data. Washington State's mandatory public reporting of healthcare‐associated infection rates is a recent example of this trend. The Department of Health is required by law to launch a public information web site by December 2009. The research was based on an evidence‐based approach to understand and meet the information needs of the public.

Originality/value

Although few studies have evaluated the usage and impact of hospital comparison web sites, these studies uniformly show relatively low usage and disappointing impact. Using the research literature, issues thought to account for poor usage and low impact, and developed design principles that address this poor past performance were identified. Throughout 2008 and 2009, successive prototypes were developed for the web site structure guided by those principles and refined each generation of prototype through focus group evaluations. This paper explains the approach, and summarizes results from the evaluations, leading to improvements before the final design first opens to the general public.

Citation

Birnbaum, D., Jeanne Cummings, M., Guyton, K.M., Schlotter, J.W. and Kushniruk, A. (2010), "Designing public web information systems with quality in mind: Public reporting of hospital performance data", Clinical Governance: An International Journal, Vol. 15 No. 4, pp. 272-278. https://doi.org/10.1108/14777271011084037