Open government data and access to public sector information is commonplace, yet little attention has focussed on the essential roles and responsibilities in practice of the information and records management professionals, who enable public authorities to deliver open data to citizens. This paper aims to consider the perspectives of open government and information practitioners in England on the procedural and policy implications of open data across local public authorities.
Using four case studies from different parts of the public sector in England (local government, higher education, National Health Service and hospital trust), the research involved master’s level students in the data collection and analysis, alongside academics, thus enhancing the learning experience of students.
There was little consistency in the location of responsibility for open government data policy, the range of job roles involved or the organisational structures, policy and guidance in place to deliver this function. While this may reflect the organisational differences and professional concerns, it makes it difficult to share best practice. Central government policy encourages public bodies to make their data available for re-use. However, local practice is very variable and perhaps understandably responds more to local organisational strategic and resource priorities. The research found a lack of common metadata standards for open data, different choices about which data to open, problems of data redundancy, inconsistency and data integrity and a wide variety of views on the corporate and public benefits of open data.
The research is limited to England and to non-national public bodies and only draws data from a small number of case studies.
The research contributes to the debate about emerging issues around the complexities of open government data and its public benefits, contributing to the discussions around technology-enabled approaches to citizen engagement and governance. It offers new insights into the interaction between open data and public policy objectives, drawing on the experience of local public sectors in England.
Shepherd, E., Bunn, J., Flinn, A., Lomas, E., Sexton, A., Brimble, S., Chorley, K., Harrison, E., Lowry, J. and Page, J. (2019), "Open government data: critical information management perspectives", Records Management Journal, Vol. 29 No. 1/2, pp. 152-167. https://doi.org/10.1108/RMJ-08-2018-0023Download as .RIS
Emerald Publishing Limited
Copyright © 2019, Emerald Publishing Limited
Open government data and access to public sector information in the UK and across the world is in transformation, and yet little attention has focussed on the essential roles and responsibilities in practice of the information and records management professionals who enable public authorities to deliver open government data to citizens. This article presents the research findings from UCL, one of the European partners in the InterPARES Trust (IPT), a multi-national interdisciplinary research project exploring digital records (www.interparestrust.org/). The article considers the perspectives of open government and information practitioners working in localities in England, set into an international academic research frame. It draws on practice experience across different types of local public authorities of the procedural and policy implications of open government data. It seeks to surface the significant but often-overlooked links between the effective management of information and the delivery of open government data.
Since 2013, UCL has run several linked projects that studied the role of the information and records management discipline in the context of new obligations on public sector bodies towards open government data (i.e. the proactive release of data by public sector organisations for re-use by third parties in the public benefit) and greater access to data for citizens. The research aims to develop a picture of implementation and compliance in the field outside central (national) government by using four case studies from different parts of the public sector in England. The lens through which it studied open government data was the professional discipline of information and records management as these professionals have a critical but under-recognised and, hence, often under-resourced role in the practical operation of open government data. It frames the findings from practice with a literature review drawing on academic research into the intersection between open government data and information management internationally. It concludes with implications for policy and practice.
In the policy context, public authorities are responding to the European General Data Protection Regulation (GDPR), which further constrains the requirements for managing and sharing personal data. Regulatory agencies including the UK Information Commissioner’s Office (ICO, 2017b) propose additional administrative data duties such as, the “duty to document”. In addition, citizens demand greater accountability and transparency in public processes. In this shifting climate, it is critical that open government data policy is better understood and framed holistically. Although related to freedom of information, access to public information and records management, as suggested by Janssen et al. (2012) and Zuiderwijk and Janssen (2014), open government data has distinct characteristics that merit investigation. For example, the relationships between open government data, public records, freedom of information, linked data and the role of information managers and data scientists are complex, overlapping and not fully understood, as set out by Luna-Reyes et al. (2014) and Shepherd (2015, 2017). As government functions are increasingly delivered by commercial and third sector bodies in partnership with the public sector, issues of data provenance, guarantees of data standards and ownership need exploration. The secondary use of open government data is a significant resource to policymakers and for academic researchers, especially if data sets can be linked, as Safarov et al. (2017), and Sexton et al. (2017), explore.
The foundation of the research was a literature review in two phases, on open government data broadly and then focussing more closely on the related privacy issues. The primary data about open government data in practice were collected in four linked case studies over a period of three years (2014-2016) in four different local public sector settings: a local government authority, National Health Service (NHS) England, a hospital trust and a university.
The first stage of the literature review identified key literature about open government data and the record keeper’s role. The second stage of the literature review focussed more specifically on the implications for information management of moves to extend open government data and access to public sector information, including medical records and patient data, in the context of data protection and privacy regulations in the UK and EU. It analysed legislation, policy documents and research literature on the re-use of public sector information, data protection and use of medical records and patient data. The focus was predominately England but also looked outward to legislation and policy in the devolved nations of the UK, as well as internationally.
The research method adopted was a qualitative, instrumental case study, extended to four cases, each with some similarities and some differences, following Stake’s (1994) typology. He identifies three main types of case studies (intrinsic, instrumental and collective), although all have the characteristics of setting out the historical and contextual background, information about the respondents and the nature of the case. Through the collective case study, the researchers aimed to provide insight into an overall research aim, to investigate the impact of open government data requirements in local public authorities in England through the lens of the practice of information and records management. Each case studied how open data were managed within one public authority, explored the reality of practice and where the roles and responsibilities lie. These case studies enable comparative work, analyzing the open government data policy setting and how it interacts with information management roles. The research explored the role of the information and records manager in practice in an open government environment, given that it is a critical actor in the delivery of open government data to citizens (Thurston, 2012).
The geographical scope was England, bounded by the specific legislative and government policy requirements: firstly, a local government authority in the South East of England responsible for providing services to 1.5 million residents, directly or with commercial and third sector bodies, including transportation, schooling, social care, housing, environment, planning, libraries and culture; secondly, NHS England, which leads the NHS, nationally, regionally and locally by setting strategy and priorities and implementing policies, distributing £100bn annually, commissioning contracts for health services and contributing to public debates on health care (given the unique size and role of this body, it was not possible to anonymise this organisation); thirdly, an NHS hospital trust which delivers clinical services to a large urban population, as in-patients and in the community from several hospital sites. This case is published by Chorley (2017). The fourth case study was a multi-faculty university employing about 1,800 staff, undertaking research, higher education teaching of 20,000 students and knowledge exchange, funded by public and private funds. These four organisational settings had significant differences from each other in terms of size (number of employees and budget), organisational priorities and culture, leadership, government policy context for core service functions and client groups. See Table I for brief details.
In each case, the specific research objectives were:
to identify the governing legislative and regulatory open government and open data frameworks;
to investigate existing organisational practices and job roles in delivering open government data and complying with public obligations; and
to develop better understanding of the critical information management issues, policies and guidance, relating to open government data.
The research process was the same for each case. UCL research ethical approval and data protection registration was obtained for the IPT project overall, 2013-2018. Consent required the anonymisation of interview participants and the organisations, except for NHS England, retaining contextual data, including job titles and organisational type.
Research data were collected through a series of 15 individual semi-structured interviews (40-60 min in duration) with information, records, data, governance and other professionals. The interviews were organised on a series of common themes drawn from the literature and designed to explore the research objectives. For example, see the summary of semi-structured interview questions as follows:
Please explain your role and the responsibilities of your department?
Guidance and Regulations:
What OGD obligations and regulations are imposed by central government?
What OGD guidance is available?
What internal and external policies are used for publishing OGD?
Is five-star scheme used?
Has OGD impacted on FOIA requests?
What metadata schema is used for OGD?
What are your responsibilities for OGD?
What challenges are there OGD?
How is authenticity, reliability and accuracy of OGD guaranteed?
Roles and Responsibilities:
Who is responsible for OGD?
Which departments publish OGD? What kind of data?
Record-keeping of open government data:
How do OGD and RM systems interact?
What functional classification is used for OGD?
Is OGD linked to other data? How?
What OGD publication formats?
How to ensure accessibility of OGD?
What OGD Policy training?
Users of open government data:
Which groups use OGD?
How does OGD encourage public participation?
What are future plans for OGD?
Data collection focussed around guidance and regulations, implementation approaches, roles and responsibilities, record-keeping and users of open government data. We did not specifically seek data about individual data sets but rather the policy and operational context in which they might be made open. Interviews were usually audio recorded and then selectively transcribed.
Documentation supplied by interviewees, including internal policies and procedures, organisation charts and manuals, was also analysed. Data coding and analysis was carried out using the pre-coordinate themes which framed the data collection (i.e. role, responsibilities and job context; governing regulations, institutional policies and guidance for information governance, open data, information and records management, freedom of information and data protection; institutional practices for open government data, information and records management, staff awareness and training; future developments) and extended by the findings of the data collection. An initial summary report of data was shared on the IPT project intranet (https://interparestrust.org/).
The literature review framed the research by drawing on academic research into open government data. The literature also helped to establish the themes which were explored in the data collection.
Novais et al. (2013) provided a literature review of open government data studies from 2007 to 2012, in which they indicated the need to broaden the geographic coverage of open government data research and to improve the quality criteria for assessing open government data, including to ensure the trustworthiness of data (Ceolin et al., 2014; Léveillé and Timms, 2015). Novais et al. (2013) suggested that the term open government data began to appear in 2007, following the publication of the original eight open data principles, which stated that open data should be complete, primary not aggregated, timely, accessible, machine-processable, non-discriminatory, non-proprietary and license-free (https://public.resource.org/8_principles.html). These principles framed much of the subsequent discussion, but many technical, managerial, organisational and cultural barriers remain in delivering open data. Kitchin (2014) surveyed the data landscape, including big and open data, and in Chapter 3, he considered the consequences and difficulties of moving from traditional closed data access to more open data access. He provided a clear historical account of the open data movement, discussed further below.
Work on models and methods for evaluating open government data and benchmarking includes Kalampokis et al.’s (2011) stage model of open government data and the Open Data Institute (2015). They built on models proposed to assess the progress of e-government to incorporate open government data more fully by outlining two dimensions (organisational and technological complexity and added value for data consumers) and four stages (aggregation of government data, integration of government data, integration of government data with non-government formal data and integration of government and non-government data with social data). The model should improve benchmarking and the construction of roadmaps for open government data. The information and records management community began to engage with open government data in early 2010s, encouraged by the development of records management guidance in the Open Government Guide (2015), an online resource developed by international civil society organisations to support governments in developing commitments for Open Government Partnership (OGP, www.opengovpartnership.org/) national action plans and by growing professional awareness of the importance of the management of the underlying data to the effective delivery of open government data to citizens. The UK and USA Governments were among the eight founding countries of the OGP in 2011 committed to fostering “a global culture of open government that empowers and delivers for citizens, and advances the ideals of open and participatory 21st century government”, based on access to information, citizen engagement, fiscal transparency and income and asset disclosure (Herrero, 2015). The influence of OGP action plans on practice and on research was considerable. For example, the UK Government (2016) national action plan for 2016-2018 developed in collaboration with civil society organisations included commitments to engage with data users about their needs and to gain their views on priority areas for the development of the open data agenda and to develop <gov.uk>, the website for government publications, to make it more open and accountable.
Privacy and data protection are often seen as problematic in an open data context, inhibiting the public good in sharing data. While sharing data may lie at the heart of the open government data agenda, from a privacy perspective, the right to limit the sharing of identifiable personal information is cast as a fundamental human right. A number of overlapping legal measures exist to protect privacy, including privacy rights, which guarantee freedom from interference; data protection, which controls the processing of personal data; and duties of confidentiality, which protect against unauthorized or unreasonable breaches of confidence (Nuffield Council on Bioethics, 2015). There is recognition that personal privacy is not always in the public interest if it impedes other fundamental human rights and interests. For example, there is a public good in the use of medical records and patient data to support advances in medical, health and scientific research. However, there is also a public good in respecting and protecting privacy, maintaining confidentiality and limiting the use of medical records and identifiable patient data. Balancing these public goods (the public good in enabling research and the public good in protecting data) is increasingly challenging given the rapidly evolving mechanisms open to researchers and others to reuse and link data (Caldicott, 2013).
Research results and discussion
This section presents and discusses the main practice findings, illustrated by quotations from the interviews, referenced by case study and interview (e.g. 1:1) as in Table I.
Who is responsible for open government data in public authorities?
Our first question sought to understand where responsibility for open government data sat organisationally and the professional groups and units that led on the issue. We were particularly looking for evidence that records and information managers were explicitly involved and their information skills acknowledged as relevant to open data. As might be expected, given that the four case study organisations had very different core mandates and significant differences in size and structure, there was little consistency in the location of responsibility for open government data policy, the range of job roles involved or the organisational structures in place to deliver this function. That each case study took such radically different approaches to delivering the same open data function was perhaps unexpected. As this was a fairly new area of work, the local authority had established an open data working group in its business intelligence team to develop policy, including governance and law, information governance, information access and records management. An operational team for information transparency had responsibility for data protection, freedom of information, data sharing, data security and environmental information but open data was not yet part of its functions. The authority sought to “make the data we are creating as an authority, as part of our daily business, available for more general use” (interview 1:3) and equated open government data with “transparency […] being able to give data to the public when they ask for it and being open about it […] being open about what we spend our money on” (interview 1:1). This authority seemed to have well-established teams to deliver existing information functions, and the cross-functional working group was making progress in defining and identifying the policy aspects of open data as a preliminary step towards making the new function part of an operational team.
In the NHS hospital trust, responsibility for open data was not yet established and no job titles referenced that role directly. As in many health settings, the overarching function was information governance, following the Caldicott (2013) principles for handling patient information across the National Health Service (2016). A traditional line management structure for information governance sat within the directorate of corporate affairs. Information governance and internal corporate records functions were fairly traditionally conceived as having a primarily corporate focus, rather than external obligations for openness and transparency, although with responsibility for freedom of information requests. One interviewee anticipated that when the hospital trust formally adopted an open data function, there would be a separation between policy and operations:
I would expect to see it coming to Corporate Affairs and the stuff that’s around releasing data, IG and FOI would probably come our way, but some of the stuff probably more about policies, about how the Trust runs itself would probably sit with the Trust Secretary. (3:1)
But in practical terms, existing staff would “just do it as we go along” (3:1). Another interviewee thought that open government and open data were not yet high priorities for the hospital trust (3:2). This position is somewhat at odds with the picture given by NHS England, guided by its obligations under the Health and Social Care Act 2012. At the time of the case study in 2015, NHS England had a policy unit to develop policies around information standards, open data and patient care data which sought to be a center of expertise, acting as a “think tank for NHS England” and learning actively “from the best across the UK and internationally” (2:3). NHS Digital collects, manages and publishes health and care data under the direction of NHS England and the Department of Health (2:4). The case study hospital trust noted that it reported large amounts of data to NHS England; much of the hospital’s open data is effectively published by a third party. However, the responsibility for open government data is not always completely clear in large and complex public services. Sometimes it is not clear how variants of data sets relate to each other, or who should publish which ones.
In the university case study, there was evidence of a very active approach to business intelligence, data assurance and responsive provision of personal data and corporate information, partly driven by the requirement to provide data to the Higher Education Statistics Authority (HESA) as a condition of grant funding. One interviewee from the strategy and planning unit admitted, however, “I’m not that familiar with the open data stuff”, which she saw as an external issue about access to data for academic research rather than a corporate issue (4:1). The university records manager’s role had expanded rapidly from managing corporate records and information to include information assets, audit and compliance and information governance. To deal with this changing environment:
We set up an Information Governance Group, […] more involved in information security and information assurance. What we’ve had to do is to bring together representation from across the HEI including security, IT, academics, our Research Information Manager who handles research data management (4:2).
The case studies suggested that proactive individuals could make progress in encouraging a public authority to develop policy and practice in open government data by bringing a group of interested officers together in a focussed, short-term way, such as on a policy working group, with the longer-term aim of embedding the new function in an existing unit. The corporate environment could help or hinder such initiatives.
No one professional group routinely took on open government data functions: it depended more on the interests of individual staff taking advantage of corporate opportunities rather than yet forming part of formal role descriptions. Job titles included directors of governance and law, information governance specialists, information access officers, data sharing specialists, corporate communications officers, business intelligence officers, heads of data policy, freedom of information and data protection officers and corporate information and records managers. Post holders had backgrounds in law, ICT, information and cybersecurity, informatics, education, social sciences and media and communications, as well as records management. The research reflected a shift in some organisations to information governance as the overarching corporate function for open government data. Information and records managers often have the skills needed to manage and deliver open data, but in practice, their employing organisations did not always exploit these skills for public benefit.
What policy and guidance on open government data exists in public authorities?
Part of the explanation for the variation of approaches to the management of open government data discussed in the previous section might be found in the varied policy and guidance context. All of our case studies were subject to Freedom of Information Act (FOIA) 2000, and the Data Protection Act (DPA) 1998, replaced by the GDPR and DPA in 2018. Although there is no open government data legislation, there is central government policy directed at public bodies encouraging them to make their data available for re-use, including the Open Data White Paper and Open Public Services White Paper (UK Government, 2011, 2012). The OGP promotes the open government agenda through action plans: the UK issued its third action plan including open data commitments in 2016. The requirement to publish certain data to fulfill the government’s transparency agenda at <https://data.gov.uk/> facilitates central agencies in publishing data.
However, considerable local and sectoral variation in policy can be found in our four case studies. Some sectors are much more highly regulated, in particular in two of our case studies in the health sector. NHS England, including NHS Digital, has a responsibility for improving the quality of health and care data and publishes a number of national guidance and policy documents. NHS Records Management Code of Practice for Health and Social Care forms part of NHS Digital’s guidance and policies on information governance, including the Information Governance Toolkit (National Health Service, 2006, 2016). Several of our interviewees commented on the lack of connection between records management and information governance in the NHS, even though these policies ought to support each other. Records management was described as “not visible or vocal enough to be involved in open government data” (2:3). Information governance, by contrast, was well developed and although, the IG Toolkit was “not directly mandated by legislation but it is […] done without question” (2:1). “The idea of patient confidentiality is one that has been at the heart of doctor-patient relationship since time immemorial” (2:4) and data confidentiality is integral to information governance in the NHS. In the NHS, “the more open you get, the more nervous people get” (2:3) and “already there are large numbers of people who do not want their identifiable data used for anything other than their direct care” (2:4), not least in the wake of failed data sharing initiatives such as care data (Presser et al., 2015; McLeod and Childs, 2018), and concerns over data sharing with commercial companies such as Google DeepMind (ICO, 2017a). Much of the data held in the NHS is patient data and identifiable personal data cannot ever be open, such data will only be published if “aggregated and anonymised” (2:4).
The national open data and transparency agendas of NHS England are not always reflected at a local level. Barriers to proactive publication of data, as one interviewee remarked, “the culture is more around publishing as little as possible”, “partly the culture, partly a lack of awareness and also partly […] the technical capability” (3:2). For policy areas not governed by legislation, such as open government data, there was a considerable delay between policy-making at the national level and the filtering down of policy and associated practice to the local level.
By contrast, the local authority case study approach to open government data was longer established, although in practice only a few specific data sets were routinely made open. The Local Government Transparency Code (2015), together with guidance from the Local Government Association, provided a framework, which was overseen locally by elected councillors. One interviewee (1:2) commented on the tension between, on the one hand, central government policy to publish as much data as possible and rules requiring publication of certain local financial data with, on the other hand, allowing the authority to decide which data was of interest to local public audiences and prioritise their resources to select and process that data for publication.
Problems with delivering open government data in practice were also identified, especially an assumption that data and information are held in well-established electronic systems. Local authorities were expected to meet at least level three of the “5 Star” scheme when publishing data (a simple indicator of the openness of a data set, such as its format and metadata, see <http://5stardata.info/>). In our case study, the local authority records management systems were still predominately based around paper-based records, while emails, HR and financial data were held in separate digital systems. Many different formats were used for the creation and business use of data sets, yet most of the data required re-formatting, additional metadata and re-presentation to make it openly accessible. For the authority to meet the requirements for opening up data and assure data reliability and quality of the data drawn from records systems, major changes in the way data and records were managed and accessed might be needed.
While open government data remains unlegislated, it is largely at the discretion of individual organisations whether they proactively publish data sets, what they publish and indeed for what purpose. The university case study interviewees were mindful of its statutory obligations to provide data to HESA and to comply with data protection and freedom of information but to go beyond that to publish large swathes of data, which might have reputational risks and would cost scarce resources, was not a current priority. As opening up data consumes resources, the corporate, individual and public benefits needed to be balanced.
If more data are to be made open, then choices have to be made by the creating organisation about which data to open. At present, these choices are neither transparent, nor are the reasons for them explained in public. Open data is generally not contextualised and users just see individual datasets: a proper understanding of open data might also require knowledge of which data was not opened.
Open government data as a policy-driven activity is subject to localised pressures. Resources, technical capabilities and strategic and local policy priorities tended to drive the choices about which data to open. Secondary data use is therefore highly dependent on local pathways, and citizens and researchers are faced with highly variable open datasets from place to place, making re-use problematic and unpredictable. The case studies suggested that consistent policies are lacking between different sectors, different organisations and indeed between different teams and units within one organisation, leading to inconsistency and a lack of transparency. While a single common mandated approach would not necessarily serve the needs of diverse organisations and their clients, such a wide range and the lack of even a common set of principles leads to confusion about open data priorities. Under UK FOIA (2000), The National Archives (2009) published a Code of Practice on Records Management providing core common requirements to support freedom of information, which individual organisations could adopt in a way suited to their own structure and mandate: no similar guidance exists for best practice in open government data provision.
Publishing data is not enough: what else is required?
As Dawes (2010) stated, publishing data is not enough, it needs to exist within a technical infrastructure and information context that renders it meaningful and usable to the public. The interviewees reported that as data were not created with public access in mind, a considerable amount of work was needed to ensure that data was re-usable, including perhaps requiring staff to change their working practices to use open data formats for business as usual (1:2). They were also concerned about problems of data redundancy, inconsistency and data integrity. One records manager (1:3) explained her responsibility for guaranteeing protective markings on records containing confidential or personal data and for ensuring the protection of personal data and privacy of individuals before data is made open. The Open Data policy in this case study gave heads of service the responsibility to ensure that data published is as accurate and usable as possible but the practicalities of implementing data checking processes systematically across the authority to ensure this were considerable, according to our interviewees (1:2, 1:4). Another interviewee (3:1) said that there was a need to rebut the misconception that publishing data would be easy and straightforward and commented “I’m sure the politicians do just think it’s the click of a button, but it’s not.”
Complete and accurate metadata sets are needed to establish the connection of the dataset to its creating context and to assist with interpretation but available metadata may be limited to that captured automatically, perhaps with some additional contextual information provided by data creators, according to one interviewee (3:2). Record creators and information governance staff were concerned with the accuracy of metadata, but the presentation and ease of use of the data and explanation of the metadata may be more important to the end user (1:2). A lack of common metadata standards for open government data and lack of standardization of terminology hindered the publication and use of open government data, according to one interviewee (1:3).
Denis and Goëta (2014), in their ethnography of open government data projects in French administrations, noted three significant operations (exploration, extraction and ramification) needed to prepare open data for release. In our study, the quality of the data and the amount of work needed to make it open affected access: cleaner data, for example without any personal data or in simpler formats which are easier to reprocess, were more likely to be made open, simply because it was easier to deal with. “Trying to work out what can be published”, as well as the technical and resource problems, was complex. Even once a decision was made to release data, “when we say it’s going to be open”, is that “just going to be for those researchers who are asking for it” or something “that we’re going to stick on our website that anyone can come onto and look at” (3:1)?
What are the information management challenges of the open government data environment?
The final issue which we sought to examine in our case studies, was to draw out some of the information management challenges of open government data and the proactive release of government data, to consider how information managers might help their parent bodies to meet them. The case study organisations faced many information and records management challenges, including the management of risk, privacy and data security. Protecting personal data, especially sensitive personal data, from inadvertent disclosure is important and risky if not done well. Cleaning up data before proactive release is time-consuming but essential to safeguarding data quality and privacy.
If data is aggregated, linked, anonymised and released openly, data inaccuracies may be impossible to identify and correct and will affect the understanding of the data. Inaccuracies in the data can occur at many stages in the record creating process, through inadvertent or deliberate mistakes in recording, through inaccurate transcription and faulty reprocessing. This risk can be mitigated by reliable internal procedures, including staff training and well-established information management policies and procedures that help to ensure proper data management from creation to public access. The authors would argue that accurate data is essential for internal corporate uses of data, as much as for open data release.
Technical capabilities of staff and of the institutional infrastructures were identified as issues by the interviewees. For instance, the large quantities of data and large size of data sets to be delivered through websites posed technical challenges, leading to websites freezing or collapsing and disruption to other digital services. Many local public sector organisations do not have the resources to establish sufficiently robust and scalable web services to deliver open government data. So far, shared open data portals outside national government have not emerged in England. Universities could develop shared open data sources for administrative and research data on the shared national models seen in the past to develop computing infrastructure (JANET network) and some research data (e.g. UK Data Archive and Administrative Data Research Network) but at present, most open data development is localised.
Metadata standards for open government data are currently lacking in principle and in practice. Metadata provision is not resource-neutral as most data is coded and described for its original business purpose rather than for re-use. At present the processing of data prior to opening is labour-intensive and, until technology-assisted routine processes are developed, is likely to remain so. The push to open large quantities of data has to be balanced with the need to ensure data and metadata quality and with effective use of scarce resources.
Many of the skills needed to deal with open government data are similar to those needed to manage access to information and many of the principles for managing open data mirror existing information and records management principles and standardised processes, such as provenance, functional classification and creation of standard metadata and description. Information managers are, therefore, well placed to play a leading role in open government data. However, to fulfill these roles and responsibilities effectively and support the development of re-usable and reliable open government data, they need to develop greater awareness of open data, better technical skills in digital data and systems, improved expertise in information security, in routines for anonymisation for personal data, in data analytics, in digital curation and in semantic web technologies. They need to deploy their skills in the assessment and management of risk, environmental and privacy impact assessment. Ensuring that information and records management skills are fit for purpose and used for the public good in an open government data setting is a critical challenge.
This study sought to investigate the impact of the open government data environment in local public authorities in England through the lens of the professional practice of information and records management. In particular, it aimed to identify key legislative and regulatory open government data frameworks and principles, through a study of the literature and to investigate existing organizational practices and job roles in delivering open government data through four case studies. The research results showed that in practice in the public authorities in the study, there was little consistency in the location of responsibility for open government data policy organisationally, nor agreement about which professional group should lead the activity, nor common approaches for the practical delivery of open data. This finding is perhaps not surprising given the different governance models adopted by the case study organisations and their different mandates. As a fairly new area of work, many different professions were involved: governance, law, information governance and records management, ICT, cybersecurity and data management. This highlighted some of the managerial and organisational barriers, which public authorities currently face. Although, a single common approach would not be appropriate for such a variety of public organisations, the lack of consistency inhibits effective sharing of best practice. If public authorities shared good practice and benchmarked across different domains and between different professional groups, then the success stories could better inform authorities whose open government data policy and practice was not yet fully developed.
A number of other barriers to open government data were identified. The technical infrastructure needed to deliver open government data, website capabilities, storage of data, metadata and the long-term provision of access still need development. This was a resource issue, both the costs of implementing improved technical systems but also in terms of ensuring the necessary staff competencies, re-training and re-skilling staff on an ongoing basis. Other technical issues related to the internal systems used to create data and records, which are not yet attuned to open data as a business as usual issue. As a result, data needs extensive re-processing before opening.
Cultural barriers still exist in organisations. Authorities did not agree about the benefits of open government data, some saw it as an exercise in transparency and accountability, others a means of ensuring internal data compliance and governance. Few seemed yet to have considered the external needs of open data users and this is an area in which further research would be beneficial to guide policy. If public authorities do not see the value to their business of making government data open, then they will not allocate the necessary resources to ensuring that it happens.
This research suggests that the skills of information and records management professionals could be better utilised to help public authorities to meet open government data challenges. The management of risk, balancing privacy and the public good in open data, against the reputational and individual risks of releasing data inappropriately, is a critical skill. Ensuring data accuracy by enabling datasets to be traceable to their original reliable data source, and ensuring the data governance systems which surround them, is also critical. Information and records managers are well placed to play a leading role in open government data but they need to explain their unique contribution more clearly in a crowded and confusing field.
This work makes three key recommendations:
Open government data policy and structures should be supported by best practice case studies and guidance. As public bodies implement open government data functions, they would benefit from the shared experience and models from best practice.
Designing information systems with the potential to support open government data could be improved if public bodies co-developed shared templates for systems requirements, including open government data metadata standards and standards for the creation of current data. At present, developments seem to be highly localised.
The job roles and organisational structures for delivering open government data initiatives should be clearly articulated and take account of the full range of information expertise including, law, ICT, cybersecurity, information governance and information and records management.
The development of a code of practice for the management of open government data across the public sector would enable these recommendations to be taken forward.
Overview of case studies
|Four IPT case studies||Date of data collection||Research assistant||No. of interviews/job roles of interviewees|
|1. Local government||June-July 2014||Jessica Page||4 interviewees:
Information access officer
Business intelligence officer
Information governance officer
|2. NHS England||June-July 2015||Emma Harrison||4 interviewees:
Senior advisor, The National Archives
Senior data sharing specialist
Head of data policy
Senior information governance advisor
|3. Hospital trust||June-July 2016||Katherine Chorley||3 interviewees:
Information governance manager
Corporate records manager
Assistant records manager
|4. Higher education institution||June-July 2016||Sara Brimble||4 interviewees:
Freedom of information and data protection officer
Research information officer
Caldicott, F. (2013), “Caldicott review: information governance in the health and care system. Information: to share or not to share? The information governance review”, available at: www.gov.uk/government/publications/the-information-governance-review (accessed 27 January 2017).
Ceolin, D., Moreau, L., O’Hara, K., Fokkink, W., V., Hage, W.R., Maccatrozzo, V., Sackley, A., Schreiber, G. and Shadbolt, N. (2014), “Two procedures for analyzing the reliability of open government data”, in Laurent, A., Strauss, O., Bouchon-Meunier, B. and Yager, R.R. (Eds), Information Processing and Management of Uncertainty in Knowledge-Based Systems, Springer International Publishing, Cham, Vol. 442, pp. 15-24, available at: http://link.springer.com/10.1007/978-3-319-08795-5_3 (accessed 5 January 2017).
Chorley, K. (2017), “The challenges presented to records management by open government data in the public sector in England: a case study”, Records Management Journal, Vol. 27 No. 2, pp. 149-158.
Dawes, S.S. (2010), “Stewardship and usefulness: policy principles for information-based transparency”, Government Information Quarterly, Vol. 27 No. 4, pp. 377-383.
Denis, J. and Goëta, S. (2014), “Exploration, extraction and ‘rawification’: the shaping of transparency in the back rooms of open data”, After The Reveal. Open Questions on Closed Systems – Neil Postman Graduate Conference, New York, NY.
Herrero, A. (2015), Access to Information Commitments in OGP Action Plans: A Report on the Progress of Reforms Worldwide, World Bank Governance Global Practice and Red de Transparencia y Acceso a la Información Pública (RTA), available at: http://redrta.org/
Information Commissioner’s Office (ICO) (2017a), “Royal Free London NHS Foundation Trust Ruling”, available at: https://ico.org.uk/action-weve-taken/enforcement/royal-free-london-nhs-foundation-trust/ (accessed 25 August 2017).
Information Commissioner’s Office (ICO) (2017b), “The extent of our care: archives, memory and information rights”, available at: https://ico.org.uk/about-the-ico/news-and-events/news-and-blogs/2017/01/the-extent-of-our-care-archives-memory-and-information-rights/ (accessed 25 August 2017).
Janssen, M., Charalabidis, Y. and Zuiderwijk, A. (2012), “Benefits, adoption barriers and myths of open data and open government”, Information Systems Management, Vol. 29 No. 4, pp. 258-268.
Kalampokis, E., Tambouris, E. and Tarabanis, K. (2011), “Open government data: a stage model”, in Janssen, M., Scholl, H.J., Wimmer, M.A. and Tan, Y. (Eds), International Conference on Electronic Government, pp. 235-246, available at: http://link.springer.com/10.1007/978-3-642-22878-0_20
Kitchin, R. (2014), The Data Revolution: Big Data, open Data, data Infrastructures and their Consequences, SAGE Publications, London, doi: 10.4135/9781473909472.
Léveillé, V. and Timms, K. (2015), “Through a records management lens: creating a framework for trust in open government and open government information”, Les Objectifs Visés Par Les Systèmes de Gestion Documentaires: La Mise En Place D’un Cadre de Confiance et de La Transparence de L’information Dans Un Gouvernement Ouvert, Vol. 39 No. 2, pp. 154-190.
Luna-Reyes, L.F., Bertot, J.C. and Mellouli, S. (2014), “Open government, open data and digital government”, Government Information Quarterly, Vol. 31 No. 1, pp. 4-5.
McLeod, J. and Childs, S. (2018), “Public trust in online records: the case of the UK care.data programme”, in Anderson, A., Becker, I.C. and Duranti, L. (Eds), Presentations at the 21st Archival Sciences Colloquium of the Marburg Archives School on Born Digital in the Cloud: Challenges and Solutions, pp. 43-64.
National Health Service (2006), “NHS records management code”, available at: www.gov.uk/government/uploads/system/uploads/attachment_data/file/200138/Records_Management_-_NHS_Code_of_Practice_Part_1.pdf (accessed 4 July 2015).
National Health Service (2016), “Information governance toolkit”, available at: www.igt.hscic.gov.uk/ (accessed 27 January 2017).
Novais, T., De Albuquerque, J.P. and Craveiro, G.S. (2013), “An Account of Research on Open Government Data (2007-2012): a Systematic Literature Review”, in Electronic Government and Electronic Participation: Joint Proceedings of Ongoing Research of IFIP EGOV and IFIP 2013, P-221, pp. 76-83.
Nuffield Council on Bioethics (2015), “The collection, linking and use of data in biomedical research and health care: ethical issues”, available at: https://nuffieldbioethics.org/wp-content/uploads/Biological_and_health_data_web.pdf (accessed 25 July 2017).
Open Data Institute (2015), “Open data maturity model”, available at: https://theodi.org/guides (accessed 26 September 2017).
Open Government Guide (2015), “Records management”, available at: www.opengovguide.com/topics/records-management/ (accessed 21 August 2016).
Presser, L. Hruskova, M. Rowbottom, H. and Kancir, J. (2015), “Care.data and access to UK health records: patient privacy and public trust”, Technology Science, available at: http://techscience.org/a/2015081103/ (accessed 6 October 2015).
Safarov, I., Meijer, A. and Grimmelikhuijsen, S. (2017), “Utilization of open government data: a systematic literature review of types, conditions, effects and users”, Information Polity, Vol. 22 No. 1, pp. 1-24.
Sexton, A., Shepherd, E., Duke-Williams, O. and Eveleigh, A. (2017), “A balance of trust in the use of government administrative data”, Archival Science, Vol. 17 No. 4, pp. 305-330, available at: http://rdcu.be/wDrx
Shepherd, E. (2015), “Freedom of information, Right to access information, Open data: who is at the table?”, The Roundtable: The Commonwealth Journal of International Affairs, Vol. 104 No. 6, pp. 715-726.
Shepherd, E. (2017), “Right to information”, in MacNeil, H. and Eastwood, T. (Eds), Currents of Archival Thinking, 2nd ed., Libraries Unlimited, ABC-Clio, Santa Barbara, CA, pp. 247-270.
Stake, R.E. (1994), “Case study research”, in Denzin, N.K. and Lincoln, Y.S. (Eds), Handbook of Qualitative Research, Sage, Thousand Oaks, CA; London.
Thurston, A. (2012), “Trustworthy records and open data”, The Journal of Community Informatics, Vol. 8 No. 2, available at: http://ci-journal.net/index.php/ciej/article/view/951 (accessed 14 August 2018).
UK government (2011), “Open public services white paper, Cm 8145”, available at: www.gov.uk/government/uploads/system/uploads/attachment_data/file/255288/OpenPublicServices-WhitePaper.pdf (accessed 3 February 2017).
UK government (2012), “Open data white paper: unleashing the potential, Cm 8353”, available at: www.gov.uk/government/uploads/system/uploads/attachment_data/file/78946/CM8353_acc.pdf (accessed 3 February 2017).
UK government (2016), “Open government national action plan 2016-18”, available at: www.opengovpartnership.org/documents/united-kingdom-third-national-action-plan-2016-18 (accessed 26 August 2017).
Zuiderwijk, A. and Janssen, M. (2014), “Open data policies, their implementation and impact: a framework for comparison”, Government Information Quarterly, Vol. 31 No. 1, pp. 17-29.
Information Governance Alliance (2016), Records Management Code of Practice for Health and Social Care 2016, Information Governance Alliance, London.
UK government The National Archives and Ministry of Justice (2009), Lord Chancellor’s Code of Practice on the Management of Records Issued Under Section 46 of the Freedom of Information Act 2000.
UK government Department for Communities and Local Government (2015), Local Government Transparency Code 2015, available at: www.gov.uk/government/publications/local-government-transparency-code-2015 (accessed 3 February 2017).
UK government (2017a), “Data Protection Act 1998 chapter 29”, available at: www.legislation.gov.uk/ukpga/1998/29/contents (accessed 2 February 2017).
UK government (2017b), “Freedom of Information Act 2000 chapter 36”, available at: www.legislation.gov.uk/ukpga/2000/36/contents (accessed 2 February 2017).
UK government (2017c), “Health and Social Care Act 2012, chapter 7”, available at: www.legislation.gov.uk/ukpga/2012/7/contents/enacted (accessed 2 February 2017).
Funding: This work was supported by the Canadian Social Sciences and Humanities Research Council (SSHRC), Trust in Digital Records in an Increasingly Networked Society (InterPARES Trust), grant number 895-2013-1004. UCL is one of the European partners in the InterPARES Trust (IPT), a multi-national, interdisciplinary research project (www.interparestrust.org/).