Guest Editorial

Citation

Thurston, A. (2014), "Guest Editorial", Records Management Journal, Vol. 24 No. 2. https://doi.org/10.1108/RMJ-06-2014-0027

Guest Editorial

Article Type: Guest Editorial From: Records Management Journal, Volume 24, Issue 2

The world is experiencing a data revolution. The idea is that data can be used to drive and support development in both the public and the private sectors. This trend, I believe, has major implications for the records and archives profession. The Records Management Journal is taking a lead in initiating debate on this issue, and I congratulate the Editor, Julie McLeod, for her astute recognition that the records management community has an important role to play in the move toward Open Data and Big Data Analytics. If records managers can bring their skills and experience to the challenges of managing and protecting data integrity through time, they will make an invaluable contribution to the lives of citizens everywhere and the health of the records management profession. I am grateful to the authors who have contributed to this Special Edition of the RMJ for moving this process forward.

At least two key interrelated trends are driving the data movement: the effort to reverse the culture of secrecy that has prevented people from knowing what their governments are doing and spending, and the massive explosion of data now being generated from new sources and channels (mobile phones, social media and Internet traffic). Data, rather than records now, are viewed widely as the basis for planning and measuring goals for economic development, for empowering citizens to hold their governments accountable and for minimising financial or environmental disasters. There is a widespread assumption that the data needed to achieve this are available but just need to be used better. I believe that using data effectively also means managing them effectively.

Open Data and Big Data are distinct concepts. Open Data are a form of proactive disclosure or volunteered data; Big Data generally refer to “observed data” drawn from the massive deluge of data being generated worldwide. Expectations for using both sorts of data are extremely high, but at present, they take almost no account of records management experience. Looking at these issues from a global development perspective can give us a sense of the scale and significance of what is happening.

Over the past few years, there has been rapidly growing interest in the potential benefits of opening publically funded data sets to highlight how aid and tax funds are spent. The hope is that transparency will raise public awareness of how public funds are used, help to hold public officials accountable, drive out waste and bureaucracy and stimulate economic growth. Early work on Open Data has focused on releasing data sets, without a methodology for ensuring data accuracy and traceability to reliable information sources. In many cases, in many countries, the records needed to provide evidence for Open Data initiatives do not exist, cannot be accessed or are unreliable. In the digital environment, the survival and integrity of records depends on a quickly changing array of hardware and software; if they are not managed professionally, their value as legal evidence and an authoritative source for Open Data initiatives can easily be compromised.

The fact is that poorly kept records result in inaccurate, incomplete or unverifiable data, which in turn can lead to misuse of information, cover-up of fraud, skewed findings and statistics, misguided policy recommendations and misplaced funding. Data sets may appear to be robust and comprehensive, but they may not be traceable to an evidentiary source, usually a record. At best, organisations using these data sets can waste resources compiling, analysing and publishing inaccurate or incomplete data. At worst, citizens and stakeholders can be misled and governments and international donors can make decisions based on unreliable data.

As the Open Data movement develops and expectations evolve, it will be increasingly important to do more than simply open data sets to the public. Questions need to be asked about the quality of the data available and its relationship to the records that should support it. While data can be a valuable indicator, if they are to be trusted they need to be substantiated. As Bill Dorotinsky, Manager, Poverty Reduction and Economic Management Network at the World Bank, has noted:

Poor economic data leads to bad economic forecasting. People assume that good economic data is there, but if it is not, work is flawed or not possible. Data should come from records – the veracity of the data depends upon the record. The quality of the records management system makes you trust or doubt data.

Are the records from which the data are derived reliable? Are they complete? Are they authentic? How were they generated, by whom and in what conditions? Is there sufficient contextual information to enable them to be understood? Are they being captured and held securely to allow comparisons over time?

Big Data raises another set of issues. For years, large businesses have been using the data they collect to gain insights on their users and become more efficient. Today, evolving technology capabilities have led to a massive explosion of real-time information at escalating speed and frequency and from a widening range of sources. The private sector is developing new innovations, technologies, methodologies and tools for transforming these massive quantities of raw, imperfect, complex and often unstructured data into useable information to enable validation and rapid insights, for instance for improving productivity and marketing. Increasingly, companies have to master these new opportunities to survive.

Development planners are aware that they can do much more with data, and the international development community has its eye on the benefits of Big Data for global development[#fn1]. They see a real opportunity to use the powerful new tools to reveal and tackle issues for public good, notably tackling hunger, poverty and disease and averting the effects of financial and climatologic shocks by responding to anomalies more quickly. While they tend to recognise that real-time information will not replace the traditional evidence base that governments use for decision-making, many believe that Big Data, understood correctly, can change outcomes like nothing else can. Governments across the world are starting to realise the power of Big Data.

However, few organisations find it easy to manage their existing data effectively, and Big Data add much greater complexity to the issue. The new data streams need to be managed to protect reliability, accuracy and representativeness as a basis for meaningful analysis and interpretation. There is a range of issues to be addressed. For instance, how should the data be stored and for how long? How should the associated metadata be managed to provide context? What are the issues for security, privacy and regulatory compliance? Records management standards, systems and controls have much to contribute.

There are great hopes for using data beneficially, but not enough thought has been given yet to how they are to be managed and preserved so that they will survive and retain integrity through time. I believe that records and data are inseparable aspects of the new information environment and that the principles of records management are essential to a successful data revolution in terms of supporting greater accuracy, higher precision and better use of information. I am looking forward to much more research in this area and to the development of new means of measuring national and organisational capacity for capturing, protecting and preserving digital records and data and the relationship between them.

Anne Thurston, Guest Editor

Note

See “Big Data for Development: Challenges and Opportunities”, UN Global Pulse Initiative, May 2012.