Data-centric public services as potential source of policy knowledge. Can “design for policy” help?

Abstract

Purpose

The purpose of this paper is three-fold. Firstly, through selected case studies, to provide an overview of how non-traditional data from digital public services were used as a source of knowledge for policymaking. Secondly, to argue for a design for policy approach to support the successful integration of non-traditional data into policymaking practice, thus supporting data-driven innovation for policymaking. Thirdly, to encourage a vision of the relation between data-driven innovation and public policy that considers policymaking outside the authoritative instrumental logic perspective.

Design/methodology/approach

A qualitative small-N case study analysis based on desk research data was developed to provide an overview of how data-centric public services could become a source of knowledge for policymaking. The analysis was based on an original theoretical-conceptual framework that merges the policy cycle model and the policy capacity framework.

Findings

This paper identifies three potential areas of contribution of a design for policy approach in a scenario of data-driven innovation for policymaking practice: the development of sensemaking and prefiguring activities to shape a shared rationale behind intra-/inter-organisational data sharing and data collaboratives; the realisation of collaborative experimentations for enhancing the systemic policy analytical capacity of a governing body, e.g. by integrating non-traditional data into new and trusted indicators for policy evaluation; and service design as approach for data-centric public services that connects policy decisions to the socio-technical context in which data are collected.

Research limitations/implications

The small-N sample (four cases) selected is not representative of a broader population but isolates exemplary initiatives. Moreover, the analysis was based on secondary sources, limiting the assessment quality of the real use of non-traditional data for policymaking. This level of empirical understanding is considered sufficient for an explorative analysis that supports the original perspective proposed here. Future research will need to collect primary data about the potential and dynamics of how data from data-centric public services can inform policymaking and substantiate the proposed areas of a design for policy contribution with practical experimentations and cases.

Originality/value

This paper proposes a convergence, yet largely underexplored, between the two emerging perspectives on innovation in policymaking: data for policy and design for policy. This convergence helps to address the designing of data-driven innovations for policymaking, while considering pragmatic indications of socially acceptable practices in this space for practitioners.

Citation

Leoni, F., Carraro, M., McAuliffe, E. and Maffei, S. (2023), "Data-centric public services as potential source of policy knowledge. Can “design for policy” help?", Transforming Government: People, Process and Policy, Vol. 17 No. 3, pp. 399-411. https://doi.org/10.1108/TG-06-2022-0088

1. Introduction: the importance of non-traditional data for policymaking

National governments worldwide are committed to making digital data a central element of their innovation agendas (OECD, 2019, pp. 145–155). As data has been linked to the possibility of innovating, the concept of data-driven innovation emerged as: “the use of data and analytics to improve or foster new products, processes, organizational methods and markets” (OECD, 2015, p. 21). Recently, a more specialised reflection has discussed the concept of data-driven innovation in relation to policymaking (Suominen and Hajikhani, 2021), as part of a field called data for policy (Verhulst et al., 2019). Across this scattered discussion, authors agree that the game-changing factor in data-driven innovation for policymaking is building policy knowledge from digital data not originally intended as evidence for policy decision-making (Durrant et al., 2018; Giest, 2017; Klievink et al., 2017; MacFeely, 2018). Doing so would allow the exploration of policy problems from a vast collection of heterogeneous non-traditional data sources (van Veenstra and Kotterink, 2017).

Unlike traditional data – usually collected for an extended period by actors with an official mandate to build evidence for public decisions (e.g. statistical offices) (MacFeely, 2018) – non-traditional data can be collected and updated faster and at a more granular level. A clear example of non-traditional data is administrative data, i.e. “datasets created primarily for administrative purposes by government agencies or other entities working on behalf of the government” (UN, 2019, p. 58). Compared, for example, to traditional surveys, administrative data could provide a more realistic picture of actual activities (e.g. records of public services use) rather than relying on people’s answers (Hand, 2020, p. 31). Governments are increasingly addressing the technical integration of non-traditional data, while their potential to foster processual innovation remains unrealised (van Veenstra and Kotterink, 2017).

However, in contrast with a decade of “big data” narrative that abstractly emphasised technological factors above all, the potential of non-traditional data suggests that data-driven innovation for policymaking will largely concern non-technological, processual and contextual challenges; for instance, the building of collaborations that support data governance and sharing across public organisations. Policymakers are already bombarded with information (Dunlop et al., 2018, p. 11); hence, the idea that they need to leverage large quantities of data seems misleading. Instead, they should be provided with insights from trusted and legitimated sources (Verstraete et al., 2021, p. 74). Accordingly, the use of non-traditional data for policymaking will have to align with the existing public bodies’ mandate and become germane to single policy context dynamics (Durrant et al., 2018; Klievink et al., 2017). Conceptually, we might start considering that the innovation locus of data-driven innovation for policymaking rests outside a reified conception of technology and within the complex policy processes where data might be collected and used (Concilio and Pucci, 2021; Giest, 2017).

Against this background, this article argues for design for policy as an approach toward the use of non-traditional data in policymaking (Leoni, 2020; Maffei et al., 2020), and asks: what might be the potential contributions of design for policy in data-driven innovation for policymaking? To answer, the article will specifically consider data generated from digital public services.

2. Data-centric public services as a source of policy knowledge

The first era of digital governance in the public sector revolved around adopting information and communication technologies (ICTs) (Tan and Crompvoets, 2022). ICTs enabled a generation of information-intensive digital services, which are today progressively automated and data-centric (Engin and Treleaven, 2019; Tan and Crompvoets, 2022). Being both material and performative (Orlikowski and Scott, 2015), this service generation is concretised by their technological infrastructures, physical touchpoints and service providers’ and users’ activities. Data recording, storage and exchange happen within and across these services, thanks to a combination of technology, routinised activities and service interactions. Policymaking practice might therefore leverage these services, essentially by using “[…] ICTs to capture the benefits of new data sources, and to support collaboration with relevant stakeholders and citizens” (van Veenstra and Kotterink, 2017, p. 1).

Aggregated administrative or micro-data can describe large-scale phenomena of public interest, broader than the individual services in which data are collected (Malomo and Sena, 2015). In this perspective, public services may increasingly not only regard service provision but become a source of information for policymaking, de facto working as digital interfaces between governments and other societal actors and probes into relevant policy areas (Giest et al., 2021). Further, they could be the basis of localised experimentation and stakeholders’ engagement in relation to a given public issue. Although many challenges remain (MacFeely, 2018), data-centric public services constitute the cornerstones for public organisations to be producers and consumers of non-traditional data, if capable of turning them into relevant and operational insights for policymaking (Ubaldi et al., 2019).

3. Contemporary discussion on data-driven innovation for policymaking

During the last years, the discussion on data-driven innovation developed closely with both technological advancements and the governments’ diverse efforts to catch up with the evolving concept of digital government/governance (Charalabidis et al., 2019). Such discourse has shifted from a focus on ICT adoption towards a data-centric perspective (Charalabidis et al., 2019; Tan and Crompvoets, 2022). This development might explain the emergence of a broad and explicit interest in data-driven innovation for policymaking, which coalesced into a dedicated, yet fragmented, field (Suominen and Hajikhani, 2021; Verhulst et al., 2019) while previously being addressed by dedicated discussions (Longo et al., 2015).

Today, the debate on data-driven innovation for policymaking seems polarised between overly optimistic and bluntly realistic (almost pessimistic) views (Vydra and Klievink, 2019). For some, data analytics and data science are innovations to be reckoned with because they reinforce scientific and evidence-based methods in policymaking (e.g. policy analysis) (El-Taliawi et al., 2021; Zhang et al., 2020). On the ground, however, the institutional use of non-traditional data for policymaking appears in its infancy, with only a few examples going beyond experimentations (Arnaboldi and Azzone, 2020; Durrant et al., 2018; Giest, 2017; Klievink et al., 2017; Poel et al., 2018; Verhulst et al., 2019). This division might be reflected in research, where authors divide between prospective and contextual views. Prospective views usually advance bird’s-eye views on the impact of data by referring to general technological applications within potential use scenarios. For example, they might discuss the potential of data-driven simulation and visualisation for computer-generated scenarios to aid policy decision-making (i.e. policy modelling) (Hagen et al., 2019). These views usually do not address specific contexts in-depth and provide illustrative examples (Dunleavy, 2016; Höchtl et al., 2016; Maciejewski, 2017; Azzone, 2018).

Conversely, the contextual views focus on specific institutional contexts in which data-driven innovations unfold, e.g. by focusing on a certain governance level, policy area or tool (Durrant et al., 2018; Giest et al., 2021; Lanza, 2021; Malomo and Sena, 2015; Williamson, 2016).

In terms of theory, within a generalised absence of frameworks (Verhulst et al., 2019), most authors would adhere to the policy cycle or policy tools perspective (Dunleavy, 2016; Höchtl et al., 2016). However, some noted that, while useful to simplify complexity, perspective as the policy cycle might obscure the internal, multiple and non-sequential levels of decision-making in the policy process (Concilio and Pucci, 2021). We further suggest these views entail a vision of policymaking that falls under the authoritative instrumental logic – i.e. they conceptualise policy as an instrumental act of problem-solving (Colebatch and Hoppe, 2018). This criticality brings to the fore the relevance of defining policymaking and its practices to understand “policy-data interactions” (Verhulst et al., 2019).

4. The distinctiveness of data-driven innovation for policymaking

Although an explicit discussion on data-driven innovation for policymaking appears recent, some public agencies have been incorporating data analytics for years (e.g. for law enforcement activities or financial fraud detection) (Athey, 2017; Dunleavy, 2016; Mureddu et al., 2022). As these solutions are increasingly enhancing with non-traditional data the core governing functions of most technically advanced public sectors, their value for policymaking should be interrogated. In our view, they constitute incremental innovations that define the use of data under well-defined framing of policy problems and solutions; therefore, they identify examples of data-driven innovation for public administration, rather than data-driven innovation for policymaking.

This distinction intends to overcome the authoritative instrumental view of policymaking described above, and encourages to look at policymaking not only as a rational act of problem-solving but as collective and practice-based process of problem definition. In fact, policymaking develops in a public and political space characterised by contested problems (Hoppe, 2011) where decisions and actions result from bargaining, mobilising available resources and exploiting windows of opportunity. It follows that non-traditional data cannot be applied instrumentally to policymaking (Giest and Ng, 2018, p. 3; Höchtl et al., 2016; Kettl, 2016) but are conditioned by individual policy practice (Durrant et al., 2018). Furthermore, as they require ecosystems of actors willing to experiment in a public decision-making process (Lanza, 2021), their use might define new experimental forms of policymaking (Concilio and Pucci, 2021). In this light, data-driven innovation for policymaking could benefit from design for policy as an approach proven to support collaborative experimentation and exploration of the problem space in policymaking (Bason, 2014). Because a gap between the two worlds still exists (Leoni, 2020), how they might converge under a new paradigm of government and governance seems relevant.

5. The design perspective towards policy innovation: design for policy

The potential of design has been a topic of increasing interest to design, public administration scholars and organisations (Hermus et al., 2020; Mortati et al., 2022). This interest is most clearly reflected in the establishment of a growing number of public sector innovation (PSI) labs and spaces worldwide, the majority of which apply design methods to innovate government policies and services (McGann et al., 2018). For this paper, we highlight three potential contributions of design for policy:

1. Anticipating long-term transitions through futures designing.

   • Design can support policymakers in building anticipatory governance (Maffei et al., 2020) and reframing political paradigms in long-term transition planning. This practice highlights the construction of problems and publics, revealing layers of the system, mediating diverse expertise and data and inviting broader participation (Kimbell, 2019).

2. Promoting collaborative governance through co-design and participatory design.

   • Design has revealed the potential of human-centred ways of governance that are “relational, networked, interactive and reflective” and reconceptualise the relationship between governments and the people they serve (Ansell and Torfing, 2014; Bason, 2014).

3. Connecting decision-making to public services through service design.

   • Design offers an opportunity to more tightly connect policies and services by conceiving services as more than the implementation of policy, but rather as the primary mechanism through which policies are realised and experienced (Junginger, 2013). The materiality of design also offers a way to make otherwise abstracted systems of government – and their attendant social, cultural and historical origins – tangible to citizens (Trippe, 2021; Tunstall, 2007).

However, despite the theorised potential and scholarly interest, the realisation of design for policy has been uneven and patchy. There has been limited success in institutionalising and legitimising design for policy outside PSI labs, for cultural, political and practical reasons (Clarke and Craft, 2019).

6. Methodology and analytical framework

To investigate data-centric public services as a source of policy knowledge, we developed a multiple case study qualitative analysis through desk research. The scope of the analysis is to advance a new theoretical approach and substantiate areas of convergence data-driven innovation for policy and design for policy. Desk research was chosen because it allows to quickly explore international phenomena and develop meta-analysis throughout several contexts, thus synthesising new inquiry fields by relying on recent secondary data, while it allows limited in-depth understanding if compared with other case-based methodologies.

Desk research was conducted as follows. Firstly, dedicated libraries and repositories from international bodies and think tanks (e.g. OECD, AlgorithmWatch, Nesta) were researched for a preliminary overview of data-driven innovation and public services, in parallel with the Data for Policy Conference proceedings (dataforpolicy.org/conference-papers). As a second step, we used keywords associated with the policy cycle to identify cases. Further, grey literature by public agencies was used to deepen the understanding of selected cases.

Cases were purposefully sampled according to the co-existence of the following inclusion criteria:

• explicit reference to the use of non-traditional data (in particular administrative and microdata);

• a focus on advanced or completed cases; and

• the sufficiency of provided information.

We conducted a meta-synthesis of cases (Miles et al., 2020) through an analytical framework that merges the policy cycle and the policy capacity framework (PCF) (Wu et al., 2018). The policy cycle provides a heuristic to address policy complexity (Bridgman and Davis, 2003), helping to specify the goal of actors in the cases considered. The PCF aims to operationally define the combination of skills and resources necessary for a governing body to perform policy functions, offering a theoretical lens that links policymaking with practices and procedures. The PCF individuates three dimensions of policy capacity: analytical, operational and political. Our analysis draws only on the analytical capacity dimension, which regards gathering evidence for policymaking (Wu et al., 2018) at three interpreting levels (following the macro-, meso- and micro-scale differentiation):

1. Systemic analytical capacity

   • Concerning the quality of system-level data collection and availability for integrated policy knowledge and an open policy process.

2. Organisational analytical capacity

   • Connecting the individual data collection to an organisational information system and infrastructure that collects and shares information across a policy sector.

3. Individual analytical capacity

   • Addressing the individual analytical and technical competencies to manage and process data and transform them into evidence for policy.

7. Case studies analysis

Table 1 displays an overview of the analytical framework across the four selected cases. For each case, we interpreted how non-traditional data from public services became a source of policy knowledge by affecting the three dimensions of policy analytical capacity.

7.1 Decidim.barcelona

Decidim.barcelona is an online open-source platform launched by the Barcelona City Council (Spain) aimed at improving participatory democratic processes in the city. The platform serves to build awareness of local issues, engage citizens in official deliberations and polling on policy proposals. Decidim.barcelona enabled the collecting of data on citizens’ opinions and ideas for agenda-setting and contributed to developing the strategic city plan from 2016 to 2019. The City Council was responsible for conducting the semantic analysis of citizens’ comments, clustering ideas and verifying their political acceptance. The agenda-setting was structured as follows: initial draft of the strategic plan involving traditional bodies for participation, citizens’ involvement through the platform and presentation of a final document submitted to the plenary of the city council.

7.2 Call detail records data for COVID-19 response

This use of Call Detail Records (CDR) data during the COVID-19 emergency resulted from collaboration among The World Bank, the Gambia Bureau of Statistics and the Public Utilities Regulatory Authority (PURA). Anonymised and aggregated CDR data were used to describe human mobility patterns and inform policy decision-making. To adopt the appropriate smart containment measures, the Gambian government strengthened existing data collection protocols from the involved stakeholders, merging data repositories, updating them in real-time and including additional indicators. They aimed to show changes in population and location of residency, demonstrate different levels of mobility over time and describe the population inflow and outflow during the lockdown.

7.3 Automated verification of medical prescriptions

The implementation of an electronic medical prescription system in 2016 allowed the Control and Monitoring Center of the Portuguese Public National Health Service to deploy an algorithmic system to detect medical prescription frauds. The Center integrated non-traditional data sources, including invoices issued by pharmacies, medical exams and electronic paper prescriptions from the databases of the Ministry of Health. Through unsupervised machine learning techniques that automatically analyse patterns, they could intercept discrepancies about the doctor and patient/beneficiary identity, allowing evidence-based monitoring procedures and public authorities intervention.

7.4 Santa Monica well-being project.

The project aimed at measuring well-being in Santa Monica (USA) between 2013 and 2015 for evaluation. After building a framework identifying the dimensions of well-being, the evaluative activity was conducted using different data sources: field surveys, administrative data sets and social media. Semantic analysis was conducted based on the well-being dimensions included in the initial framework. Results include the creation of the Wellbeing Index, which supports in identifying policy priorities.

8. Discussion: what role for design?

The analysis allows several potential considerations on the role of design for policy in data-driven innovation for policymaking through data-centric public services (see Figure 1). Three main aspects appear particularly defining of the potential design contribution in this scenario:

1. sensemaking and prefiguring for data sharing and data collaboratives;

2. collaborative experimentation for new data-driven policy indicators; and

3. service design approach for data-centric public services.

8.1 Sensemaking and prefiguring for data sharing and data collaboratives

The selected cases suggest that organisational interests, goals and capacity must be aligned within (Klievink et al., 2017) and across a data ecosystem for realising data sharing and data collaboratives (Susha et al., 2017). In the Portuguese case, for instance, territorial health agencies had to coordinate with the central government to develop a system for policymaking support based on strategic data sharing. If this strategic data sharing is ensured, systemic analytical capacity can be enhanced, and territorial data can support targeted public funding (Azzone, 2018). Design can help strengthen these partnerships through sensemaking and prefiguring activities (Maffei et al., 2020) meant to shape a commonly shared rationale behind intra-/inter-organisational data sharing. For practitioners, this may be valuable as the adoption of mission-oriented policymaking increases, fostering a sense of how each partner contributes to the mission by sharing data or joining a data collaborative. Further, stakeholders can adjust their sharing practices within the existing legal frameworks by considering possible future data uses and misuses and ethical considerations.

8.2 Collaborative experimentation for new data-driven policy indicators

The Santa Monica Well-being Project suggests a lack of systemic analytical capacities when it comes to fit non-traditional data into well-established indicators, which, nonetheless, must be linked to trusted indicators with a well-established data backrun for being used in policy decision-making (Vydra and Klievink, 2019, p. 3). Design might contribute to experimental participatory and prototypal activities that connect with the institutional level (Deserti et al., 2020). In practice, it might experiment with customised indicators that can be fed with non-traditional data (e.g. from prototypal services), eventually becoming an established knowledge resource. This implies the potential to enable collaborative approaches with civil society for infrastructuring them into institutional indicators, and aligns with the existing interest in citizen-generated data (Ponti and Craglia, 2020).

Finally, a service design approach for data-centric public services can connect high-level policy decisions with the socio-technical and contextual materiality of the public services and activities where data is collected. The cases selected suggest that the main design challenge is co-creating a data collection process with service actors while maintaining data quality. This highlights challenges for accessibility and individual capacities. For example, Decidim.barcelona influenced organisational analytical capacity by establishing a digital platform for citizen engagement. To lead to relevant policy knowledge, the platform should consider potential representativeness biases (Giest and Samuels, 2020) which itself depend on its design. In cases such as the one in Portuguese Public National Health Service, a service design approach can intervene ex ante to support individual analytical capacities and to integrate seamless data collection procedures in public services for policy purposes, therefore, accounting for both the service’s immediate needs and the strategic-/system-level need to create evidence for policy strategies. For practitioners, there are implications for designing services not only to achieve specific objectives, but also to contribute to interoperability challenges in data-centric public services that support the production of new policy knowledge, and governance and normative arrangements to support the generation and consumption of such data. Given the adoption of service design approaches in many public administrations, this may present a more concrete and scalable contribution of design for policy.

9. Conclusion

The increasing availability of non-traditional data collected through data-centric public services represents an unprecedented opportunity to develop new policy knowledge. As the review here presented suggested, this topic seems to be gaining increased attention and it is giving new momentum to past reflections on ICT-enabled innovation in government. We argued that turning non-traditional data into relevant evidence and knowledge for policymaking entails firstly a conceptual reframing of the policymaking process, without which it would be harder to isolate the pragmatic challenges. Following the path traced by important voices in policy studies (Colebatch and Hoppe, 2018), data-driven innovation for policymaking demands we take distance from an authoritative instrumental logic of policymaking because the co-creative nature of data-centric public services inevitably multiplies the loci of knowledge creation and subjects that create it.

The article argued for the potential of design to help connect the former and account for the materiality and contextuality of the latter, thus steering data-driven innovation for policymaking towards sustainable and socially acceptable policy practices. It identified three potential contributions of design for policy:

1. sensemaking and prefiguring for data sharing and data collaboratives;

2. collaborative experimentation for new data-driven policy indicators; and

3. service design approach for data-centric public services.

In terms of theoretical implications, the article has contributed to the existing discussion on policy–data interactions (Verhulst et al., 2019) and in line with previous similar efforts (Klievink et al., 2017) attempted to discuss data-driven innovation for policymaking outside the abstract and mythological narrative surrounding big data.

This was supported by an analysis of four case studies, drawing on the policy capacity framework (specifically, the analytical capacity dimension) and the policy cycle. Although limited in its empirical depth and external validity, this analysis offers an original framework to read data-driven innovation for policymaking, based on analytical capacity (thus institutions and people), rather than the reification of technological means. Outside disciplinary silos, we hope to see further – and, in our opinion, much needed – interdisciplinary studies on this theme and its practices.

Figures

Figure 1.

Role of “design for policy” in data-driven innovation for policymaking through data-centric public services

Table 1.

Effect on analytical policy capacity of non-traditional data use across case studies

Case name	Country	Policy cycle stage	Effect on analytical policy capacity	Main reference
decidim.barcelona	Spain	Agenda-setting	Systemic: public opinion influences agenda	Peña-López (2017)
			Organisational: meta.decidim, a community with representatives of citizens and civil actors, is responsible for designing and updating decidim.barcelona
			Individual: training sessions by public officers to enable citizens’ e-participation; data analysis and aggregation for the prioritisation of policy agenda by part of the Municipality
Call detail records data for COVID-19 response	Gambia	Policy formulation/decision-making	Systemic: not clear/accessible	Arai et al. (2021)
			Organisational: collaboration between Public Utilities Regulatory Authority (PURA), The Gambia Bureau of Statistics (GBoS), Ministry of Health, mobile network operators; a secure file transfer protocol (FTP) enables transfer of data from third parties
			Individual: use of the Hadoop platform for distributed processing of large data sets; strengthening of data collection protocols
Automated verification of medical prescriptions	Portugal	Policy implementation	Systemic: not clear/accessible	AlgorithmWatch (2020)
			Organisational: improved data collection through the electronic medical prescription system; access to an integrated data system of the Public National Health Service (NSN)
			Individual: data processing by part of Control and Monitoring Centre
Santa Monica Well-being Project	USA	Policy evaluation	Systemic: direct access to publicly available data through social media platform	OECD (2017)
			Organisational: enhanced connection with non-governmental institutes (i.e., research units) responsible for developing measurement
			Individual: use of the Wellbeing Index as a framework for planning the budgets and programmatic priorities; commissioned/outsourced data collection and aggregation activities