Data literacy training and use for educational professionals

Abstract

Purpose

A literature review of 28 data literacy, education articles from 2010 to 2018 was conducted to gain a better understanding of the current state of data literacy research.

Design/methodology/approach

A systematic literature review of ERIC, Education Source, and JSTOR was conducted. Articles were included in this literature review if they focused on “data literacy” for K-12 teachers or leaders.

Findings

Results demonstrated that the concept of data literacy has become more concrete, but there is still disagreement about the parameters of the construct. While data literacy was shown to be gaining in importance, training from schools of education were focused heavily on assessment literacy. Four recommendations are made as follows: (1) create skill-focused educator prep programs, (2) encourage opportunities for collaboration, (3) model data use from both quantitative and qualitative sources and (4) investigate the role of technology and big data on data literacy.

Research limitations

The scope of this literature review was very narrow and, as such, does not fully encapsulate data-driven decision-making in K-12 education overall.

Originality/value

Data literacy is important for both teachers and leaders, as educational environments strive to better understand individual learners and improve learning outcomes. This literature review looks to pull together the current status of data literacy research with hopes of inspiring more targeted research that influences training practices for both teachers and leaders.

Citation

Henderson, J. and Corry, M. (2021), "Data literacy training and use for educational professionals", Journal of Research in Innovative Teaching & Learning, Vol. 14 No. 2, pp. 232-244. https://doi.org/10.1108/JRIT-11-2019-0074

The concept of data literacy refers to “the ability to transform information into actionable instructional knowledge and practices by collecting, analyzing, and interpreting all types of data” (Gummer and Mandinach, 2015, p. 2). The need today for educational professionals to have strong data literacy skills tracks its start to the US federal law No Child Left Behind (NCLB) in 2001. NCLB increased the role of the federal government in education, traditionally viewed as a state power under the Tenth Amendment of the US Constitution, and introduced the topic of accountability for student learning outcomes based on standardized test data (Wiener and Hall, 2004). NCLB was revised in 2015 to the Every Student Succeeds Act (ESSA), which granted more flexibility in student achievement tracking by state while still requiring overall accountability measures (U.S. Department of Education, 2015). Today, almost two decades after the initial passage of NCLB, many school systems have “accountability departments” to track student achievement via standardized test scores and other relevant achievement metrics. In addition, as of 2017, 39 states required that educational professionals be evaluated, at least in part, by student achievement data metrics (Ross, 2017). The use of data-driven decision-making in education is only likely to grow as big data sources, the computational abilities of machines and the cultural importance of data increases, further emphasizing the foundational importance for teachers and leaders to be data literate.

Despite a history of federal accountability laws and fast-paced changes toward a data-driven society, training on data literacy skills are shown to be lacking depth and authentic application, both at the higher education level for pre-service professionals (Cowie and Cooper, 2017; Mandinach et al., 2015) and during on-the-job professional development for in-service professionals (Schildkamp and Poortman, 2015). Data literacy incorporates aspects of statistical literacy, assessment literacy, pedagogical knowledge and data-driven decision-making under one umbrella. While the specific skills and knowledge for this construct seem to be still evolving in the literature, the definition by Gummer and Mandinach (2015) is efficient in describing the big picture skills for educators in using data to guide students for instructional practices. It is for this reason that data literacy was chosen as the focus of this literature review.

The literature landscape on data literacy is rather small but offers insight into suggested practices for preparing professional educators to operate in a data-driven world. This paper reviews the literature within the current decade, from 2010 to 2018, on data literacy for educational professionals. Search procedures, descriptive analysis of the literature and a discussion of recommendations for the improvement of current practices are covered below.

Methods

A search of major databases for educational literature was conducted in the spring of 2019. Searches of ERIC, Education Source and JSTOR were conducted, in that order, using the search terms “data literacy” and education. Data literacy was placed in quotes to ensure that the two words appeared together in the article. The search was limited to peer-reviewed journal articles that were published between 2010 and 2018 in English. Education Source returned the most articles (n = 58), followed by JSTOR (n = 45) and ERIC (n = 43). A large portion of articles showed up in multiple databases. See Table I for a count of articles used from each database; articles were only counted once from the database where they were first located.

Abstracts were reviewed for relevance during the search of each database. If the abstract contained information on data literacy for K-12 teachers or leaders then articles were downloaded for in-depth review. Literature was not included if it had populations of media specialists, librarians, students or was literature focused on higher education. Higher education articles were included, however, if they focused on the training of K-12 teachers or leaders. In addition, articles were not included that focused on big data, educational data mining or data analytics, unless these had a specific connection to data literacy as defined above. A database was created that included relevant information on each article, including: study purpose, population of focus, methodology, operationalization of data literacy, results and theoretical frameworks. The specific search parameters helped ensure that almost all articles targeted for in-depth reading, via the abstract, were included in this review. Table I shows the high rate of articles identified from abstract review that were also used in this paper.

The scope of this literature review was specifically narrow. “Data literacy” was searched in each database using quotations to ensure that this exact term was present in the title, abstract or article. This narrowness of scope allowed for a tight analysis around the still emerging topic of data literacy in education, but it did not include analysis of articles that solely discussed topics of assessment literacy, research literacy, data-driven decision-making, mathematical-statistical literacy and/or big data. While these topics had overlap, they are different, stand-alone constructs overall. Further review, clarification and differentiation of these topics are suggested for the future.

Results

The results of the literature review are based on 28 peer-reviewed, journal articles and are broken into three sections as follows: an analysis of definitions of data literacy across articles, a descriptive analysis of similarities and differences of articles and an analysis of themes in data literacy topics, with educational role broken into portions, on teachers, both pre-service and in-service, and leaders, both principals and middle-level facilitators.

Defining data literacy

It has taken time to build a common understanding of data literacy in the field of education (Hamilton et al., 2009; Mandinach and Gummer, 2013). The articles in this literature review reinforce this claim from Mandinach and Gummer (2013), as the articles on data literacy prior to 2013 all had slightly different definitions and sources of data literacy. Vanhoof et al. (2011) said data literacy was “the strategies, skills and knowledge needed to define information needs, and to locate, evaluate, synthesize, organize, present and/or communicate information as needed” (Williams and Coles, 2007, p. 188). This definition covers relevant data-based skills but lacks any mention of action based on the data. Athanases et al. (2012) defined data literacy as “the capacity to understand how to generate, interpret, and use data in teaching” (p. 6). This definition, based on one created by Popham (2008), includes using data in teaching; an updated article by Athanases, Bennett and Wahleithner, in 2013, changed to a more detailed definition, from Love et al. (2008), that leaves out explicit connection to informing and changing instruction. They said that “data literate educators (1) conduct collaborative inquiry to promote equitable achievement, (2) work with available data and data educators collect, and (3) learn to understand and name how they derive meaning from data” (p. 12). Hamilton et al. (2009) defined data literacy as “the ability to ask and answer questions about collecting, analyzing, and making sense of data” (p. 47). This definition, also lacking a direct connection for using data to inform instruction, showed up in two articles in this literature review (Reeves and Honig, 2015; Verbiest et al., 2014).

The most common definition of data literacy was from one of five articles by Mandinach et al. (Gummer and Mandinach, 2015; Mandinach, 2012; Mandinach and Gummer, 2013; Mandinach and Gummer, 2016; Mandinach et al., 2008). While their definition of data literacy has gotten more detailed and technical over time, the base definition is “the ability to understand and use data effectively to inform decisions. [Data literacy is] composed of a specific skill set and knowledge base that enables educators to transform data into information and ultimately into actionable knowledge” (Mandinach and Gummer, 2013, p. 30). These articles have worked to identify major stakeholders (Mandinach and Gummer, 2013), built a conceptual framework (Gummer and Mandinach, 2015), evaluated university teacher training programs (Mandinach et al., 2015) and developed more clear teacher skills, knowledge and dispositions (Mandinach and Gummer, 2016). These articles by Mandinach et al. have become foundational literature on the topic of data literacy in education, as is demonstrated from the high number of citations they receive in the other data literacy articles. Only seven of the 28 articles included in this literature review do not use an article by Mandinach et al. as a reference.

Even though there is an increased consensus on the terms and definitions, a common academic vocabulary around data literacy skills and knowledge is still evolving in the field. This was evident even within this literature review, which had a narrow scope of only including articles that specifically mention data literacy, over other data competencies. Verbiest et al. (2014) used the term “data-wise” as a synonym for data literacy (p. 64). One other article used this term, but it was the name of a teacher preparation program at Harvard called Data Wise and was not intended as a synonym for data literacy (Bocala and Boudett, 2015). Bocala and Boudett (2015) did, however, equate data literacy with data inquiry. Data inquiry referenced collaborative groups of educators using data literacy skills together to inform instructional practices. Finally, LaPointe–McEwan et al. (2017) seem unclear on the relationship between evidence literacy and data literacy. At one point, they referred to evidence literacy as a synonym to data literacy but then later stated that a lack of evidence literacy in teachers hurts their data literacy and research literacy skills, insinuating that these are separate constructs that influence one another (LaPointe-McEwan et al., 2017). The lack of clear constructs and academic vocabulary around data literacy in these articles may hinder the development of programs that adequately train professionals in the skills and knowledge of data literacy, as it is important to have a common language and understanding of these topics when developing curriculum.

On the other hand, some authors work to clearly differentiate data literacy from other related constructs. Dunlap and Piro (2016) discussed the importance of statistical literacy on data literacy for educators. They defined statistical literacy as the understanding of statistical terminology and ability to practice statistical techniques with sample data sets (Dunlap and Piro, 2016). Cowie and Cooper (2017) refer to mathematical and statistical literacy as the base skills needed for educators to build strong data literacy skills. They used a theoretical framework created by Pierce et al. (2014) on “teacher professional statistical literacy”, to clearly differentiate the statistical knowledge needed for successful application of data in instruction (Cowie and Cooper, 2017, p. 149). LaPointe–McEwan et al. (2017) differentiated between research literacy and data literacy; their results showed that even with an increase in data literacy skills research literacy skills were still lacking in middle leaders and in-service teachers. This lack of research literacy skills was demonstrated in the results when teachers and middle leaders did not question data given to them from experts or seek out research findings that would verify their conclusions. Finally, the construct that seems to cause the most confusion with data literacy is assessment literacy. Mandinach and Gummer (2013) stressed that assessment literacy is a type of data literacy but that not all data literacy is assessment literacy. This was a common point in the work of Mandinach et al. (Mandinach and Gummer, 2016; Mandinach and Jimerson, 2016). Assessment literacy is defined as data knowledge, skills and uses from traditional or standardized assessments (Mandinach and Gummer, 2013). While assessment literacy is one part of data literacy, it is not the only part, as educators should be using a variety of other data points, like motivation, behavior or perceptions (Mandinach and Gummer, 2013). These additional types of data are often qualitative in nature and not often the focus of teacher or leader training on data-driven decision-making. Mandinach et al. (2015) conducted a study of 208 colleges and universities involved in teacher training, collecting survey results from each school and analyzing 80 syllabi to assess the level and type of data literacy training that was occurring. While the survey results suggested that data literacy was offered at most schools, either in a stand-alone course or embedded into courses on other topics, the syllabi review found that the focus was on training educators in assessment literacy, over building a variety of data literacy skills. Outside of articles by Mandinach et al., only Cowie and Cooper (2017) spent time differentiating between assessment literacy and data literacy. They stressed that pre-service teachers need to be trained in data literacy, assessment literacy and mathematical/statistical literacy (Cowie and Cooper, 2017). Three articles used the term data literacy but ultimately described assessment literacy (Dunlap and Piro, 2016; Piro et al. 2014; Piro and Hutchinson, 2014). This further demonstrated the unclear line between assessment literacy and data literacy as constructs. Clearer lines between these two constructs may help universities, colleges and school systems to design stronger training programs that ensure training on creating instructional decisions using more than just assessment data.

Unfortunately, a few sources in this literature review talked about data literacy but never specifically defined what the construct meant (Hewitt and Chopin, 2015; Schildkamp and Poortman, 2015). Hewitt and Chopin (2015) detailed an exercise used in a data literacy class for pre-service administrators but did not cover their definition of data literacy. Finally, Schildkamp and Poortman (2015) noted that individual teachers in data teams lacked data literacy skills, but they did not go into detail on specifically what encompassed this concept. They generically stated that data literacy had to do with knowledge and skills for data use but then they seemed to equate data literacy to mathematical/statistical skills. As shown above, a variety of constructs are perceived differently by different researchers. This reinforces the need of researchers to clearly operationalize how they view these constructs, for clear comparison on both definition and measures.

One of the biggest challenges in moving forward with data literacy training and use for teachers and leaders is building a common understanding of the knowledge, skills and vocabulary around the topic. While Mandinach and Gummer (2016) have made real progress in both defining these constructs and moving the academic field toward a common verbiage, these examples show there are still different interpretations of the parameters of data literacy in comparison with other constructs.

Descriptive analysis of similarities and differences

Even though articles on data literacy have steadily increased in number from 2010 to 2017, they are still small in total as only 28 articles were relevant to this literature review from three educational databases. 2015 shows eight dedicated data literacy articles, a spike in comparison with the other years. This is due to a themed issue, number four, of the Teachers College Record on data-driven decision-making, where five of the eight articles from 2015 were found (Bocala and Boudett, 2015; Gummer and Mandinach, 2015; Jimerson and Wayman, 2015; Mandinach et al. 2015; Schildkamp and Poortman, 2015). See Figure 1 for an article count over time.

None of the 28 articles covered research samples and/or were written by authors in Africa, Asia or South America. All articles were from North America (n = 19), Europe (n = 7) and Oceania (n = 2). The United States had the most articles, followed by the Netherlands. Literature on data literacy from other parts of the world may call the construct by a different name. This gap may also be due to a search of only academic articles that were written in English. Further research into data literacy in other parts of the world would add value to this literature area.

Out of the 28 articles in this literature review, almost a third of the articles were on recommendations for best practices or works conceptualizing data literacy, while the other two-thirds had original data collection. There was a pretty even spread in the type of research conducted for original data collection across the articles of this literature review with six quantitative, nine qualitative and five mixed methods.

Data literacy by educational role

The majority of articles in this literature review focused on data literacy training at the college/university level for pre-service teachers (n = 11). A lesser number looked at training for leaders (n = 4) and in-service teachers (n = 7) or a combination of these populations (n = 6). Literature on teachers, both pre- and in-service, focused almost exclusively on investigating the effects of an instructional technique on teacher data literacy competencies. Pre-service teachers were investigated in relation to a college-level preparation course, and in-service teachers were investigated in relation to professional development opportunities. Studies on leaders mainly focused on principals, with one focused on middle leaders/facilitators (LaPointe-McEwan et al., 2017).

Most articles on data literacy are focused on the population of teachers, either the college or university level, for pre-service teachers or professional development trainings for in-service teachers. While the training location and work environment may be different between these two populations, the overall recommendations for both groups focused on the same topics as follows: importance of collaborative inquiry, breakdown of specific skills and dispositions needed for success, as well as instructional design evaluations.

The traditional design of schools can make collaboration between educators difficult as both time and facility limitations can keep teachers isolated. Multiple articles on data literacy stressed the importance of collaboration in building a positive school data culture. Data chats in professional learning communities (PLCs) are becoming more common in educational settings as a way for teachers to discuss a variety of student achievement data and progress with similar subject or grade level colleagues. Piro and Hutchinson (2014) and Piro et al. (2014) investigated using data chats, similar to the concept of a PLC, with pre-service teachers. This teacher education program partnered with local school districts to obtain anonymized student data so that pre-service teachers would have the opportunity to practice in a more authentic way. Their results demonstrated that the pre-service teachers found that involvement in the data chats helped them to both better understand statistical concepts and feel more prepared for classroom application of data (Piro et al., 2014; Piro and Hutchinson, 2014). Jimerson and Wayman (2015) described six areas that in-service teacher professional development needed to incorporate as a focus, with one of these areas stressing the importance of sharing information and collaborating with colleagues. This qualitative study highlighted in-service teacher perceived benefits of collaboration, even when that collaboration needed to occur outside of work hours due to the time constraints of educators. They also discussed some perceived negatives in collaboration; for example, one teacher noted that some peers felt defensive in conversations around data, worrying that they would be judged negatively solely based on the data. Overall, Jimerson and Wayman (2015) noted that the school districts involved in created policies to show support for teacher data collaboration, but they were found to be lacking in actual implementation support.

A subset of articles on in-service and pre-service teachers focused on the development of data literacy skills and the dispositions that help them be the most effective. Mandinach and Gummer (2016) used input from experts, professional organizations and policy requirements to build a conceptual framework of skills and knowledge needed for teachers to be data literate at each stage of data collection, analysis and implementation. They emphasized that these skills need to be taught early in a teacher's training and/or career, to have a real impact on that teacher's practice. In addition, they discussed dispositions or habits of mind that are important for educators to have in order to be data literate. These were not included in the conceptual framework Mandinach and Gummer (2016) created but included that teachers needed to believe that all students can achieve, in the importance of collaboration, and in the ethical use of data and protection of student data, among others. Bocala and Boulder (2015) emphasized the role of habits of mind in successful data use, analysis and implementation for teachers. They described a key point in Harvard's Data Wise program, for both pre- and in-service teachers, was to build habits of mind that focused on the importance of collaboration, using evidence in decision-making and having a shared commitment to a data-driven, decision-making cycle (Bocala and Boulder, 2015). Cowie and Cooper (2017) stressed the importance of mathematical and statistical skills for teachers beyond just those teaching mathematics and science in schools. Through a mixed methods design, they collected data from pre-service educators and college professors, finding that professors did not explicitly teach mathematical or statistical skills to pre-service teachers, which may have contributed to the pre-service teacher results that highlighted a lack of confidence in those areas (Cowie and Cooper, 2017). Programs for teachers, both in-service and pre-service, need to not only have curriculum that makes room for data literacy skills but also need professors and trainers who explicitly model and discuss data processes and mindsets.

A lot of articles focused on populations of teachers analyzed a specific course, program or instructional activity. Most of these attempted some objectivity by ensuring that at least one author was not involved in the teaching of the course in order to assess the results from a more objective eye; however, not all studies were designed in this manner, leaving some to simply describe an instructional sequence with no empirical evidence of student impact or success. Van Geel et al. (2017) conducted a large-scale quantitative study that used pretests and posttests to measure the data literacy levels of 1,182 Norwegian pre-service teachers before and after training on making data-based decisions with an electronic student management system. While they found that differences in pretest scores could largely be equated to differences in level of education, bachelor's degree or master's degree, the posttests erased those differences, demonstrating the success of the intervention on improving pre-service teacher data literacy skills (Van Geel et al., 2017). Carey, Grainger and Christie (2018) evaluated one Australian undergraduate teacher preparatory course that focused on building data literacy skills and knowledge in teachers. The course had two parts as follows: a classroom component and a field work component. Initially, many students were concerned about their lack of ability to act on data in the classroom even though they reported feeling confident in their overall understanding of statistical concepts. After the field experience, all students reported an increase in confidence and preparedness for using data in the classroom. Reeves and Honig (2015) investigated undergraduate, pre-service teacher perceptions of a data literacy intervention as part of an elementary education assessment course. The intervention had students practicing to analyze a variety of data and making instructional decisions. They used pre-formatted Microsoft Excel sheets to practice visualizing and interpreting data in a scaffolded way. Survey results indicated an increase in pre-service teacher self-efficacy from this experience that was designed to be more authentic to the classroom experience of a teacher.

While teacher data literacy is a main focus, the data literacy skills of leaders are also key for teacher and school success, overall. The Council of Chief State School Officers (CCSSO) has integrated some data literacy skills into recommended standards for administrators by stating, “administrators [need] to know how to collect and use data to identify goals, assess organizational effectiveness, and promote organizational learning” (Mandinach and Gummer, 2013, p. 31). Six articles in this literature review focused on the population of school leaders' training and use of data literacy skills. Five of these dealt with school administrative leaders like principals or assistant principals, with one focused on middle leaders/teacher facilitators. The articles on leadership and data literacy focused on three main topics as follows: helping administrators build a culture of data literacy that supports teachers, preparing administrators for meeting state requirements when using data in teacher evaluations and training administrators on using data for school-based policy changes.

A school's culture of data use can have serious impacts on the way in which educators think about and use data (Farrell and Marsh, 2016). School leaders have an important role to play in setting a school's culture around data use. Uiterwijk-Luijk et al. (2017) surveyed 79 in-service primary school principals in the Netherlands. They found that principals with higher degrees were statistically significantly better at communicating a culture of inquiry to their staff, indicating that education may be one aspect that prepares administrators for adequately building a common culture within their schools. Principals with higher perceived rates of self-efficacy were also found to be better at stimulating data literacy within their teachers (Uiterwijk-Luijk et al., 2017). Schildkamp and Poortman (2015) conducted qualitative case studies of data teams made up of teachers and administrators. They noted the negative effect a leader could have on the entire culture of the data team by using “shame and blame” tactics that keep teachers from feeling trust to ask questions and take calculated risks that may benefit students (Schildkamp and Poortman, 2015, p. 35).

As of 2017, 39 states required that educational professionals be evaluated, at least in part, by student achievement data metrics (Ross, 2017). The use of student achievement data in teacher evaluations requires administrators who are fully trained to not only guide teachers through the process but to have a deep understanding of data complexities that prepare them for real-world application that meets the law's requirements while being fair to the educators they represent and lead. Hewitt and Chopin (2015) wrote a non-empirical summary of a case study used in two administration prep courses at the higher education level. This case study asked future administrators to put themselves in the shoes of a middle school principal who needed to make tough personnel decisions based on data standards set by the state. The authors claimed that through this process these future administrators could gain a better understanding of the strengths and hurdles that come along with using student achievement data for teacher evaluations. Hewitt and Chopin (2015) described the scenario in depth but did not measure or record responses of the administrative students involved with using the case study in class. Student evaluative data are needed to demonstrate the effectiveness of a particular instructional idea. Future literature on suggested best practices needs to include, at least on a small scale, some evaluation of effectiveness for those who may try to implement the practice.

A final topic seen in the literature on data literacy and leadership was regarding the ways that administrators created schoolwide policy changes based on data. Verbiest et al. (2014) was a non-empirical review of online modules that trained pre-service administrators on data literacy topics. They suggested that training for future administrators need to be grounded in learning activities and approaches that mirror the data situations that they faced in real life while on the job. In addition, they highlighted the importance of administrators having a grasp on both the external, like accountability, and internal, like developmental, purposes of data (Verbiest et al., 2014). Training for in-service administrators was shown to be useful in building data skills and in improving data attitudes; however, administrators were found to lack in actual use and application of those skills and attitudes in everyday leadership at their schools (Uiterwijk-Luijk et al., 2017; Vanhoof et al., 2011). LaPointe-McEwan et al. (2017) investigated collaborative inquiry through the relationship of teachers and leaders. They conducted focus groups of K-12 teachers and middle leaders, also known as teacher facilitators, over the course of a long-term professional learning workshop that focused on building data literacy through collaborative inquiry techniques. They found that in the early stages, in-service teacher data literacy skills were directly related to facilitator data literacy skills (LaPointe-McEwan et al., 2017).

Discussion

Based on the results of this literature review, we have created four recommendations for improving data literacy for professional educators as follows: (1) create more skill-focused educator preparatory programs at colleges and universities, (2) encourage opportunities for collaboration between educators, (3) model and encourage data use from both quantitative and qualitative sources and (4) investigate the role of technology and big data on data literacy.

It has been almost two decades since the passage of NCLB; yet, many states have limited or unclear licensure requirements around topics of data literacy. Many higher education schools of education look to state requirements when designing curriculum; as such, many of these schools have not adjusted in the manner necessary to adequately prepare educators, both leaders and teachers, for the data-driven society of today (Gummer and Mandinach, 2015; Mandinach et al., 2015). Some schools are slow to change because they have faculty who may feel uncomfortable with data literacy skills themselves or do not see value in data-driven decision-making (Cowie and Cooper, 2017). Even if a higher education teacher or administrator preparatory program includes data literacy skills and knowledge, they often only focus on the assessment literacy side by focusing on analyzing assessments by standards using quantitative, statistical techniques (Mandinach et al., 2015). Schools of education now have conceptual frameworks of skills and knowledge, which can be used as a starting point in making curricular changes (Gummer and Mandinach, 2015; Mandinach and Gummer, 2016). From there, schools of education need to bring together a variety of stakeholders so that diverse perspectives are represented to ensure that teachers learn to analyze and use data thoughtfully to avoid implicit bias with minority students and those with disabilities (Athanases et al., 2012; Bertrand and Marsh, 2015). Finally, school districts need to build stronger connections with local higher education institutions to help increase capacity and expertise for professional development around data literacy skills and knowledge (Ebbeler et al., 2017).

Collaboration between educators in schools and programs with positive data cultures and supportive leadership has been shown to improve data literacy skills (Piro et al., 2014; Piro and Hutchinson, 2014). School districts need to not only create but support policies that build structures for more efficient and effective teacher-to-teacher collaboration. This may take the form of PLCs or data chats. For these to work, and not be a waste of time, leadership must share a positive vision of collaborative inquiry and must focus on proper implementation throughout the year (Hoogland et al., 2016; Uiterwijk-Luijk et al., 2017). In addition, schools of education must train pre-service teachers to participate in data chats and train pre-service administrators to participate, train and support their staff in data chats. This will require higher education institutions to build partnerships with local school systems so that pre-service educators can have more authentic experiences by being placed directly in a school or by being able to analyze real student data (Reeves, 2017).

Teachers must be exposed to data-driven decisions that were made using both quantitative and qualitative data (Athanases et al., 2013; Bocala and Boudett, 2015; Verbiest et al., 2014). There is currently a heavy focus on student quantitative data from summative assessments. While this data has value, there are other pieces of information on demographics, from observations, or on the student's motivation that may also add value when a teacher is deciding the best instructional path for a particular student. Trust can be developed if higher education professors and educational leadership teams in schools model data-driven decision-making that uses qualitative data too.

Finally, only a few of the articles on data literacy include discussions of building the technology skills necessary for data literacy in a big data world (Jimerson and Wayman, 2015; Mandinach et al., 2015; Van Geel et al., 2017; Verbiest et al., 2014). Technology is not separate from data literacy. Many teachers will access data, both qualitative and quantitative, through districtwide or schoolwide databases. Student information systems, learning management systems and online testing systems are just a few of many platforms that teachers need to learn, to navigate, extract data and interpret visualizations. The role of technology in data-driven decision-making will increase as machine learning and artificial intelligence systems make recommendations based on big data from student keystrokes, progress and outcomes tracked in learning systems. Data literacy skills are foundational for the ability to not only understand the machine-based recommendations being made for students but also for educators to trust those recommendations. More research is needed on the way that teachers interact with technology when making data-related decisions – in both small and big data circumstances.

Future work

There is room for a variety of different avenues of new work around the topic of data literacy. First, further refinement and clarity of key constructs may help to build stronger recommendations for training and state licensure requirements around data literacy for both teachers and leaders. Second, additional empirical-based studies are needed to investigate best practices for effective data literacy training that has authentic application to classrooms and schools. Finally, understanding data literacy's place in the fast-changing world of big data and educational data mining is critical to ensure that professional educators have the foundation of skills needed to successfully use these resources for their intended instructional benefit for students.

Limitations

This literature review covers a very specific search parameter, and as such, may leave out articles that define data knowledge, skills and use in education differently from “data literacy”. Sources were identified during review of the targeted articles that were in the date range but did not appear in the search. As snowballing techniques were not used here, these articles were not included. This highlights that there may be additional information on data-driven decision-making or statistically guided instructional change that is not present in these results.

Conclusion

The early 2000s ushered in an age of data-driven decision-making in education through federal legislation; yet, almost two decades later, both literature and skill training on data literacy has been slow to develop. This literature review found that progress is being made on a common data literacy definition and understanding of relevant skills and knowledge needed for practitioners, but found that training programs, both pre-service preparation and in-service professional development, have been slow to keep up, for a variety of reasons. With increases in artificial intelligence technology and computational speeds for large datasets, the field of education is likely to experience more of a push to data-driven decision-making at all levels. It is critical that teachers and leaders be trained to have a strong foundation of data literacy skills, to ensure not only an understanding of data collection and statistical techniques but also understand how that data should be used to inform instruction.

Figures

Figure 1

Breakdown of included articles by year

Table I

Search results by database

Database	Total results	Total pulled	Total included
ERIC	42	20	19
Education Source	58	9	8
JSTOR	45	1	1