Guest editorial

Citation

Wardrip, P.S., Abramovich, S., White, A., Penney, L., Chang, S. and Brahms, L. (2021), "Guest editorial", Information and Learning Sciences, Vol. 122 No. 3/4, pp. 121-126. https://doi.org/10.1108/ILS-03-2021-261

Designing, evaluating and assessing learning and engagement in school, library, museum and community makerspaces

Introduction

Over the past 10 years, interest in and the use of making and tinkering in education has continued to increase. In an effort to support and promote this educational innovation, this special issue aims to further understanding of learning and engagement in educational makerspaces by bringing together research articles that explore conceptualizing and designing assessment from a variety of approaches and theoretical frameworks.

The eight innovative articles included in this special issue address conceptual, critical and empirical topics relevant to design, assessment and evaluation in educational makerspaces. These articles draw from a range of methods including design-based research, qualitative interpretive analysis, case study and interviews, as well as quantitative experimental research, surveys and electrodermal activity analysis. The articles are also grounded in maker-based learning experiences across settings such as K-12 schools, public libraries, afterschool programs and maker events. Together, this research addresses timely questions and presents compelling research findings that move forward our understanding of and conversation about learning through making. In doing so, they may also advance, encourage and even spark us to consider future questions and the methods, modes and motivations of assessment within and across diverse learning disciplines and contexts.

Background on maker-based learning

Making and learning in makerspaces – recognized as a social, technological and economic movement – is characterized by participants’ interest-driven engagement in creative production at the crossroads and fringes of disciplines such as science, technology, engineering, art and math (Dougherty, 2016; Sheridan et al., 2014; Author, 2016). This movement continues a long pedagogical tradition of learning-by-doing (Dewey,1916/2009; Harel and Papert, 1991) and mirrors natural human practice and culture; humans make things for pleasure (Korn, 2015) as well as to survive and make a living (Rose, 2014; Vossoughi et al., 2016). In education, making and tinkering (a similar practice to making) are avenues for disciplinary learning science, technology, engineering, art and math (Calabrese Barton and Tan, 2018; Honey and Kanter, 2013; Peppler and Bender 2013), workforce development (Anderson, 2012; Hatch, 2014), persistence in the face of failures during the development of innovative and entrepreneurial skills (Benton et al., 2013) and technical literacy (Lande and Jordan, 2014).

The practice of making and tinkering as an educational approach has spread across a wide variety of settings, such as libraries (Cun et al., 2019; Bowler and Champagne, 2016; Chang et al., 2019), museums (Wardrip and Brahms, 2015; Gutwill et al., 2015; Wardrip et al., 2016), K-12 schools (Eriksson et al., 2018; Koh et al., 2019; Wardrip and Brahms, 2016) and afterschool settings (Bevan et al., 2017; Barton et al., 2016). Since the Obama administration advocated for a Nation of Makers (White House, 2014), educational makerspaces have been growing in schools around the USA (Kim et al., 2018). Similarly across the world, makerspaces and maker-based learning has been growing in schools and other educational institutions (Xi et al., 2017; Vuorikari et al., 2019; Rayna and Striukova, 2020).

Assessment is integral to good instruction (Shepard, 2000), and designers of makerspace learning experiences are increasingly using assessment and evaluation approaches to establish learning goals and objectives and to research the ways in which the designed experience actually meets them. As making has become a stronger educational movement, the need to identify and document the learning that is taking place from and through making is critical to advance both theory and practice, conceptualizing and supporting making as a learning process. Maker educators also need assessments to provide meaningful feedback to learners, to justify programs and curricula, to make claims about learning and engagement and to support their own professional development and reflection.

But despite the necessity of assessment, researchers and practitioners have both noted the challenge of assessing maker-centered learning in educational settings (Lin et al., 2020) and informal, media-rich environments (Lemke et al., 2015). Maker-based learning experiences are motivated by a diversity of goals that guide educators (Wardrip and Brahms, 2020). In addition, students often make different products in their time working on maker projects, which means that the evidence of learning might look different across multiple products (Keune and Peppler, 2019). Furthermore, there is often the intention of not interrupting the learner’s experience while the learner is engaged in a maker project making it desirable to embed assessments in the activity (Murai et al., 2020). Fundamentally, it can be challenging to enact assessment practices that align practically and epistemologically with learning experiences that are active and physical, like making (Gillies and Boyle, 2011).

Current research on assessment work in makerspaces has taken a variety of forms. Lin et al. (2020) have noted in their review of the literature on technology-rich maker activities that five types of assessment tools have been prevalent in other studies. These include artifact assessments, tests, surveys, interviews and observations. However, many of the studies from the Lin et al. (2020) review focused on assessment tools that are not used by educators. Instead, they are assessment tools used by researchers and evaluators. In more analog-focused settings, observation tools and practices are used to identify evidence of learning (Gutwill et al., 2015; Kumar et al., 2019; Martin et al., 2019) as well as embedded strategies (Murai et al., 2020). Much of the existing research in this area also involves time-consuming ethnographic methods that necessitate the funding, time and expertise of trained evaluators or learning researchers (Sheridan, et al., 2014; Buchholtz, 2014). There still is a large need for research on tools and measures that practitioners, as well as researchers and evaluators, can use to capture and utilize meaningful and relevant data to improve their practice.

Moving the field forward – articles in this issue

The articles in this special issue push forward the research around formative assessment, highlighting the value of looking closely at the making and learning process. Often, makerspaces are environments where this process can naturally happen, but that may not be the case with more traditional learning approaches and programs. Thinking processes, reflection, iteration, even social engagement are often significantly less visible or apparent when one right answer is required. More broadly, the articles in this special issue suggest that we can see and assess learning and engagement in a variety of ways by using an assessment repertoire. For example, Lee et al. (this issue) look at measuring the quality of engagement in maker activities that involves electrodermal activity data as well as first-person video. Chen and Bergner (this issue) consider the teachers’ reflections as a source of assessment as well as an area for their own capacity development.

The articles in this issue also suggest implications for the design of maker-based experiences and how that design is linked to assessment. For example, Nixon and colleagues (this issue) consider how we might engage in assessment when the maker-based learning experience is a one-time workshop or other brief form of making. In addition, (Humburg et al., this issue) explores the design of maker-based learning experiences from a lesson planning perspective. In this case, lesson plans serve as a tool for designing for making, but also serve as a site of evidence for the educators’ motivation, beliefs and capacity to design these experiences. Moreover, (Worsley et al., this issue) investigates different ideation strategies in designing maker-based learning experiences. And more fundamentally, (Lee, this issue) seeks to make a link between aspects of a maker experience and student engagement. These four studies contribute to the conversations of how educators can intentionally guide the engagement of learners through the ways in which the learning experience is designed and facilitated.

The articles in this issue also consider assessment of and for maker-based learning experiences from the perspective of reflection. (Rosenheck et al., this issue) explores the role of the artifacts generated from the maker experience as a means of both supporting reflection, as well as building students’ capacity for reflection. Along the same lines, Kim et al. (this issue) probe the use of student voice as a vehicle for reflection within the course of the maker experience. In both cases, reflective practice seeks to gain insights about what learners are thinking and making their thinking more visible.

Connecting to broader issues

The research presented in this issue reminds us of fundamental issues of assessment. While educators value different learning goals or different forms of engagement from their learners, it is important to know, for each learning opportunity, what is being assessed and what constitutes evidence of what is being assessed. This aligns with the view of assessment as a form of evidence-based argumentation (Gorin, 2012), and within maker-based learning experiences, pushes on educators’ instructional practice in unique ways. Evidence of learning is often embedded in the process and actions of making and collecting that evidence cannot disrupt the flow of that process.

In addition, who is assessing is another consideration. The articles in this issue draw attention that learners, in both formal and informal settings, are assessors as well as the educators. This is a shift from traditional thinking, since previous research on assessment in maker-based learning experiences have been focused on measurement being carried out by an external professional, like an evaluator or researcher (Lin et al., 2020). Assessment literacy for maker educators may include clarifying learning objectives or goals (Bergner et al., 2019; Wardrip et al., 2018) and aligning assessment tools to the needs of the educators (Cun et al., 2019). Similarly, positioning learners as the assessors, or at least having a role in the assessment process, requires further work for building their capacity to carry out the assessment. This represents an extension for student agency within maker-based learning environments that previous research has already advocated (Clapp et al., 2016).

Concluding remarks

Beyond these examples, more work can be done to consider other assessors. For example, what is the role of self-assessment (in addition to peer assessment) or experts and community members as assessors? Making might offer interesting opportunities to tap into a broad array of members of the community since hands-on creation and tinkering broadly encompasses different types, like traditional crafts-people, handy home fixers as well as computer scientists and engineers. This can further connect maker-based learning experiences to contexts and skills of professional settings. These considerations tie the value of making being to its workforce development outcomes and can also be integrated into the design of learning trajectories beyond K-12.

As assessments are implemented, we should also consider for whom the assessment is useful or relevant. The assessments may look different and generate different types of data if they are intended for learners instead of educators. Furthermore, are assessments useful for furthering instruction? Demonstrating one’s own skill or competence? Showing effectiveness of a particular program? These all are critical pursuits for designing and implementing an assessment, and future research should investigate how assessments shift and what capacities are necessary to facilitate those shifts. This is not isolated to maker-based educational experiences. However, these issues create opportunities for research and practice to gain understanding with respect to these questions.

Additionally, the diversity of what students make creates opportunities to challenge standardization in assessment. Making leads to an opportunity to broaden the palette of evidence for learners to demonstrate skills or competence. For example, addressing a design challenge might have a common goal, but the learners probably, and hopefully, will address the challenge in a variety of ways.

The pandemic brought into sharp relief the long-standing issues with claiming that standardized testing is necessary. If standardized testing is not sufficient to solve educational challenges, how might we actually evaluate and value what is learned? And how can standardized norms be re-evaluated to broaden our definitions of what learning is important and how students of all backgrounds can be supported to succeed?

More and more, the maker learning field is looking critically and carefully at issues of equity, by acknowledging and pushing for not only a broader understanding of what maker learning looks like, how it’s defined, how it’s labeled and who is involved in such definitional work. This involves ensuring that what counts for making is broadly defined. And this recognizes that expertise for making resides widely in our communities, both in professional as well as amateur makers. Plus, related assessment practices should be relevant to personal contexts and to a diversity of future learning paths and goals.

As research on maker-based learning experiences continues to grow, research focused on understanding learning and engagement is essential. The articles in this special issue aim to deepen the conversation in the field of maker-based learning around assessment. These articles also encourage future questions and investigation of diverse and meaningful assessment moving forward.