Exploring how teachers suggest informing designs for learning

Abstract

Purpose

An issue that has received limited attention is how teachers make informed decisions when designing learning activities, which makes effective use of resources and technologies. The aim of this paper is to explore how teachers suggest informing their designs for learning.

Design/methodology/approach

The study is based on documentation from workshops where adult education teachers were encouraged to suggest how a design for learning can draw on what is known, how the design can be evaluated and how the design and lessons learned can be shared. The data was analyzed using thematic analysis.

Findings

Most teachers did not explicitly suggest how to inform their designs for learning, although some suggested to draw on learning theories. The teachers were able to provide many examples of how their designs for learning could be evaluated based on student perceptions and assessment. They primarily suggested to discuss and share their designs and lessons learned at the school.

Practical implications

The workshop format could be adapted and used in practice.

Originality/value

Relatively little attention has been paid to how teachers inform their designs for learning. A key challenge for teachers is to understand design for learning as an informed practice, which could be integrated as part of their daily work.

Citation

Hrastinski, S. (2021), "Exploring how teachers suggest informing designs for learning", International Journal of Information and Learning Technology, Vol. 38 No. 4, pp. 381-392. https://doi.org/10.1108/IJILT-02-2021-0036

Introduction

Learning design aims to support teachers to make pedagogically informed decisions when designing learning activities, which makes effective use of resources and technologies (Conole, 2013). Another aim of learning design is to make the design process more explicit and shareable (Agostinho et al., 2011). Learning design approaches help teachers to shift from focusing on content to learning activities and student experiences (Conole, 2013). However, learning itself cannot be designed, we can only attempt to design for learning (Kali et al., 2011). Therefore, the term design for learning is preferred in this paper. Many models attempt to guide teachers to take advantage of digital technologies when designing for learning (Bower and Vlachopoulos, 2018).

An issue that has received limited attention is how to inform designs for learning (McKenney et al., 2015), although there are exceptions. For example, Michos and Hernández-Leo (2018) studied teacher communities that discussed designs for learning. They showed how data and visualizations can foster in-depth reflection. Teachers could be informed by different resources, such as research and best practices, by reflecting on how such resources could be applied in their teaching context (Oliver and Conole, 2003). Being informed is also about reflecting and acting on analyses and evaluations of designs for learning to improve student learning and the quality of teaching (Zwozdiak-Myers, 2018). Thus, informing designs for learning could be based on resources, but also reflections and evaluations of previous designs. Laurillard (2012) argues that teachers should use what is known and iteratively improve designs for learning, which lead us to questions, such as the following: How can teachers inform their designs for learning by drawing on what is known, such as research findings and best practices? How can teachers evaluate their designs for learning to improve the designs and their knowledge? How can teachers share their designs and what they learn?

This paper aims to explore how teachers suggest informing their designs for learning. Teachers are suggested to inform their design ideas based on what is currently known and continually improve their design ideas by conducting evaluations. Moreover, teachers are suggested to share their designs for learning and lessons learned to support the improvement of what communities of teachers can accomplish (Scardamalia and Bereiter, 1994). The study of this paper is based on workshops where adult education teachers were encouraged to suggest how their designs for learning could be informed and evaluated and how the designs and lessons learned could be shared. The adult education teachers were teaching courses corresponding to K12 education aimed at adult students that need to complement their previous studies, for example, to obtain an upper secondary diploma, to achieve eligibility requirements for university studies or Swedish for immigrants. The paper is guided by the following research questions:

RQ1.

How did the teachers suggest drawing on what is known when designing for learning?

RQ2.

How did the teachers suggest evaluating their designs for learning?

RQ3.

How did the teachers suggest sharing their designs for learning and lessons learned?

Theoretical background

In this section, a background is provided to drawing on what is known, evaluation of design for learning and sharing designs for learning and lessons learned, all of which could be considered to be key aspects when regarding design for learning as an informed practice.

Drawing on what is known

Designs for learning are often based on implicit assumptions and personal reflections (Bates, 2015; Oliver et al., 2017; Price and Kirkwood, 2014). Teachers are faced with many complex variables and need to make decisions on how to design and teach a course to the best of their knowledge and based on “what is known” (Bates, 2015). Different types of resources could inform design for learning, which are discussed below. These are learning theory, research findings, best practices and teacher knowledge.

Learning theory. There are many different theories of learning, which at first might seem conflicting but might be complementary (Sfard, 1998). Many learning design models consider learning theory in some way. While some learning design models are flexible (Young and Perović, 2016), others are underlined by certain learning theories, typically socioconstructivist theories (Bower and Vlachopoulos, 2018). It can be noted that many learning design models consider learning theory in some way. While some learning design models are flexible (Young and Perović, 2016), others are underlined by certain learning theories, typically socioconstructivist theories (Bower and Vlachopoulos, 2018).

Research findings. It has been argued that teaching should be a research-based profession (Hargreaves, 2007). More realistically, it can be claimed that research should be regarded as one factor that could inform teachers' work (Biesta, 2007). Encouraging teachers to find research on relevant issues is sensible if they are encouraged to use it critically and thoughtfully and by reflecting on how it can be used to inform their design work (Oliver and Conole, 2003). Although models that aim to connect teaching and research have been proposed over the years (Atkinson, 2001), it is unusual that designs for learning are based on research (Price and Kirkwood, 2014). Many teachers need support in curating and tailoring research to increase their use of research (Joram et al., 2020).

Best practices. The term best practice suggests that there is a best approach that is the optimal solution to complex problems (Coffield and Edward, 2009). Hargreaves (2004) argues that “‘good practice’ and ‘best practice’ are [often] treated as synonyms, although clearly ‘best practice’ suggests a practice that has been compared with others and has proved itself better than other ‘good’ practices” (p. 72). It might be worthwhile sharing teaching experiences and designs even though they have not been found “better” than other practices. What might be preferable or “best” is often a judgment that needs to be made by teachers in ways that are relevant in their contextualized settings (Biesta, 2007).

Teacher knowledge. Teacher knowledge is based on reflective thinking, described as “active, persistent and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it and the further conclusion to which it tends” (Dewey, 1933, p. 9). Shulman (1986) introduced the idea of pedagogical content knowledge (PCK). Instead of focusing on subject matter knowledge or pedagogy, he suggested that it is important to consider the relationship between the two. PCK is about “the ways of representing and formulating the subject that make it comprehensible to others” (Shulman, 1986, p. 9). In their extension of PCK, Mishra and Koehler (2006) argue that teachers also need technological knowledge, which in turn interacts with pedagogical and content knowledge. Based on the fact that teacher knowledge is complex and situated, it is argued that teachers need to adapt what they do to different students and contexts (Wilson et al., 1987).

In Figure 1, different types of resources that could inform designs for learning are summarized. These resources range from being more personal and contextual, which are typically very relevant to the individual teacher, to more collective and abstract, which are of interest to many but need to be adapted critically and thoughtfully (Oliver and Conole, 2003).

Evaluation of design for learning

Evaluation can simply be defined as “determining the effectiveness.” This has little meaning until we answer the question: “In terms of what?” (Kirkpatrick and Craig, 1967). Course evaluations are often teaching and teacher-focused. They have a fire alarm function rather than focusing on course development (Edström, 2008). The complex nature of digital education calls for using multiple methods and sources of data to understand student learning, satisfaction and participation. Quantitative analyses can yield useful information on learning outcomes and related variables. Qualitative data can help us better understand observed differences and student perceptions (Gunawardena et al., 2000).

Bennett et al. (2017) investigated teachers' design work and found that more than half of the interviewed teachers described that they identified future changes, typically after the course was completed, but some also made notes of their ideas during the course. These reflective activities contributed to developing teacher knowledge and iteratively refining their courses over multiple offerings. Thus, we can distinguish between two types of evaluation: formative evaluation while teachers teach and summative evaluation after they have taught. Hrastinski (2021) explored how 31 blended and online courses were evaluated by experienced practitioners as part of an edited book. They used course evaluations, questionnaires as a complement to standardized course evaluations, data from digital platforms, assessment results, focus groups, written reflections from students and class discussions with students. These teachers can be assumed to be early adopters of technology (Rogers, 2010). A question that needs to be further explored is how we can engage, not only the early adopters and those with a special interest, but most time-pressed teachers in iterative evaluation and improvement of their designs for learning.

Sharing designs for learning and lessons learned

One of the aims of design for learning is to share the designs (Conole and Wills, 2013). There is also a long tradition to develop digital technologies to support codesign and collaboration between teachers (Hernández-Leo et al., 2014). Different types of structured and semistructured representations, such as pedagogical patterns (Bergin et al., 2012), have been suggested as ways to share designs for learning, although these have not been adopted by most teachers (Oliver et al., 2017). Less focus has been put on how to share lessons learned (Conole and Wills, 2013; Laurillard, 2012). Teachers typically enjoy sharing experiences with colleagues, especially within their subject discipline (Conole and Wills, 2013; Oliver et al., 2017). Prestridge (2019) has developed a categorization of teachers' use of social mediated spaces for their professional learning, which indicates different ways teachers could share design ideas and lessons learned in digital settings. The categorization includes four categories: info-seekers that seek to find new knowledge or ideas and resources from social mediated spaces; info-networkers that also share the ideas and resources with colleagues; self-seeking contributors that share knowledge, ideas or curriculum materials to get feedback, ask specific questions or acquire additional information; and vocationalists that engage as a member of a community of learning that aims to contribute with new knowledge. Collectively sharing and improving ideas is an important dimension of situated learning and knowledge development through awareness and reflection (van der Linden and McKenney, 2020; Scardamalia and Bereiter, 2003).

Method

Research setting

The empirical data is based on workshops with adult education teachers. The purpose of the workshop for each participant was to suggest a design for learning, how the design drew on what is known, how to evaluate the design and how to share the design and lessons learned. The workshops were facilitated by the author as part of a teacher conference on digital learning in a Swedish region. The introduction to the workshop was intentionally short in order not to bias the results. The workshop instructions are presented in Table 1. The process of the workshops was as follows: (1) introduction by the facilitator, (2) individual work, (3) pairs of teachers provided feedback to each other, (4) individual work and (5) brief presentations of some of the designs for learning. All designs for learning were documented by using a digital form. Each participant individually submitted answers according to the themes/question outlined in Table 1.

Two workshops were conducted for 60 min with nine participants in each workshop (see Table 2). There were 11 females and seven males. The years of teaching experience ranged from 1.5 to 40 years (M = 17, SD = 10). The teachers represented many different subject disciplines. Two of the teachers (F8a and F8b) worked together in the workshop. The study complies with research ethics rules and guidelines (Swedish Research Council, 2017). It was voluntary to participate in this study and the data has been anonymized. All teachers who completed the digital form chose to be part of the study.

Data collection and analysis

The documentation from the workshops (i.e. the text submitted by using a digital form) was analyzed using thematic analysis, which can be described as a method for identifying, analyzing and reporting themes within data (Braun and Clarke, 2006). The process of coding is an essential part of thematic analysis (Miles and Huberman, 1994), where data are organized into meaningful themes (Braun and Clarke, 2006). All the texts from the digital forms were anonymized and exported to text format. The data were imported and analyzed by using the open-source software RQDA that is used for qualitative data analysis (Huang, 2014). A semantic approach was adopted since the documentation from the workshops was in written form. This means that the explicit meanings of the data (i.e. what the participants had written) were analyzed (Braun and Clarke, 2006).

The themes were deductive and based on the research questions. The subthemes were identified inductively. Braun and Clarke (2006) provide guidelines for doing thematic analysis. Qualitative analysis guidelines are not rules and should be applied flexibly to fit the research questions and data (Patton, 1990). In Table 3, the phases of the thematic analysis are described. The description of the process has been adapted from Braun and Clarke (2006). Qualitative analysis is a recursive process where movement between the phases is typically necessary, which was also the case in this study (Braun and Clarke, 2006).

Results

The themes and subthemes, which are described in more detail below, are presented in Figure 2.

Drawing on what is known

There were few examples where the teachers explicitly described how they informed their design, despite being encouraged to do so. Only one subtheme was identified: learning theory. In the learning theory theme, the teachers suggested using different theoretical perspectives.

This can be linked to how repetition is important for learning. Behaviorism - repetition. (M5)

The circle model [genre pedagogy], joint reading and processing of central parts on smartboards. (F10)

Given the socio-cultural perspective, it is important that students get the chance to create knowledge together. This can lead to different views. (F6)

Vygotsky maintains that students need inspiration, motivation and encouragement in their learning. (F8)

Quick feedback is known to be important for learning, feasible with digital tools. (F8)

There was one exception, a teacher explicitly acknowledging teacher experience.

My experience is that students also have to learn how to produce mathematics (writing on a paper) to gain knowledge. (F6)

Evaluation of design for learning

The teachers suggested different ways to evaluate their designs. Two subthemes were identified: student perceptions and assessment. Three teachers suggested conducting a discussion with the students.

[T]he group evaluates orally: How did it feel? What was positive? What could have been done differently? (M5)

Three teachers suggested designing a survey.

Survey… With questions about how the student experienced the way of working and what they have learned. (M7)

[S]urvey on the different parts (recorded lessons, test questions and collaboration tasks). (M11)

Two teachers proposed using a standardized course evaluation survey. One of these teachers suggested complementing the survey with open questions that the students were to complete in writing. Another teacher exemplified what kind of open questions to use.

Finally, the student evaluates in writing with questions of the type: How does it feel now? Would you be able to give a speech again? Would you be able to give a speech outside the school?” (M5)

Two teachers suggested conducting an evaluation together with the students by asking specific questions using the smartboard or a smartphone app.

Quick evaluation on the smartboard by the students. Did you learn imperatives? Can you use it? (F9)

One teacher suggested conducting individual informal discussions with students during the course and a more formal discussion on the course and their learning after the course.

The second subtheme was how to use assessment to evaluate the design for learning. Two teachers suggested analyzing the quality of student texts. Two teachers suggested following student development over time to see if their abilities improved. For example, in one course on study skills, it was suggested to “check that the students de facto find information faster” (F13). Two teachers suggested using automated quizzes to give the students and teacher fast feedback. One teacher suggested conducting a pretest–posttest where “students answer a new quiz with a similar design but new questions” (F6) to investigate if student results improved following the intervention. Similarly, one teacher suggested comparing student results with previous years. One teacher wanted to conduct informal individual discussions with students throughout the course and a more formal discussion after the course with a focus on the course design and student learning.

[D]iscussion in conjunction with feedback on examination, as this is the first part of the course, so you can adjust the upcoming and a more thorough evaluation after the course has been completed. (M17)

Sharing designs for learning and lessons learned

The teachers suggested different ways to share their designs for learning and lessons learned. Two subthemes were identified: collegial learning and digital sharing of designs for learning. Six teachers suggested different ways to share experiences at the school.

Pedagogical meetings among colleagues. (M7)

Teachers in different subjects at the school can be involved. (F15)

Me and the other responsible teacher will probably show other colleagues who are interested in a similar distance structure. (M17)

Four teachers proposed sharing experiences in their teacher team: “conversations at the subject meeting at work” (F4) and “[w]e share within the working group” (F8). As evident above, most suggestions in the theme collegial learning concerned experiences at the school. However, two teachers suggested sharing experiences in regional networks: “collaboration between municipalities within established networks” (F15).

The second subtheme was digital sharing of designs for learning. Five teachers provided examples of how to share their design with their colleagues by sharing files in shared folders and learning management systems at the school. The following is an example of a more systematic approach.

The idea is that I will build together and try out a material that will later be handed over to other [Swedish for immigrants] colleagues so that they have something to start from and be able to work with in future classes. (F13)

Two teachers suggested to share the design on a blog and one teacher suggested YouTube.

Discussion

The first research question investigated how teachers suggested drawing on what is known when designing for learning. There were some examples where teachers were inspired by different learning theories: behaviorism, genre pedagogy and sociocultural learning theory. This finding can be compared with current learning design models that sometimes are underpinned by learning theories, typically socioconstructivist theories (Bower and Vlachopoulos, 2018). Most teachers who used learning theory as a foundation gave exclusivity to one learning theory, which might serve the interest of certain students but might disadvantage others (Sfard, 1998). Here, learning design models, such as the ABC model, which encourage basing designs on different, complementary “learning types” (Young and Perović, 2016) might be useful to encourage teachers to consider alternative or complementary approaches. In line with previous research, the teachers did not draw on research findings. Teachers might need support in curating and tailoring research to increase their use of research (Joram et al., 2020). Thus, it seems necessary to support teachers in how to identify research findings that could inform their designs for learning. There was only one example where teacher experience was explicitly recognized.

These findings can be compared with Agostinho et al. (2018) that identified the types of support the teachers reported using in their design work: colleagues, literature, workshops and seminars, conferences, institutional support services and enrolment in postgraduate study. It is notable that few teachers explicitly described how they used resources to inform their design, despite being encouraged to do so. This finding is aligned with previous research, which found that teachers followed a top-down approach where they initially considered the learning outcomes, scope of content, ideas for activities and assessment strategy (Bennett et al., 2017), but typically do not draw on external sources of knowledge, such as research findings (Price and Kirkwood, 2014).

The second research question investigated how teachers suggested evaluating their designs for learning. We can distinguish between two types of evaluation: formative evaluation while teachers teach and summative evaluation after they have taught. The teachers provided many different examples of how to evaluate their designs, by drawing on student perceptions and assessment. They suggested investigating student perceptions by discussions with students, in some cases complemented by using a smartboard or app to ask specific questions as a basis for the discussion, surveys, complementing standardized course evaluation surveys with open questions and individual discussions with students during and after the course. The teachers proposed drawing on assessment in different ways: analyzing the quality of student texts, following student development over time, automated quizzes, pretest–posttest of student results and comparing student results with previous years. These findings can be compared with a study that explored how university teachers evaluated 31 blended and online courses. They used course evaluations, questionnaires, data from digital platforms, assessment results, focus groups, written reflections from students and class discussions with students (Hrastinski, 2021). Thus, there are many different methods that teachers can use when evaluating designs for learning. The complex nature of evaluating digital education calls for using multiple methods and sources of data to understand student learning, satisfaction and participation (Gunawardena et al., 2000). There seems to be a need to develop a more thorough understanding of the benefits and limitations of using the identified evaluation methods.

The third research question investigated how teachers suggested sharing their designs for learning and lessons learned. They proposed drawing on collegial learning to share and discuss experiences, mainly at the school and in teacher teams. The teachers also suggested sharing designs digitally, primarily with colleagues at the school, but there were also occasional suggestions to share designs by using blogs and YouTube. Thus, the teachers seemed willing to move beyond information consumption toward sharing designs for learning and lessons learned (Prestridge, 2019), but preferably at their school. Although researchers envision different ways that teachers could share their designs and lessons learned, ranging from design representations to journal papers (Bergin et al., 2012; Bernstein and Bass, 2005), the findings of this paper are aligned with research suggesting that teachers typically want to discuss their ideas, experiences and designs with colleagues, preferably from their subject discipline (Conole and Wills, 2013; Oliver et al., 2017). In light of these findings, it might be more realistic to accept that most teachers prefer to share designs for learning and lessons learned at their school. If a school encourages at least some teachers to take on more outbound roles, the school will have an influx of ideas, designs for learning and lessons learned, which could be adapted by colleagues to their local context.

Limitations and further research

A limitation of this study is that it is based on workshops with teachers. The teachers suggested a design for learning, how to draw on what is known, how to evaluate the design and how to share the design and lessons learned. What the teachers plan to do will not necessarily correspond with what they do in practice. It would be valuable to explore similar research questions in “real” design and teaching practice. That said, it is also important that research explores teacher ideas and suggestions, which can guide future research and practice. It also needs to be acknowledged that this study was based on a specific type of adult education corresponding to K12 education aimed at adult students that need to complement their previous studies, for example, to obtain an upper secondary diploma, to achieve eligibility requirements for university studies or Swedish for immigrants.

This paper and related studies have identified different ways that teachers draw on what is known to inform their designs for learning (Agostinho et al., 2018) and how they evaluate their designs for learning (Hrastinski, 2021). It was surprising that other types of resources, such as best practices and experiences of colleagues were not acknowledged. Previous research has found that teachers enjoy sharing experiences with other teachers (Conole and Wills, 2013), but a question that remains is to what extent and how such experiences are then used to inform teachers' design work. There are opportunities to develop more systematic approaches to how designs for learning could be informed and evaluated. One such example is how learning analytics could be used to inform designs for learning (Michos and Hernández-Leo, 2020; Wiley et al., 2020). However, as found in this paper and related research (Agostinho et al., 2018; Hrastinski, 2021), there are many other resources and methods for evaluation that could support informing designs for learning, which also needs to be further developed and investigated in future research.

Conclusions

This paper explored how adult teachers suggested informing and evaluating their designs for learning and how they suggested sharing these designs and lessons learned, all of which could be considered to be key elements of informed design for learning. It was found that teachers typically referred to learning theories to inform their designs for learning. However, other types of resources, such as research findings, best practices and experiences of colleagues, were not explicitly suggested. Teachers seem to need support to identify research and to understand how research findings are relevant in their setting (Joram et al., 2020). The teachers provided many examples of how to evaluate their design based on student perceptions and assessment. They suggested discussing their experiences with colleagues and sharing their designs digitally, although most preferred doing this locally at the school.

In conclusion, the teachers were aware of how their designs for learning could be evaluated and how to share the designs and lessons learned at the school. Teachers seem to need support to understand how to inform their designs. They could also be encouraged to share their designs and lessons learned beyond their school when it makes sense. A key challenge for future research is to put more focus on design for learning as an informed practice, to support the development of teacher knowledge and the improvement of future designs for learning.

Figures

Figure 1

Examples of how designs for learning could be informed

Figure 2

Subthemes organized according to research questions

Table 1

Workshop instructions

Background	Describe the module, e.g. name, purpose, level, scope/credits, approximate number of participants
Design	Describe and motivate how the module will be designed Why and how should digital technologies be used?*
Evaluate	How should the module be evaluated?*
Share	How can lessons learned be shared with colleagues?* Locally, nationally, internationally?

Note(s): *Motivate based on resources and experience. Examples of resources and experience: course evaluations, research papers, books, your experience, experience of colleagues, blogs and social media and best practices

Table 2

Participants of the workshops

Teacher	Workshop	Gender	Teaching experience (years)	Subject discipline
M1	1	Male	30	Philosophy
F2	1	Female	10	Health
M3	1	Male	17	Mathematics
F4	1	Female	17	Literature
M5	1	Male	1.5	Mathematics
F6	1	Female	3	Psychiatry
M7	1	Male	29	English
F8a	1	Female	2	Health
F8b	1	Female	20	Health
F9	2	Female	13	History
F10	2	Female	9	Swedish for immigrants
M11	2	Male	9	Swedish
M12	2	Male	20	Biology
F13	2	Female	13	Study skills
F14	2	Female	40	Chemistry
F15	2	Female	21	Swedish
F16	2	Female	30	Health
M17	2	Male	13	Digital skills

Table 3

Phases of the thematic analysis

Phase	Description of the process
Familiarizing yourself with the data	Data were read and reread, and initial ideas were noted
Generating initial codes	Initial codes were generated. Data relevant to each code were collated
Collating codes into deductive themes	The codes were collated into deductive themes based on the research questions
Inductively searching for subthemes	The subthemes were inductively identified
Reviewing themes	It was checked if the coded extracts work in relation to the themes, subthemes and the entire data set
Defining and naming themes	Names and definitions of each sub theme were generated
Producing the report	Selection of extracts to quote and production of the paper

Source(s): Adapted from Braun and Clarke (2006), p. 87