The role of qualitative research in Change Laboratory interventions

Abstract

Purpose

This contribution investigates methodological questions concerning Change Laboratory interventions. It contemplates the research questions: To what extent the Change Laboratory can be situated within the panorama of qualitative inquiry? If so, to what extent can the methods and strategies of inquiry help improve a Change Laboratory intervention?

Design/methodology/approach

To answer the first question, this paper makes an overview on key terms of qualitative research; subsequently, it presents the characterising features of the Change Laboratory. Then, it takes a historical perspective and compares the Change Laboratory firstly against action research, and secondly with design experiments. To answer the second section, it examines a case study of Change Laboratory with teachers that the first author facilitated. Next, it displays how trustworthiness was ensured through a thick description and member checks.

Findings

The paper argues that the Change Laboratory is a strategy of inquiry; it aligns with the characteristics of qualitative research, and it follows the agenda of a participative paradigm. Furthermore, the methods and strategies of inquiry such as thick descriptions and member checks, not only can improve rigour and validity of the intervention but also strengthen the outcomes of the Change Laboratory itself.

Originality/value

The Change Laboratory is well defined as a formative method, but not fully understood as an investigative method. Although scholars discussed methodological issues of Cultural Historical Activity Theory in diverse articles, the relationship between the Change Laboratory and qualitative inquiry has remained unclear.

Citation

Morselli, D. and Marcelli, A.M. (2022), "The role of qualitative research in Change Laboratory interventions", Journal of Workplace Learning, Vol. 34 No. 2, pp. 215-228. https://doi.org/10.1108/JWL-08-2020-0140

1. Introduction

In the educational sciences there is an increasing interest on Cultural Historical Activity Theory (CHAT) (Sannino and Sutter, 2011). In recent years, CHAT scholars developed formative interventions such as the Fifth Dimension, the Clinic of Activity, as well as the Change Laboratory (Sannino, 2011) to bolster change in work and educational settings. The Change Laboratory, for example, is a “method for developing work practices by the practitioners. It facilitates both intensive, deep transformations and continuous incremental improvement” (Engeström, Virkkunen, Helle, Pihlaja and Poikela, 1996, p. 11). Although scholars recently discussed methodological issues of CHAT in diverse articles (Sannino, 2011; Engeström, Sannino, and Virkkunen, 2014; Gutiérrez, Engeström, and Sannino, 2016; Laitinen, Sannino, and Engeström, 2016; Sannino, Engeström and Lemos, 2016), the relationship between the Change Laboratory and qualitative inquiry remains unclear.

While detailed instructions are provided in the shape of “argumentative grammar” (Engeström, 2011, p. 607), the issue remains of how the Change Laboratory can qualify as a class (or type) of research methodology against the wider background of educational research. The issue of methodological placement is acknowledged for the first time by Postholm (2015, p. 46), who claims: “Relatively little research has been done on methodology within the CHAT framework”. Postholm reviews a series of methodological articles on behalf of CHAT-oriented scholars, but finds them insufficient: at best, there is emphasis on the role of discursive practices in making contradictions emerge and in gathering participants around a common goal. Even recent applications on Change Laboratory interventions suspend their judgement concerning the methodological intricacies of CHAT.

Consequently, this contribution will answer to two explorative questions:

Q1.

To what extent can the Change Laboratory be situated within the panorama of qualitative inquiry?

Q2.

If so, to what extent can inquiry help improve a Change Laboratory intervention?

To answer the first question, this paper starts by providing an overview of key terms in qualitative research; subsequently it presents the characterising features of the Change Laboratory, action research and design experiments. Then it compares the Change Laboratory first against action research, and secondly with design experiments. The aim is making systematic comparisons to find commonalities. To answer the second section, this article shows as case study Change Laboratory with teachers that the first author facilitated. Next, it displays how validity was ensured through thick description and member checks.

2. Locating the Change Laboratory within the panorama of qualitative research

As stated in the introduction, the relationship between qualitative research and CHAT has so far remained unclear. However, the basic methodology to carry out field research used by CHAT scholars, that is the Change Laboratory, has been compared by its scholars to two methodologies commonly used by qualitative researchers: action research and design experiments. Hence, this paper tries to infer the positioning of the Change Laboratory within the panorama of qualitative research by using action research and design experiments as liaison. This section firstly defines key terms of qualitative research that are essential to understand the qualitative research process like “research paradigms and perspectives” and “strategies of inquiry” that will be used later in the discussion; subsequently it presents action research and design experiments as strategies of inquiry used by qualitative researchers.

2.1 An overview of key terms pertaining qualitative research

Qualitative research emerged as a critique and as alternative paradigm to positivism (Ravitch and Carl, 2015). Positivism believes that the world consists of two levels, and while the appearance of events and social phenomena is continuously changing, there are universal laws expressing unchanging order, that must be observed systematically by an objective and neutral knower to be understood and explained in terms of cause-and-effect relationships (Hughes, 2010); knowledge can be proved to be true or false, and experiments rigorously conducted in controlled circumstances and that are replicable can find these universal laws. Qualitative research is distinct type of research that crosses disciplines, fields and topics, with an interconnected and surrounding group of themes, assumptions and concepts, including the associated traditions (Denzin and Lincoln, 2018). Although any explanation must be situated in such varied panorama, a general definition is:

Qualitative research is a situated activity that locates the observer in the world. Qualitative research consists of a set of interpretive, material practices that make the world visible. These practices transform the world. They turn the world into a series of representations […]. At this level, qualitative research involves an interpretive, naturalist approach to the world. This means that qualitative researchers study things in their setting, attempting to make sense or interpret phenomena in terms of the meanings people bring to them. (p. 10)

While quantitative research wonders about how many instances of a certain type are present, qualitative inquiry inspects the factors (material or symbolic) to which people orient themselves in their contexts of daily life (Erickson, 2018). In such broad panorama, the term inquiry is better positioned, as it has minor positivistic implications than research, and better aligns with the open and indeterminate ends, the ambiguity, the praxis, the pedagogies of freedom, of liberation and resistance typical of qualitative research (Dimitriadis, 2016).

Qualitative inquiry is structured by paradigms taken as a basic ensemble of beliefs that orient action (Lincoln, Lynham, and Guba, 2018, p. 214). Such beliefs are inspired by ontology (led by questions such as “What is the essence of human being? What is the nature of reality?”), from epistemology (“What is the nature of knowledge and what are the sources of knowledge?”) and from methodology (“How does human being knows the world or can acquire knowledge of it?”). These assumptions model the way researchers look at and operate in the world. As a matter of fact, all qualitative research is interpretive, since led by a series of beliefs concerning the world and how it can be studied. At present five leading paradigms – albeit continuously evolving – structure qualitative inquiry: positivist and post-positivist; critical and feminist; constructivist-interpretivist; and participative-postmodern-poststructural (Denzin and Lincoln, 2018). Participative paradigms, for example, rest on a co-created reality and put the primacy on practical knowing; the aim of research is transformation based on democratic participation between the subject(s) and researcher. Participants are hence seen as co-researchers, they are initiated to the research process, and learn actively through the direct involvement in the process; the control is shared between researcher and subjects. Besides these leading paradigms, Lincoln et al. (2018) identify several perspectives that are less unified and solid than a paradigm, yet they share a set of methodological assumptions or a specific epistemology, of which some examples are Marxism, indigenous methodologies, decolonising research, humanism and critical pedagogy.

An important milestone of research is the selection of an appropriate strategy of inquiry. This describes a set of capabilities, practices and assumptions that the researcher can deploy as s/he moves from the paradigm or perspective to the “real” world. Strategies of inquiry “put the paradigm of interpretation into motion. Strategies of inquiry, at the same time, also connect the researcher to specific methods to collect and analyse empirical materials” (Denzin and Lincoln, 2018, p. 59). Action research and design experiments are often deployed in qualitative inquiry. Concerning the former, “in the literature, the term ‘action research’ covers a diverse range of approaches to enquiry, always linked in some way to changing a social practice.” (Kemmis, McTaggart, and Nixon, 2013, p. Regarding the latter, the aim of design experiments is both to refine practice and answer theoretical questions to inspect how learning environments influence the variables in teaching and learning:

Design experiments were developed as a way to carry out formative research to test and refine educational designs based on theoretical principles derived from prior research. This approach of progressive refinement in design involves putting a first version of a design into the world to see how it works. (Collins et al., 2004, p. 18)

Design experiments can be considered however, as mixed methods research, since they “mix and match qualitative and quantitative methodologies in order to describe the phenomena” (Brown, 1992, in Collins et al., 2004, p. 28). It is due to this consideration that the next section discusses the most widespread research strategy within CHAT, the Change Laboratory.

2.2 The Change Laboratory

Throughout its history CHAT has shown to be an activist theory to burgeon novel practices. This is traceable to the Marxist idea of revolutionary practice, which emphasises that theory is useful to explain and analyse the world around us, as well as to promote change and generate new practices (Sannino, 2011). The Change Laboratory comes from Developmental Work research, the first research strategy developed from the ‘80s within CHAT to carry out research and develop collective work practices (Virkkunen and Newnham, 2013). The Change Laboratory is an “intervention toolkit”, a microcosmos whereby potentially innovative working modalities can be experimented (Engeström and Sannino, 2010, p. 15). The basic idea consists in organising a space preferably within the organisation (Engeström et al., 1996), and endowing it with a broad set of tools and concepts that allow the practitioners (generally from 15 to 20 over 8-10 weekly sessions plus follow up) to analyse work practices, and subsequently design and experiment with new tools and concepts.

The main principles of analysis are (Engeström, 2001; Engeström, 2011) as follows: the unit is represented by collective activity systems that are oriented to the object and mediated by artefacts (tools and concepts); an activity system takes shape and is transformed over long periods, and an historical perspective is key to understand its potentials and challenges; contradictions are structural tensions between and within systems of activity that accumulated historically, they are employed during the analysis as trigger for development and change; during the intervention the participants develop their agency; innovation and transformation of practices are seen as an expansive formation of concepts.

Among the tools used the sessions there is the triangle of human activity (Engeström, 2015, p. 63) and the cycle of expansive learning (Engeström, 1999, p. 384) which certainly constitute an epistemological advancement on top of the ontological picture. This is an epistemological double entendre: the triangle, for example, provides the inquirer with a viable investigative target, that is, the idea that activity systems constitute unit[s] of analysis; Second, the triangle is itself a meta-epistemological stance, inasmuch it describes how things stand in relation to each other, and which types of mutual acquaintances they can develop. The epistemological soundness of CHAT enables investigators to derive methodological instructions from it. Methodology is “[what] defines how one goes about studying any phenomenon,” and results from “ontological and epistemological” assumptions about the (social) world (Sławecki, 2018, pp. 14–15).

2.3 A comparison between the Change Laboratory and other research strategies

While the Change Laboratory has never been labelled by its designers as qualitative or qualitative type of research, its designers have compared it with other strategies of inquiry, namely action research and the design experiments described above. At first Engeström et al. (1996) argues that an earlier similar approach is action research; the two research strategies, however, diverge in the contents, in the concepts used, in the notion of change, and in the distance from practice of what they analyse. While action research is concerned with group dynamics, the Change Laboratory deals with present, past and future characteristics of the work activity. Regarding the main concepts, action research uses the laws of group behaviour, whereas the Change Laboratory makes use of the triangular model of human activity and the cycle of expansive learning (Engeström, 2015). Concerning the notion of change, in action research positive loops result in positive interaction, whereas in the Change Laboratory resolving a systemic contraction brings a novel model of activity (Engeström et al., 1996). Fourth, action research does not discuss the content of practices, whereas the practitioners in the Change Laboratory observe and transform their practices on site.

Fifteen years later, Engeström (2011) argued that the Change Laboratory is a formative intervention, and he contrasts formative interventions to ‘linear interventions’ exemplified by design experiments. Design experiments inform a linear view of interventions that characterise the Gold Standard of educational research, and while the Gold Standard promotes interventionist research, it points also out the importance of randomised controlled trials and the careful selection of control groups, thus putting emphasis on quantification. Linear interventions assume that the researcher already knows what he or she would like to implement and what type of change bring about, thus defining in advance the intervention and the expected outcomes. In so doing, they rule out important phenomena such as power relationships, resistance, and agency. At the same time, the need for generalisability implies wide statistical samples and different sites of research, thereby indicating how it could be recognised as a positivist paradigm of research. There are, hence, four important differences between formative interventions and linear interventions, in the outset, in the process, in the outcomes, in the researcher’s role. During formative interventions, the participants face a contradictory object nested in their work activity. They progressively analyse and expand their activity by devising a new concept, therefore the contents are unknown to the researchers at the beginning like in linear interventions. Second, formative interventions are negotiated with the participants, who eventually are expected to lead the change process. By way of contrast, in linear interventions the participants are expected to execute the research design. Third, while linear interventions aim to create a standardised solution, formative interventions aim to design an innovative concept. Eventually, while in linear interventions the researcher controls the variables, in formative interventions he or she sustains a change effort guided and owned by the participants.

2.4 Connecting Change Laboratory with qualitative inquiry

As anticipated in the introduction, to locate the Change Laboratory in the panorama of research, this section will proceed to make systematic comparisons. First it will compare Change Laboratory with other strategies of inquiry, then it will compare the Change Laboratory with the definitions of qualitative inquiry. Finally, it will hypothesise which research paradigm and perspective the Change Laboratory belongs to.

2.4.1 The Change Laboratory is a strategy of inquiry.

Concerning research strategies, action research is a strategy typical of qualitative research (Kemmis et al., 2013); the comparison between Change Laboratory and action research made by Engeström et al. (1996) does not point out differences that place the Change Laboratory outside qualitative inquiry such an emphasis on quantity, replicability, and generalisation of the outcomes. Moreover, Engeström (2011) criticises design experiments for being too concerned with standardisation, randomised controlled trials, wide statistical samples, control of the variables, and for informing a positivist research paradigm, which are all elements characterising quantitative research. By way of contrast, the Change Laboratory is concerned with power relationships, resistance, agency, and the participants taking the lead of the intervention, which represent themes typical of qualitative research. Additionally, Engeström suggests that in the Change Laboratory it is not possible to know at the beginning what the outcome of the intervention will be, which means that the research design is emergent, and this represents another feature of qualitative research (Ravitch and Carl, 2015). Qualitative inquiry develops concepts and hypotheses during the research and while collecting data, rather than formulating hypotheses at the beginning and setting up well defined concepts (Flick, 2018a, 2018b). These qualities are shared by the Change Laboratory, as it is aimed to develop innovative concepts rather than generate a standard solution (Engeström, 2011). In line with these arguments, Yanchar (2011) sees the Change Laboratory as an “explicitly interpretive, contextual inquiry” (p. 179), while Eri (2013) defines the Change Laboratory a “qualitative intervention methodology” (p. 2459).

2.4.2 The Change Laboratory is in line with the aims of qualitative research.

Concerning the definition of qualitative inquiry, for Denzin and Lincoln (2018) the most characterising feature of inquiry is to be intrinsically interpretive. So is the Change Laboratory, as key terms like object of the activity, contradiction, expansive learning and activity system can only be inferred through the lenses of the theory. The second commonality is the setting. Inquiry approaches “the world(s) ‘out there’, instead of doing studies in specialised research settings such as laboratories” (Flick, 2018a, 2018b, p. 5); In the Change Laboratory, the sessions are organised on the shopfloor (Engeström et al., 1996). Third, qualitative inquiry accesses “experiences, interactions and documents in their natural context and in a way that gives room to the particularities of them and the materials in which they are studied” (Flick, 2018a, 2018b, p. 5). The same accounts for the Change Laboratory, since it collects and analyses the workplace specific practices and the practitioners’ experience (Virkkunen and Newnham, 2013).

Fourth, interpretive traditions of qualitative inquiry commit to criticise the positivist and post-positivist research program (Denzin and Lincoln, 2018, p. 40), and so does Engeström (2011) when he contrasts the Change Laboratory to positivist Gold Standard of educational research. The fifth communality is that qualitative inquiry and the Change Laboratory share a pragmatist view of inquiry. For Denzin and Lincoln (2018, p. 18) pragmatism is central for qualitative inquiry as theoretical and philosophical concern. Similarly, from a CHAT perspective, Gutiérrez et al. (2016, p. 277) suggest that participatory design research (including formative interventions) has a pragmatic conceptualisation, since it “identifies pressing problems of practice with participating community members”. In a similar fashion Virkkunen and Newnham (2013, p. 108) explain that during the Change Laboratory sessions it is key “breaking the participants’ abstract generalisations and myths and helping them to see the object of their activity more historically and concretely in its broader context”. The sixth communality regards the agenda. Gutiérrez et al. (2016) argue that formative interventions as form of participatory research design will be interested in equity and diversity and will therefore employ decolonising approaches to study what it means to carry out research in non-dominant communities; These however, are topics typical of inquiry. Similarly, while drafting the fourth generation of CHAT studies targeting objects of human activity such as poverty, social change and pandemics through heterogeneous coalitions (Engeström and Sannino, 2020), Sannino (2020, p. 1) calls for “an agenda aimed at disclosing how the utopia of eradicating homelessness is being enacted,” which aligns well with the pedagogies of freedom, liberation and resistance that for Dimitriadis (2016) characterise qualitative research.

2.4.3 The Change Laboratory follows a participative research paradigm.

After having suggested that the Change Laboratory is a strategy of inquiry, this section tries to identify qualitative paradigms and perspectives leading the Change Laboratory. Concerning paradigms and perspectives characterising inquiry, CHAT can be considered a distinctive perspective. Since its first generation of thinkers during the Russian Revolution, a period of strong turmoil, it has been oriented by Marxism (Sannino and Sutter, 2011), which is characterised by materialism and dialectics, and its main roots can be found in Vygotsky, Leont’ev and other theorists like Il’enkov, Davydov, Bakhtin and Bateson (Engeström and Sannino, 2010). Furthermore, while CHAT is a perspective on its own, it is possible to assert that a participative paradigm characterises formative interventions (including the Change Laboratory). Scaratti et al (2017) contend that strong partnerships between the researcher, the practitioners and the relevant stakeholders are key to create the type of knowledge to generate relevant social value. Gutiérrez et al. (2016) argue that formative interventions are a promising form of participatory design research (PDR):

We find that PDR’s new sensibilities have significant alignments with designed interventions organized as social design and formative interventions insofar as PDR is concerned with learning, specifically transformative forms of learning in which people can become designers of their own futures. This is well in line with the ethos of CHAT as an activist approach (p. 276).

3. Using the methods of inquiry to ensure in a Change Laboratory intervention

After having located the Change Laboratory within the panorama of qualitative inquiry, this paper shows how the methods and strategies of inquiry can complement an intervention to ensure validity and will use a Change Laboratory intervention conducted in 2016 as a case study that is a contemporary, real-life event bound in space and time (Yin, 2009). Data validation – or better trustworthiness in qualitative inquiry – connects to how researchers can affirm that the results are trustful to the experience of the participants (Ravitch and Carl, 2015). Trustworthiness calls for the importance of ensuring credibility and rigor to qualitative inquiry. Correspondingly, it requires acknowledgement of the complexity of the participants and contextualising their lives appropriately and deeply. This paper will concentrate on thick descriptions and member checks as possible examples of trustworthiness and rigour; more data about the specific intervention is available in Author (2019).

In qualitative research, a thick description depicts the context of research through writing, the goal being to describe carefully and thoroughly the important events concerning the research context, the individuals participating in the study and their related experience (Ravitch and Carl, 2015). Such description should permit the reader to comprehend the background so that she or he is able to make own hypothesis on the quality of research and the interpretations made by the research. Concerning member checks, it is an oriented process centred on the person to challenge the researcher’s interpretations, this is done by putting the participants in the conditions to talk about the research, so that they can feel free to agree or disagree with the research results (Morse, 2018).

3.1 Thick description of the intervention

In 2015 the first author of this contribution had obtained a European grant and was looking for a secondary school in which to carry out a Change Laboratory intervention. It was natural to select the technical institute where he had already carried out previous research with both students and teachers. The school chosen was an Italian technical secondary school in a small city of Northern Italy, and since its foundation in the 70s had hosted a sole five years’ course for surveyors. However, after the school reform of 2008, the school differentiated its offering with three courses: in Media Communication Graphics, in Logistics and in Surveying. While the institute had always hosted roughly 500 students, after the school reform the surveying course was losing new enrolments, while the newly opened course in Graphics and Communication was dramatically improving the number of students, and the Course in Logistics was steady at roughly 15 enrolments per year. At the beginning of this European project, it was unclear who the participants would have been; while the school director would have liked the intervention to be conducted in her school, a group of possible volunteering participants and an activity with an issue to be tackled had not been identified (for example a class, a course, teachers teaching a common subject and the like). The first step was to conduct observant participation from December 2015 to February 2016 to gather data and find possible triggers for the Change laboratory.

The researcher talked to teachers, went into the classes, participated in the school open day as well as the open day of the “competing” technical school. He also made phone calls to the alumni of previous two years, looking for differences between the alumni who finished their study before the school reform and the students who completed after the reform was implemented. The outcomes of observant participation and students’ survey were presented during the school council of February 2016. One issue was that an effect of the reform could have been NEET surveying alumni, that is not engaged in Education, Employment and Training. The second outcome pertained the scarce attractiveness of the open days especially for surveyors, where the combination of old premises and old workshop areas, lack of projects, and lack of collegiality could have caused a drop of new enrolment students in the surveying course. A subsequent discussion started during the school council to choose the target of the intervention and the participating teachers, which would have been the teaching body of the surveying course, with the objective to debate and find solutions for the dramatic fall of students.

The Change Laboratory sessions took place from February to April 2016 with 14 teachers (especially vocational instructors) and workshop assistants. The eight Change Laboratory sessions were followed by 3 follow-up sessions, while the new idea developed during the sessions was implemented in the classes for the subsequent two school years. Figure 1 depicts the new enrolments students in surveying alongside 10 years; When this chart was used during the first session the aim was to discuss what had led to such a drop of new students. Some of the reasons emerged during the focused discussion were external, such as the crisis in the estate sector or the school reform, and others were internal to the institute, like the lack of cooperation between teachers and the lack of common projects that could be used as flagships to promote the course.

During the second session the participants did an historical analysis of their course. In a banner they drew a timeline in which they wrote the school main events from its creation: the school directors, the school reforms, the courses, and the teachers in the room. When asked to find a clear between the two most important periods characterising the history of the school, the time that had marked the deterioration of the course from the “golden era” to a state of crisis and uncertainty coincided with the school reform of 2008. A design teacher stated during this session:

From the start of the reform the word disorientation is the most correct [to define how we feel]. Many of us do not agree with the reform since it is unconvincing when compared to the previous. Many of us think it worsen the state of things. This is because our course does not focus anymore on the preparation of a technician, rather it focuses on general education matters, and this left us confused. (Translated from Italian).

In the third meeting the participants compared the curriculum before the reform with the curriculum after the reform to find weaknesses and strengths. To do so, on a banner they drew a table with two columns, each of which had one of the course programs, and in the rows, they found distinctive elements for comparison. They also used different colours to mark whether the difference was positive or negative: black represented negative, green was positive, and grey showed areas that were potentially positive, yet to be developed. Among the shortcomings was the relative fall of number of teaching hours concerning technical subjects, but some of the possible benefits - though to be developed (hence “grey areas”) were the strong increase of work experience and the introduction of the workshop assistants as teachers’ helpers. During the fourth session, two experts from local business unions came to describe the new surveyor that the enterprise needed, though for many teachers it was nothing new, as one teacher in topography comments the following meeting:

Work experience does not frighten us, we did it before the reform and we know that our students benefit from it. We are aware that our students will do find a job, that is, we knew already he things we heard from the industry representatives. [Our problem is rather:] How can we have the world understand that, despite the crisis in the estate sector, our technician is still irreplaceable?

The fifth session is characterised by further analysis of the problems. A design teacher said that:

I would like to remind one of our weaknesses which I care a lot about: we do not work as a team both from the training and education point of view. Other schools make a so called “agreement of responsibility”, they meet at the beginning of the year for more times and agree on educational and training values. We prefer to do an easy half an hour meeting at the beginning of the year, which is not a serious commitment on our side.

Discouragement prevailed, which was shown in a topography teacher claiming:

Now we are talking [about problems and solutions], in any case next year I will not be here since my teaching post will be vacant (because of the lack of students).

At the end of the session however, a design teacher commented:

I think that we cannot continue to talk about what we lack. From the next time we have to start analysing concrete proposals of solutions or instruments. The analysis has already been done.

In between the 5th and 6th session the participants received a home assignment on concrete proposals on the future of the surveying course to be presented during the sixth session. After having heard all the proposals, the discussion focused on a common and vocational multidisciplinary project, as summarised by the topography teacher:

Six teachers, three of section A and three of section B, [with topics] such as Constructions, Land Valuation and Topography. These teachers agree to perform a common project during the school year, and the project is coordinated by the workshop assistants.

The same topography teacher later in the session explained how this project could concretely help increase the number of new enrolments:

With this multidisciplinary project we show that we cooperate and that the students can synthetize all that they have learnt [in the diverse technical subjects]. We make a [concrete] project and then we present it, for example during the open days, and see what happens.

In the following 7th and 8th sessions the participants plan the multidisciplinary project in detail, so that it can be delivered the following school year; the 8th session is a technical council to approve officially the multidisciplinary project. The participants took a month before the subsequent follow-up meeting to experiment with new didactics such as teamwork and flipped classroom that could have served to deliver the multidisciplinary project the following school year. The first follow-up session in May 2016 discussed the benefits of these didactics, although they were not employed during the project. Towards the end of the subsequent school year the researcher gathered evidence on how the multidisciplinary project was going through focus groups in the two classes and interviews with the workshop assistants and technical teachers participating in the Change Laboratory.

3.2 Member checks

After almost one school year of implementation of the new idea, the interest was to better understand the new object (the interdisciplinary project), and this could only be done by asking the participants involved in the new activity: the teachers, the workshop assistants and the students. The researcher interviewed six teachers, two workshop assistants and made a focus group with the two classes implicated in the project. The focus of interviews and focus groups revolved around: a) questions on the object such as: “Could you explain me what the multidisciplinary project is?”; b) questions asking for comparisons like: “How do you think the multidisciplinary project differs from a “regular” project?”; c) questions on the final outcome such “To what extent do you think the multidisciplinary project could impact the new enrolments in surveying?” The interviews and focus groups where fully transcribed and organised through multiple readings. The thematic analysis (Author, 2019) made by the researcher four themes: historical antecedents (only for the experienced teachers who had participated years before in multidisciplinary projects), features, potentials, and challenges. The following table shows one example of excerpt of interview from the topography teacher on the features of the multidisciplinary project. (Table 1)

This thematic analysis finding four themes was summarised and turned into a graphic representation in form of conceptual map. The intention was to use it during the second follow-up session as a participant validation strategy.

Figure 1 shows the different actors (teachers, students, workshop assistants), features, potential and challenges, and themes emerged during the recursive analysis. In the follow up session, participants included teachers, workshop assistants and two students from each class who were involved in the project.

At the top there are three themes regarding the interdisciplinary project: it is founded on authentic tasks, it bridges technical subjects, and the main didactic is group work. These features are emphasised in bold characters, since all parties agreed on these features, including teachers, technical assistants, and the students of classes A and B. This finding was subsequently validated in form of member-checks during a follow-up session with teachers, technical assistants, and representatives from the students. Figure 2, however, also allowed the participants to discuss and reflect on similarities and differences on the application of the concept of multidisciplinary project in the two classes, and also how the project could be improved the following year, especially on how to deal with the challenges, for example by having the workshop assistants acquiring a major role, how to make the project more multidisciplinary, how to improve teachers’ coordination and participation, and how to better integrate the multidisciplinary project in the curriculum.

3.3 Discussion

The previous two sections have shown a possible use of qualitative validation strategies. However, not only these strategies are valuable to ensure validity, but they also align well with CHAT. Concerning thick description of the intervention, for example, for Engeström (2001, 2011) historical contextualisation of the intervention is essential. It is also key that the researcher inspects the local history of the organisation, its object, and the concepts and tools that characterised it. Regarding the member checks, the discussion with participants of Figure 2 was not only good to validate the results but was also good for them to reflect on their project and see what needed to be done the following school year to strengthen it. This corresponds to the expansive learning action of evaluating the process, as well as consolidating the outcome and making it a stable practice (Engeström, 2015). It can be consequently argued that the member checks contributed to strengthen the outcomes of the Change Laboratory itself.

4. Conclusions

This paper sought to illustrate how the Change Laboratory, by taking up an ontological and epistemological stance, offers stable grounds for the construction of a viable methodological research framework. Here, ‘methodology’ was equated to the level of formative interventions, an umbrella term for the varied implementations of CHAT-oriented activities in educational settings, the Change Laboratory being one of their resulting toolkits (Engeström and Sannino, 2010, p. 15). Once the methodological relevance of formative interventions was conceptually qualified, a question was raised as per what type of research methodology is being called for when undertaking a Change Laboratory: in fact, although it generally leads to educational improvements, it is not always clear to what extent interventive processes can also be treated as epistemically sound, and, as such, transferable and qualitatively valid investigative approaches bearing on educational phenomena. In other words, the Change Laboratory is well outlined as a formative method, but not fully understood as investigative method.

This paper has argued that the Change Laboratory is a strategy of inquiry, it is in line with the characteristics of qualitative research and follows the agenda of participative paradigm. Hence, as interpretive research, the Change Laboratory shares with action research and design research the aim of bringing about change; while it was conceived within the CHAT theoretical perspective, it also follows the agenda of a participatory paradigm. In response, this paper has suggested that the methods and strategies of inquiry could strengthen the rigour of the Change Laboratory, and a specific Change Laboratory intervention showed how trustworthiness was sought by means of a thick description and the member checks.

One issue about situating the Change Laboratory within qualitative research concerns the use of numbers. While numbers inform a quantitative approach to research, alternatives within qualitative inquiry have been put forward to overcome a strict, mono-dimensional, and often useless usage of figures. Yanchar (2011) showed examples of use of numbers used in CHAT when inspecting workplace learning. Numerical data are deployed in inquiry to “enrich and clarify explicitly interpretive accounts of human phenomena in real-world settings – accounts typically produced by qualitative researchers” (p. 181). Hence, numbers can be used in Change Laboratory, taking such forms as descriptive statistics. Moreover, while Change Laboratories are a distinctive strategy of inquiry, some authors already claimed that they can be complemented with other strategies. Postholm (2020), for example, suggests that action research, the Change Laboratory and action learning can be fruitfully combined depending on the needs of the participants. Concerning the research implications, however, this paper does not suggest combining diverse research strategies to carry out change. Instead, the argument is that a formative intervention can be seen as qualitative research process with diverse research phases, where the sessions with the participants constitute certainly the peak, but not the only component. Moreover, the use of diverse strategies to gather data such as participant observation at the beginning of the intervention or focus groups and interviews with participants when the new object or idea is stabilising, as well as qualitative methods to analyse and validate data, can help not only improve the rigour and quality of the research, but also the outcomes of the formative intervention itself.

Figures

Figure 1.

The number of new enrolments students in the surveying course from 2007 to 2016

Figure 2.

Graphic representation of the analysis of interviews and focus-groups with teachers, students and workshop assistants

Table 1.

Excerpt of interview on the features of the new object.

(Speaking turns 43-48). (Interviewer). What is the difference between the interdisciplinary project and another project you ran in the school? I know you carried out diverse projects. (Topography teacher). Do you remember that the last year we carried out the project on the gardens in another school? There is a difference though – the present multidisciplinary project deals with a building requiring more care compared to a garden. Let’s say that the students’ answer has been a little confused because they are not used to working on concrete things. (Interviewer). So, it’s a more practical project, more realistic. (Topography teacher). It’s certainly practical and realistic. That’s what we wanted. This is what we added to what we normally do. As I told you, we have always performed the survey out in the field. However, in previous projects the following step, that is the insertion of this survey into the program of the Real Estate Registry, had only been outlined. (Interviewer). This time they do it for real. (Topography teacher). Yes, this is a problem that we don’t normally deal with. Let’s say that this time we spent a lot of time on it so that the students could deal with reality. For example, the program does not work if you put a comma instead of a dot to separate the decimals. These obstacles however put the students into troubles and delay their work, so they very much feel the workload

Source: (From Author, 2019, p. 89)