Testing ZELPH ['sɛlf] – a self-assessment instrument to surface intended and unintended outcomes of work-integrating pedagogies

Abena Dadze-Arthur (Department of University Continuing Education and Teaching and Learning, FernUniversität in Hagen, Hagen, Germany)
Anita Mörth (Department of University Continuing Education and Teaching and Learning, FernUniversität in Hagen, Hagen, Germany)

Journal of Work-Applied Management

ISSN: 2205-2062

Article publication date: 10 February 2021

Issue publication date: 26 April 2021

Downloads
215

Abstract

Purpose

This paper's twofold purpose is, first, to present ZELPH ['sɛlf], a self-assessment instrument that enables those developing the pedagogy of work-integrating study programmes in higher education (HE) systematically to surface the intended and unintended outcomes of their programme's approach to integrating professional practice into an academic course. Secondly, the paper reports on a small pilot study with programme staff from five different HE institutions in various countries who tested ZELPH.

Design/methodology/approach

ZELPH operationalises aspects of key theories on work-integrating learning pedagogy, and thereby enables a simplified depiction of the reality of combining classroom-based and worksite-based learning. Programme staff from Germany, the United Kingdom, France, South Africa and Taiwan applied the instrument to their respective work-integrating study programmes and evaluated its perceived value and feasibility.

Findings

The findings suggest that ZELPH offers value as a practical instrument, in particular to those less familiar with developing work-integrating learning pedagogy as well as to those keen to compare programmes across national, cultural and institutional contexts.

Originality/value

ZELPH contributes to addressing the lack of practically applicable instruments to support the design and international benchmarking of work-integrating learning pedagogy in HE.

Keywords

Citation

Dadze-Arthur, A. and Mörth, A. (2021), "Testing ZELPH ['sɛlf] – a self-assessment instrument to surface intended and unintended outcomes of work-integrating pedagogies", Journal of Work-Applied Management, Vol. 13 No. 1, pp. 36-50. https://doi.org/10.1108/JWAM-11-2020-0047

Publisher

:

Emerald Publishing Limited

Copyright © 2020, Abena Dadze-Arthur and Anita Mörth

License

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode


1. Introduction

Worldwide, work-integrating higher education (HE), which is broadly defined as interlinking academic scholarship with professional practice, is increasingly becoming inevitable (Billett, 2014; Lester and Costley, 2010; Nottingham, 2017; Talbot, 2017). However, translating the theoretical concepts of work-integrating HE into pedagogic changes poses ample challenges, not least because it disrupts both “the discipline-based organisation of knowledge, and the signature pedagogies of individual professions” (Lester et al., 2016, p. 20). In the Australian context, Billet (2014) has lamented the lack of clarity regarding the use of pedagogic means to achieve the intentional integration of academic and experiential learning for specific educational purposes. In the US context, Raelin (2007) has long highlighted that a practice-based pedagogy is capable of accomplishing a diverse array of goals, including imparting an increased ability for critical reflexivity and meta-competent learning, evolving students to become both knowledge experts and masters of practice, and/or enabling graduates to adopt and inhabit various life roles and professional identities. Given the variety of purposes that a work-integrating pedagogy can achieve, and the lack of clarity as to how to accomplish them, it is not surprising that in the German context, Borgwardt (2015) has called for new tools and instruments, which are capable of facilitating result-oriented pedagogic approaches to work-integrated learning that achieve clearly defined outcomes. While there is some practical guidance available, for example, in the form of the United Kingdom (UK) Quality Code for Higher Education (Quality Assurance Agency QAA, 2018), such guidance tends to be embedded in the regulatory and statutory requirements of national policy contexts; oscillate between pedagogic, organisational, structural and institutional concerns; or fail to address the entire continuum which should also include work-integrating non-degree certificate programmes. The ZELPH model, with ['sɛlf] being an acronym from the model's German name “Modell zur zielgerichteten Einbindung des Lernorts Praxis in das Hochschulstudium”, aims to address the gap by offering an instrument that is capable of supporting the realisation of effective, situated and outcome-oriented pedagogies underpinning the entire spectrum of work-integrating learning provisions in HE. In recognition of the diversity of perspectives along a wide range of work-integrating pedagogic practice (see e.g. Nottingham, 2012; Lester et al., 2016; QAA, 2018), ZELPH values educational stakeholders as experts of their own practice, and thus seeks to serve as a self-evaluation instrument. Specifically, it is an analytical tool that enables those developing, employing and evaluating work-integrating learning pedagogy in HE systematically to surface the intended outcomes, as well as the unintended consequences, of their particular pedagogic approach to integrating professional practice into an academic course. Thereby, the model strives to add value in three ways. First, it aims to facilitate increased strategic decision-making in the design of work-integrating programmes. Second, it seeks to support a more outcome-oriented approach to making a case for getting such programmes approved by organisational or institutional entities, such as university boards and committees. Third, the model hopes to encourage increased comparisons between the pedagogies of various work-integrating study programmes across space, place and time, and across cultural, structural and institutional contexts, and thereby contribute to creating opportunities for knowledge exchange among educational stakeholders. This article reports on developing the ZELPH model (see Table 2) and testing its value and feasibility within the context of a small pilot study with educationalists across five countries.

2. Creating the ZELPH model

Seeking to develop a model that offers practical value to those designing, delivering and evaluating the pedagogy of work-integrating study programmes, the authors of this article operationalised a selection of theoretical aspects, which Mörth et al.'s (2018) study had shortlisted as relevant to the practice of academic work-integrating learning (Töllner, 2010). In a preliminary test, the ZELPH model was employed to analyse two good practice cases of work-integrating study programmes – a cooperative education programme at bachelor level in the USA, and a degree apprenticeship course at master level in the UK (see Dadze-Arthur et al., 2020; Dadze-Arthur and Mörth, 2020). The preliminary test results confirmed that ZELPH is capable of serving as a practical instrument in systematically capturing key aspects of a work-integrating programme's pedagogy across time, space and context (Dadze-Arthur and Mörth, 2020).

Subsequently, and in preparation for the pilot study that this article reports on, the authors conducted a second preliminary test, which involved asking the director of a work-integrating study programme at a German university to apply ZELPH to her educational offer. The authors shared with her both the ZELPH model, which included short explanations for each constitutive category (see Table 2), and the aforementioned analysis of the two good practice cases in order to illustrate the possible application of the instrument. The second preliminary test confirmed the model's principal value and applicability but also helped with identifying areas for improvement. Minor enhancements involved tweaking the explanations for some categories to increase clarity and tighten the scope, as well as introducing subheadings to more visibly structure the categories. The major change undertaken consisted of adding Section VII to the model with the aim to get respondents explicitly to state the programme outcomes. Here, the rationale was that in order to evaluate a study programme's pedagogy, we must consider the programme's methods and activities within the context of the outcomes it achieves. Section VII proffers two categories with a view to distinguish between the outcomes achieved regarding knowledge and skills versus the outcomes attained in terms of strengthening the agency of self-directed learners. Given that learning outcomes can be both intentional and incidental (Merriam and Bierema, 2014), each of the two categories differentiates between strategic or intended outcomes, and incidental or unintended outcomes.

3. Theoretical grounding of the ZELPH model

Keeping in mind that ZELPH operationalises a shortlist of the theoretical building blocks of a pedagogy for work-integrating education, let us now turn our gaze to the theories proffered by the literature on HE work-integrating learning (see e.g. Lillis and Bravenboer, 2020; Wall and Hindley, 2019; Lester et al., 2016; Costley et al., 2010; Boud et al., 2001; Boud, 2001; Nottingham, 2012, Cooper et al., 2010; Billett, 2011). For the purposes of developing the ZELPH model, this study adopted the usefully broad conceptualisation proposed by Wall and Hindley (2019, p. 1), suggesting that “work-integrating education” is understood as a type of education that “broadly connects practice settings as a location or vehicle of learning”. By premising ZELPH on such an all-encompassing definition, this study recognises the contested nature of work-integrating learning and seeks to embrace its many variants, including, for example, work-based learning, experiential learning and internship learning. Furthermore, in considering theories that were predominantly proffered by Western scholars, the authors accept that the discourse is rooted in Western paradigmatic conceptions of education, and thus fails to include the episteme of other, non-Eurocentric knowledge systems (Adebisi, 2016; Castro Romero and Capella Palacios, 2020).

3.1 Basic characteristics

Lester et al. (2016), Nottingham (2012) and Lillis and Bravenboer (2020), amongst others, offer empirical evidence for the considerable variety of work-integrating programmes in the UK alone, and demonstrate both commonalities and differences in underlying concepts as well as in the modes of delivery (Lillis and Bravenboer, 2020). While Mörth et al.'s (2018) study had echoed the literature's findings relating to the variety of work-integrating educational offers and the range of delivery modes, it had also confirmed both Wall's (2013) observation regarding the variability of the integration of disciplinary knowledge in practice, and Edwards et al.'s (2015) findings concerning the different priorities set by institutions in terms of the integration of academic theory with professional practice. Taking account of these insights, the ZELPH model begins the self-evaluation exercise by providing an opportunity to map the particular type of work-integrated learning of a programme (i.e. cooperative education, degree apprenticeship etc.) and the mode of delivery (i.e. on campus or blended, part-time or full time etc.) (see Table 2). Stating these very basic characteristics of an educational offer helps to typify it, and thus demarcates the limits set by its very type and delivery mode, which, in turn, allows for placing the evaluation in its appropriate context.

3.2 Work as place of learning

Notwithstanding the variety of approaches, they all have in common that work, paid or unpaid, is part of the curriculum and plays a role as a learning site (see e.g. Lillis and Bravenboer, 2020; Wall and Hindley, 2019; Lester et al., 2016; Edwards et al., 2015; Nottingham, 2012; Costley et al., 2010; Cooper et al., 2010). Accordingly, the model's category integration of professional practice into the study programme is designed to shed light on the ways in which work is integrated and valued as a learning site in a systematic manner. Thereby, the category invites details on how integration happens, for example, at curricular level, or at a formal institutional level, such as through a memorandum of understanding with an employer. Drilling down further into the specific conditions of work as a learning site, scholars concur that the students' activities on the job should amount to “a learning enterprise that, while commonly undertaken at work, is not identical to work” (Boud, 2001, p. 50). Accordingly, the category activities at the workplace aims at surfacing whether students complete regular work tasks or whether they are asked to do experiential learning tasks, such as shadowing a senior executive and subsequently reflecting on it in a learning journal. In this context, the literature argues that it also matters when and for how long the students' employment takes place (Lester et al., 2016; Cooper et al., 2010; Billet, 2011). Consequently, ZELPH includes a category entitled time at the workplace, which, despite its ostensible simplicity, reveals much about a programme that, for example, only requires one three-month work placement throughout a four-year programme, compared with another educational offer as part of which students work and study in parallel for four years.

3.3 Value of professional skills and competencies

In the literature, the debate on work as a pertinent learning site within the context of a culture that assumes primacy of subject discipline is reflected, in practice, in the value that educational providers assign to professional skills and competencies. In real life, it directs, for example, developments in negotiated curricular and learning shell frameworks, or in enabling access to HE for non-traditional and lifelong learners (see e.g. Talbot, 2017; Lester et al., 2016; Bravenboer and Lester, 2016; Lillis and Bravenboer, 2020; Wall, 2013; Boud, 2001; Costley et al., 2010). In an effort to operationalise the level of recognition that is afforded to practitioner knowledge, the ZELPH model includes the category entry admission/requirements, which is broad enough to also cater for those programmes that only require students to meet academic requirements. Probing more explicitly into the perceived value of non-academic and/or professional knowledge, the subsequent category accreditation of prior, non-academic competencies is aimed at surfacing whether the programme under scrutiny considers non-academic competencies important enough to translate them into academic credits. In recognition of departments without frameworks for non-academic competencies, the category generic, non-subject specific competencies provides an opportunity to identify learning outcomes that are not discipline-centred and not academic, but relevant to becoming a skilled practitioner.

3.4 Acquiring professional skills and competencies through the academic component

Learning that is relevant to professional practice occurs not only on the job but also in academic components of a work-integrating study programme. While the literature agrees on the importance of defining learning outcomes relating to professional skills and competencies (see e.g. Lillis and Bravenboer, 2020, Lester et al., 2016; Bravenboer and Lester, 2016; Cooper et al., 2010; Boud, 2001), some scholars argue for the relevance of employment- or sector-specific learning (Bravenboer and Lester, 2016; Cooper et al.), while others, especially those discussing learner-centred provisions, make a case for generic learning in an effort to capture transdisciplinary learning that occurs when work is the frame of reference instead of academic disciplines (Boud, 2001; Costley et al., 2010). With a view to reflect the key points of this debate, ZELPH proffers the category approach to interdisciplinarity to reveal whether interdisciplinary learning is achieved, for example, by design (i.e. a compulsory module in another department) or by methods (i.e. tasks that require an interdisciplinary response). ZELPH also features the category approach to integrating themes, topics and issues, which relate to the study field's associated professional practice, into teaching and learning activities on campus with a view to surface how the academic component draws on real-life professional practice, such as asking students to analyse their own practical experience by applying the precepts of academic theory. The subsequent category approach to teaching and learning invites respondents to identify, at a high level, which variant their pedagogy for instilling both academic and professional knowledge falls under, that is, work-based, work-integrating, experiential learning and so forth.

3.5 Personal growth of the reflective student

According to the literature, a key feature of work-integrating programmes is that students are expected to act as self-directed learners and maximise the benefits offered by multiple learning sites (Lillis and Bravenboer, 2020; Lester et al., 2016; Gibbons et al., 1994; Schön, 1983, 1987; Wenger, 1998). In practice, the implication is that work-integrating programmes must instil in students a “strong ethos of reflexivity and practitioner enquiry”, which enables them to cement, consolidate and apply learning that occurs in the classroom and at work (Lester and Costley, 2010, p. 564). In order to achieve such personal growth among students, scholars highlight the importance of supportive activities before, during and after the time at the workplace (Lillis and Bravenboer, 2020; Cooper et al., 2010; Billett, 2011), as well as the role of academic advisers and workplace supervisors in engaging students through reflexive learning activities (Lester et al., 2016; Boud, 2001). Accordingly, ZELPH includes the categories reflexive methods to embed the learning of professional practice, which might involve methods such as learning journals, reflective commentaries and 360 feedback, and approach to developing students' professional identities, which could involve, for instance, career counselling, peer networking and so forth.

3.6 Programme outcomes

Defining learning outcomes is an inevitable part of curriculum development in all HE programmes. It is perhaps slightly more complex in work-integrating programmes, considering that learning outcomes ideally reflect occupation- and sector-specific competencies in addition to academic competencies (Lillis and Bravenboer, 2020; Bravenboer and Lester, 2016; Lester et al., 2016; Nottingham, 2012; Brennan and Little, 1996). Given that ZELPH is designed to assess the pedagogy specific to combining academic and professional learning, it proffers the category outcomes of the programme's particular approach to interlinking academic theory with professional practice. It invites respondents to articulate that the end result is, for example, a loose, complementary juxtaposition of theory and practice, or a formal and systematic fusion of the two. The model also seeks to elicit the outcomes achieved by applying the particular pedagogy to students, which is operationalised through the category outcomes of the programme's particular approach to guiding learners towards achieving learning outcomes, professional competencies and personal maturity. It aims to evoke the impacts on learners, such as an ability to cope on their own in the world of work or an ability to take on leadership roles.

However, not only the formally agreed programme outcomes are pertinent to evaluating the pedagogy of a work-integrating programme. Incidental learning outcomes are equally relevant because they constitute the “side effects” of an educational initiative (see e.g. Merriam and Bierema, 2014; Billett, 2011). They are the result of “what students experience when they engage with what was intended through what is enacted, and how they learn through that experiencing, even that which is unintended by those who plan and enact the curriculum” (Billett, 2011, p. 2). If unintended learning outcomes are found to be undesirable, the circumstances that create them need to be identified and adapted (Portelli, 1993). Correspondingly, if they are viewed as desirable, they can be made explicit and included in the list of outcomes that are formally agreed (ibid). ZELPH reflects these theoretical considerations by inviting respondents to list unintended programme outcomes, such as students' emotional resilience or mistrust of conventional academia.

4. Methodology

4.1 A methodological approach that is fit for purpose

As is expected from an effective methodology (Bryman, 2001), all the study's decisions concerning research design, methods and analysis were guided by its overall research question: To what extent is the ZELPH model both a valuable and feasible instrument in surfacing the intended outcomes, as well as the unintended ones, of a university department's particular pedagogic approach to integrating professional practice into an academic programme of study? By suggesting that research design is the glue that bonds a research project together, Trochim (2005) highlights the pertinent role of an effective research design in giving logic and structure to the enquiry. This, in turn, enables the researchers to answer the research question and also assures the readers that resulting insights are robust, valid and worth paying attention to (Denzin and Lincoln, 2011). Accordingly, this research project chose a pilot study design, which allows for evaluating an instrument in a form that is similar to the final one and identifying opportunities for improvement (Denscombe, 2010; Majumdar, 2008). A pilot study design was congruent with this enquiry's objective to conduct a small-scale preliminary test of ZELPH's expediency and its ease of application across cultural, institutional and structural contexts. The research project opted for a qualitative research approach because it allowed for investigating in depth the perceptions and experiences of programme staff piloting the model, while being in line with the project's idealist ontology and constructivist epistemology (Denzin and Lincoln, 2011; Bryman, 2001).

4.2 Collecting data across five countries

The sampling strategy was based on reputation (also called expert or judgemental sampling) with experts being defined as university staff members, who are experienced in the design, delivery or oversight of work-integrating study programmes. Keeping in mind our aim of cross-context applicability, we sought to include experts from a variety of cultural contexts, resulting in five staff members from five countries (Germany, Great Britain, France, Taiwan and South Africa). Although reputation sampling cannot claim to reflect the theoretical population, this was not an issue within the context of this particular project because pilot studies are usually conducted on small, non-probability samples (Northrop and Arsneault, 2008). In a first step, the research participants were sent a document that detailed the ZELPH model, along which they were asked to analyse the pedagogy of their respective work-integrating study programmes. In an effort to illustrate the application of the model, the document also included exemplary analyses of the two good practice cases. In a second step, the respondents were interviewed about their experience of filling in the ZELPH model's categories with a semi-structured topic guide that focused on the perceived value of the model and the feasibility of applying it. As advised by Denscombe (2010), the interview questions were phrased as unambiguously and neutrally as possible to minimise bias and avoid influencing the respondents. All interviews were conducted and audio-recorded with the help of web conferencing tools or video chat applications, and they lasted between 30 and 45 min.

4.3 Data analysis

The recorded interviews were transcribed and subsequently analysed by employing an abductive thematic analysis that enabled the researchers to surface the understandings and misunderstandings of the respondents in applying the model (Blaikie, 2007). In an effort to ensure the validity and credibility of the findings, the researchers took a two-pronged approach: First, two researchers analysed all interviews separately and then conducted a workshop to triangulate their findings. This ensured the verification of emerging themes and ensured that no aspect was overlooked. Second, the results were confirmed by scrutinising the respondents' approach to filling in the ZELPH model categories. It allowed for cross-checking, for example, whether a self-reported ease in filling in a certain category was indeed reflected by the way he or she answered the relevant section in the template. Furthermore, ethical concerns were met by ensuring participants' confidentiality and anonymity and by working in accordance with the university's data protection policy as well as the German Society of Sociology's Code of Ethics.

5. Presentation of the pilot study's results

The abductive thematic analysis revealed three non-confounded categories of responses: conditional value, practical feasibility and programme outcomes. This confirmed that the interviewers effectively probed the respondents in line with the pilot study's aim to assess the perceived value and feasibility of the ZELPH model. For each of the three categories of results, the analysis surfaced themes that provided further insights on the extent and range of the category, termed “thematic scope of category” (see Table 1).

5.1 1st category of result: conditional value

All participants concurred that the model is essentially a good idea and potentially of practical value as a self-evaluation tool across different contexts, albeit under certain conditions. As is the nature of any instrument, or for that matter of any means to an end (see Rønnow-Rasmussen and Zimmerman, 2005), the analysis showed that the ZELPH model is seen as conditionally, or extrinsically, valuable, with its worth depending upon the conditions under which it is employed. Specifically, two conditions emerged as pertinent to determining the value of ZELPH:

5.1.1 Condition 1 – concerned with work-integrated learning (WIL) pedagogy

Respondents who were less familiar with the intricacies of a work-integrating pedagogy emphasised the model's considerable practical and theoretical merits. In terms of pedagogic practice, this group of interviewees reported that ZELPH offers an effective support tool for busy academics. From a theoretical perspective, they appreciated that the model outlines the pivotal conceptual building blocks of interlinking traditional academic teaching with experiential learning in the workplace. Thereby, the model was able to stimulate systematic reflection and assist in identifying gaps or room for improvement. One participant, for example, became aware that the recognition of prior learning had been overlooked in her programme, while another respondent deliberated the need for more reflexive activities in her programme. The following verbatim quotation epitomises the feedback of those interviewees who found the model to be practically and theoretically useful:

As I was reflecting on the way we are doing it and the way the model proposes to do it, I think teaching work-integrating programmes is a skill that has to be learned. And being a skill that has to be learned, it must be grounded in certain pedagogic practices, and also in theories (…) So, I think you have been able to come up with a major approach. (Respondent 2)

Interestingly, the notion that the ZELPH model offers value to those concerned with the pedagogy of WIL was also supported by those who found the model to be of limited merit. These interviewees reasoned that they found the model to be of little value because they already have considerable expertise in the pedagogy of work-integrated teaching and learning. Their main concerns revolved around overcoming organisational issues, such as institutional, structural and/or cultural barriers, on which the model includes no categories. Admittedly, the reality in academia is that unless work-integrating learning initiatives are part and parcel of a university's strategic objectives, they tend to be relegated to the fringe of the institution's activities (Dadze-Arthur et al., 2020). There, these programmes struggle for visibility, recognition and in some cases even for survival, and they often tend to be only tolerated for their ability to generate revenue (ibid). However, finding that ZELPH only offers value to programme staff concerned with pedagogic issues confirmed that the model is a function of the purpose it seeks to serve. The following excerpt nicely illustrates this point:

The reason it [the ZELPH model] does not fit us, really, is because it talks about issues in relation to work-integrating learning as pedagogy, and that's actually the least of our problems. (…) Our main issue is trying to keep in business and deal with the university. (…) These are the real issues for us. (…) They are how on Earth do you have, basically, an alternative model of education within an existing framework, which is not understood? (Respondent 4)

5.1.2 Condition 2: seeking to benchmark good practice in WIL

The second condition upon which the value of ZELPH seems to be contingent relates to the desire for benchmarking. Some respondents recognised the model as useful for comparing the pedagogy of work-integrating programmes across institutions, contexts and cultures and suggested that ZELPH could serve purposes of peer exchange around good practice. Here the caveat is, of course, that while the model seeks to be widely applicable across different contexts, it can never claim universal relevance, and its value and applicability will inevitably be limited by the “situationality” of individual work-integrating programmes. The following quote usefully sums up this point:

Even though knowledge is highly situated and everybody's circumstances are different (…), I strongly believe that it [the ZELPH model] offers the perfect opportunity to benchmark one's programme internationally. Especially in terms of the integration of professional practice. (Respondent 1, translated by authors)

5.2 2nd category of result: practical feasibility

The data analysis also showed that, in general, the participants found ZELPH to be practically feasible and relatively easy to apply. The analysis surfaced two aspects based on which respondents seemed to judge its practical feasibility.

5.2.1 Constituent categories

Overall, the model was said to be well structured, with the constituent categories productively forming a coherent whole. The categories were perceived as usefully building on each other and reflecting most aspects that are considered essential in realising work-integrated education. A couple of aspects were identified as unhelpfully mixed in with other components and deserving of their own explicit category, which included the employer perspective and the assessment of learning outcomes (see section “Improving ZELPH”). The following quote helpfully captures the overall sentiment regarding the systematic unfolding of the model's constituent categories:

There are no overlaps in terms of the categories and I think following them is a good approach for instructors to know better how to proceed in their teaching. (Respondent 5)

Drilling down into each of the individual categories, the interviewees concluded that most of them were discrete and well explained. That said, some respondents helpfully highlighted a few categories that could have been even clearer by avoiding education-specific jargon, or where this was not possible, by providing examples. For instance, in order to illustrate what is meant by the concept of “generic, non-subject specific competencies”, participants suggested to list examples of competencies, such as team working, communication skills and project management skills.

The language was a little bit complicated because I'm not an education major so sometimes, when I deal with those education words, I need to ponder them. But most of them I think are clear, no problem interpreting those categories. (Respondent 5)

In the two cases in which one or more categories remained unanswered in the ZELPH template that respondents filled in before the interview, it turned out that the reason was not a lack of comprehension. One participant explicated that she had left one category without reply because it probed into an aspect that had not been considered in the programmes' pedagogy. In the same vein, another interviewee revealed that she needed to retrieve information from another member of the team because she was not involved in the aspect of the study programme that the category in question enquired about.

5.2.2 Exemplary application

The interviewees agreed that the two examples from the USA and the UK further helped to illustrate the categories in addition to stimulating reflection on the respondents' own pedagogic approach to work-integrated teaching and learning. In particular, non-native English speakers reported being better able to comprehend the scope and depth of the categories by studying the examples. Having said that, one interviewee had failed to notice the examples but still encountered no problems in employing the template for the purpose of mapping and analysing the pedagogic approach of his university's WIL programme. The below quotation illuminates the respondents' overall experience of the exemplary application of the two case studies to the ZELPH model:

The examples were very helpful! Originally, I did not understand completely in the first place. But I saw your example from Drexel University and the university from the UK and then I understood completely and could easily apply the model. (Respondent 5)

5.3 3rd category of result: programme outcomes

The sample agreed that the ZELPH models Section VII, which asks respondents to identify intended and unintended programme outcomes, stimulated reflection the most.

5.3.1 Intended programme outcomes

The majority of the interviewees reported that listing the strategically planned outcomes triggered deliberations as to whether their programme's particular pedagogic approach, which they had just explicated along the model's categories, was indeed purpose-driven and outcome-oriented.

Especially having to list the planned goals really helped to remind me of why we are doing all what I had just listed in the previous categories – because these are the goals we are trying to achieve! (Respondent 1, translated by authors)

5.3.2 Unintended programme outcomes

While respondents concurred that the intended outcomes were relatively easy to identify on the basis of established departmental programme objectives, they reported that diagnosing the unintended programme outcomes proved particularly challenging. As a result, some interviewees left these categories either blank or used the space to report problems or tensions, such as the conflict between employers and the university or challenges posed by legal constraints. One interviewee proposed to include in the model's explanations suggestions for possible sources that could help with identifying unintended programme outcomes, such as student or instructor feedback and employer evaluations. The following quote epitomises the interviewees' experience in addressing the categories relating to unintended programme outcomes:

In fact, these categories on unintended consequences were the most difficult ones. They were the ones which I had to ponder the longest because, of course, you check whether you have achieved your targeted outcomes. But normally you do not check the outcomes you never planned. (Respondent 1, translated by authors)

6. Improving ZELPH

Following the pilot study, a few further minor enhancements were undertaken, involving additional examples and tweaking some of the categories' explanations. For instance, in recognition of the experienced difficulties in identifying unintended outcomes, suggestions for ways in which respondents could surface these are now included in the latest version of the ZELPH model. The more substantial change included adding two new categories, the first one being systematic involvement of employer in the study programme. Although the subject of employers was originally already addressed through the category integration of professional practice into the study programme, the pilot study brought to light that an additional category on the topic was required. The newly added category invites respondents explicitly to detail the variety of contributions employers make to the programme under scrutiny, which may range from providing suitable opportunities for on-the-job learning to employers being involved in grading students' assignments. The pertinence of this category is corroborated by the literature, which emphasises that “without the explicit and tangible support of the organizations in which learners work (…) there are severe limits to what can be achieved” (Boud et al., 2001, p. 4).

The second newly added category approach to assessing learning outcomes and student performance reflects the distinctness of work-integrated learning in combining academic and professional knowledge, which must be mirrored in the assessment of learning outcomes. The literature supports this by asserting that relevant assessment depends and draws on the academic and professional learning outcomes and involves stakeholders from both sites (Costley et al., 2010; Boud, 2001; Cooper et al., 2010).

6.1 The ZELPH model

Following the insights gained from the pilot study, the researchers adjusted the ZELPH model, resulting into the following version.

7. Conclusion

This article sought to present the ZELPH model as an analytical and reflective tool for developing and evaluating work-integrating HE programmes, placing a focus on intended programme outcomes while also facilitating the revelation of unintended ones. The article offered evidence for the model as an instrument that, under the right conditions, can practically contribute to systematically assessing the pedagogy of work-integrating programmes, but also enabling comparisons across place, space and time. The project's greatest limitation includes the narrow scope that is inherent to pilot studies. However, the pilot study was only intended as an initial test to confirm the model's value and feasibility and justify future broader studies that involve larger, representative groups of respondents. Another limitation is that the model cannot operationalise every possible aspect of work-integrated pedagogy. Therefore, the model has sought to factor in the diversity of work-integrating approaches in HE by encouraging self-reflexivity along a shortlist of theoretical criteria that previous studies had identified as pertinent to realising such programmes, notwithstanding different contexts and intended outcomes. Overall, this paper's findings contribute to a programme of research that seeks to pave the way for the development of practical tools and instruments capable of facilitating outcome-oriented pedagogic approaches to work-integrated teaching and learning.

Categories of results

Category of resultThematic scope of category
1. Conditional valueCondition 1: Concerned with WIL pedagogy
Condition 2: Seeking to benchmark good practice in WIL
2. Practical feasibilityConstituent categories
Exemplary application
3. Programme outcomesIntended programme outcomes
Unintended programme outcomes

Illustration of the ZELPH model

Sections/CategoriesQuestions underpinning the categories
I. Basic characteristics
Type/category of work-integrated study programmeWhich category of work-based learning does this particular programme fall under?
Delivery mode of study programmeIs the study programme's mode of delivery flexible, i.e. in terms of time, place and pace?
II. Interlinking of academic teaching and professional practice
Integration of professional practice into the study programmeIs the student's work experience systematically integrated into the study programme, i.e. organisationally, institutionally, pedagogically, etc.?
Systematic involvement of employer in the study programmeIs the employer systematically involved in the study programme, i.e. in the design or delivery of learning, creating learning opportunities at the workplace and/or assessment? Or in the governance of financing of the study programme?
III. Types and objectives of work-based activities
Time at the workplaceAt which point(s) during their study programmes are students expected to spend time at the workplace, and for how long?
Activities at the workplaceWhat activities/tasks are students expected to pursue/undertake at work, and for what purpose?
IV. Relevance of professional skills and competencies
Entry/admission requirementsWhat are the admission requirements for applicants to the work-based programme of study? Do they include, for example, prior professional experience and/or professional qualification?
Accreditation of prior, non-academic competenciesIs it possible to accredit a student's prior, non-academic learning (including formal learning such as vocational education as well as any informal learning)? If so, what is its relevance to the programme?
Generic, non-subject specific competenciesDo generic, non-subject specific competencies form part of the programme's learning outcomes (e. g. team working, time management, communication skills, project management skills)? If so, who develops them, at which point and for what purpose?
V. Pedagogic approach
Approach to teaching and learningWhat approach to teaching and learning is taken that includes one or more work components in the study programme (e.g. work-based, work-integrated, project-based)? If the approach is not systematic, please describe the specific methods in a few words
Approach to interdisciplinarityIs the programme of study interdisciplinary, and if so, how are interdisciplinary learning outcomes achieved, i.e. is interdisciplinarity mainstreamed across the programme, or are there discrete interdisciplinary elements at course- or module level?
Approach to integrating themes, topics and issues, which relate to the study field's associated professional practice, into the teaching and learning activities on campusHow are themes, topics and issues, relating to the study field's associated professional practice, integrated into the teaching and learning activities on campus? For example, do students draw on their respective individual professional experience, or are they asked to work on general case studies that are illustrative of the field of study's associated professional practice, or do they work in professional labs etc.? What is the aim of the particular approach taken?
VI. Personal growth
Approach to assessing learning outcomes and student performanceHow and by whom are learning outcomes and student performance assessed (e.g. project at work that is assessed by employer, research paper about a practical intervention at work assessed by academic examiner and employer)?
Reflexive methods to embed the learning of professional practiceWhat reflexive methods are employed at which points throughout the programme to enable students to embed their learning of professional practice (e.g. learning diaries on a weekly basis, researching own practice once per semester, self-evaluation at the end of the work component)?
Approach to developing students' professional identitiesWhat approach is taken to support students in developing a professional identity? Who supports students in this process? At which point in the study programme does this happen?
VII. Intended and unintended programme outcomes
Upon reflecting on the answers provided in the above table, the integration of professional practice into the academic programme of study leads to the following outcomes (considering, for example, student or instructor feedback, employer evaluations, personal observations of learners' behaviours or instructors' dilemmas might help to identify unintended consequences)
Intended programme outcomesUnintended programme outcomes
Outcomes of the programme's particular approach to interlinking academic theory with professional practice.
Outcomes of the programme's particular approach to guiding learners towards achieving learning outcomes, professional competencies and personal maturity

References

Adebisi, F. (2016), “Decolonising education in Africa: implementing the right to education by re-appropriating culture and indigeneity”, Northern Ireland Legal Quarterly, Vol. 67 No. 4, pp. 433-452.

Billett, S. (2011), Final Report. Curriculum and Pedagogic Bases for Effectively Integrating Practice-Based Experiences, Australian Learning and Teaching Council, Strawberry Hills, NSW.

Billett, S. (2014), “Integrating learning experiences across tertiary education and practice settings: a socio-personal account”, Educational Research Review, Vol. 12, pp. 1-13.

Blaikie, N. (2007), Approaches to Social Enquiry: Advancing Knowledge, Polity Press, Cambridge.

Borgwardt, A. (2015), Investitionen der Wirtschaft in Hochschulbildung – Wer profitiert? Konferenz am 24. November 2014, Friedrich-Ebert-Stiftung und Stifterverband für die Deutsche Wissenschaft, FES, Berlin, Vol. 11, Schriftenreihe Hochschulpolitik.

Boud, D. (2001), “Creating a work-integrating curriculum”, Boud, D. and Solomon, N. (Eds), Work-integrating Learning. A New Higher Education?, SRHE and Open University Press, Buckingham, pp. 44-58.

Boud, D., Solomon, N. and Symes, C. (2001), “New practices for new times”, Boud, D. and Solomon, N. (Eds), Work-integrating Learning. A New Higher Education?, SRHE and Open University Press, Buckingham, pp. 3-17.

Bravenboer, D. and Lester, S. (2016), “Towards an integrated approach to the recognition of professional competence and academic learning”, Education + Training, Vol. 58 No. 4, pp. 409-421, doi: 10.1108/ET-10-2015-0091.

Brennan, J. and Little, B. (1996), A Review of Work Based Learning in Higher Education, CHERI and Open University Press, London.

Bryman, A. (2001), Social Research Methods, Oxford University Press, Oxford.

Castro Romero, M. and Capella Palacios, M. (2020), “Co-constructing a decolonising praxis in academia through dialogues and pedagogical experiences between the UK and Ecuador”, International Review of Psychiatry, Vol. 32 No. 4, pp. 365-373, doi: 10.1080/09540261.2020.1762548.

Cooper, L., Orrell, J. and Bowden, M. (2010), Work Integrated Learning. A Guide to Effective Practice, Routledge, New York, NY.

Costley, C., Abukari, A. and Little, B. (2010), Literature Review of Work Based Learning: October 2008, Middlesex University, London.

Dadze-Arthur, A. and Mörth, A. (2020), “Zur zielgerichteten Einbindung des Lernorts Praxis in das Hochschulstudium”, Cendon, E., Wilkesmann, U., Maschwitz, A., Nickel, S., Speck, K. and Elsholz, U. (Eds), Wandel an Hochschulen? Entwicklungen der wissenschaftlichen Weiterbildung im Bund-Länder-Wettbewerb ‘Aufstieg durch Bildung: offene Hochschulen’, Waxmann, Münster, pp. 148-165.

Dadze-Arthur, A., Mörth, A. and Cendon, E. (2020), International Trailblazers: Work-Integrating Higher Education in Selected Higher Education Institutions in the US, England and Denmark. Results of an International Case Study Research Project, available at: http://nbn-resolving.de/urn:nbn:de:0111-pedocs-188708 (accessed 30 October 2020).

Denscombe, M. (2010), The Good Research Guide for Small-Scale Social Research Projects, 4th ed., Open University Press, Maidenhead.

Denzin, N. and Lincoln, Y. (2011), The SAGE Handbook of Qualitative Research, SAGE, Thousand Oaks.

Edwards, D., Perkins, K., Pearce, J. and Hong, J. (2015), Work Integrated Learning in STEM in Australian Universities: Final Report, Australian Council for Educational Research, Melbourne.

Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P. and Trow, M. (1994), The New Production of Knowledge. The Dynamics of Science and Research in Contemporary Societies, Sage Publications, London.

Lester, S. and Costley, C. (2010), “Work-integrating learning at higher education level: value, practice and critique”, Studies in Higher Education, Vol. 35 No. 5, pp. 561-575, doi: 10.1080/03075070903216635.

Lester, S., Bravenboer, D. and Webb, N. (2016), “Work-integrated degrees: context, engagement, practice and quality”, A Literature Review for QAA: September 2016, QAA, London.

Lillis, F. and Bravenboer, D. (2020), “The best practice in work-integrated pedagogy for degree apprenticeships in a post-viral future”, Higher Education, Skills and Work-Based Learning, Vol. 10 No. 5, pp. 727-739, doi: 10.1108/HESWBL-04-2020-0071.

Majumdar, S.R. (2008), “Using the survey as an instrument of inquiry in research”, in Yang, K. and Miller, G. (Eds), Handbook of Research Methods in Public Administration, Taylor & Francis Group, Boca Raton, FL.

Merriam, S. and Bierema, L. (2014), Adult Learning. Linking Theory and Practice, Jossey-Bass, San Francisco.

Mörth, A., Schiller, E., Cendon, E., Elsholz, U. and Fritzsche, Christin (2018), “Theorie und Praxis verzahnen in Studienangeboten wissenschaftlicher Weiterbildung. Ergebnisse einer fallübergreifenden Studie”, available at: http://nbn-resolving.de/urn:nbn:de:0111-pedocs-157116 (accessed 30 October 2020).

Northrop, A. and Arsneault, S. (2008), “Sampling and data collection”, Yang, K. and Miller, G. (Eds), Handbook of Research Methods in Public Administration, Taylor & Francis Group, Boca Raton, Florida, FL, pp. 213-239.

Nottingham, P. (2012), An Exploration of How Differing Perspectives of Work Based Learning within Higher Education Influences the Pedagogies Adopted, (Ph.D theses) University of London, Birkbeck.

Nottingham, P. (2017), “Re-evaluating work-based learning pedagogy”, Higher Education, Skills and Work-Based Learning, Vol. 7 No. 2, pp. 129-140, doi: 10.1108/HESWBL-11-2015-0057.

Portelli, J.P. (1993), “Exposing the hidden curriculum”, Journal of Curriculum Studies, Vol. 25 No. 4, pp. 343-358, doi: 10.1080/0022027930250404.

Quality Assurance Agency for Higher Education (2018), UK Quality Code for Higher Education: Advice and Guidance – Work-Based Learning, QAA, Gloucester.

Raelin, J.A. (2007), “Toward an epistemology of practice”, The Academy of Management Learning and Education, Vol. 6 No. 4, pp. 495-519.

Rønnow-Rasmussen, T. and Zimmerman, M.J. (Eds) (2005), Recent Work on Intrinsic Value, Library of Ethics and Applied Philosophy, Springer, Vol. 17.

Schön, D.A. (1983), The Reflective Practitioner. How Professionals Think in Action, Basic Books, New York, NY.

Schön, D.A. (1987), Educating the Reflective Practitioner, Basic Books, New York, NY.

Talbot, J. (2017), “Curriculum design for the post-industrial society: the facilitation of individually negotiated higher education in work based learning shell frameworks in the United Kingdom”, in Nata, R.V. (Ed.), Progress in Education, Nova Science Publishers, New York, NY, Vol. 44, pp. 127-161.

Töllner, A. (2010), “Modelle und Modellierung”, in Bandow, G. and Holzmüller, H. (Eds), Das Ist Gar Kein Modell!, Gabler, Wiesbaden, pp. 3-21, doi: 10.1007/978-3-8349-8484-5_1.

Trochim, W.M. (2005), The Research Methods: The Concise Knowledge Base, Atomic Dog Publishing, Cincinnati, OH.

Wall, T. (2013), “Diversity through negotiated higher education”, Bridger, K., Reid, I. and Shaw, J. (Eds), Inclusive Higher Education: An International Perspective on Access and the Challenge of Student Diversity, Libri Publishing, Middlesex, pp. 87-98.

Wall, T. and Hindley, A. (2019), “Work-integrated learning for sustainability education”, in Leal Filho, W. (Ed.), Encyclopedia of Sustainability in Higher Education, Springer, Cham. doi: 10.1007/978-3-319-63951-2.

Wenger, E. (1998), Communities of Practice: Learning, Meaning, and Identity, Cambridge University Press, Cambridge.

Acknowledgements

This study was funded by Germany’s Federal Ministry of Education and Research (Bundesministerium für Bildung und Forschung) as part of the wider research programme, which accompanied the joint Federal Government-Länder Competition “Advancement through Education: Open Universities”.

Corresponding author

Anita Mörth is the corresponding author and can be contacted at: anita.moerth@fernuni-hagen.de

About the authors

Dr Abena Dadze-Arthur is a research associate at the University of Hagen in Germany, and an associate at the University of Birmingham in the UK. Her research interests focus on the transferring and brokering of knowledge across and within institutional and cultural boundaries, as well as situated agency and cognition under conditions of change.

Anita Mörth is research associate at the University of Hagen in Germany. Her research interests focus on key concepts of current and future formations of continuing higher education, as well as conceptions of learning, gender and diversity.