Measuring the quality of workplace learning environments – a qualitative meta synthesis of employee questionnaires

Viola Deutscher (Business School, University of Mannheim, Mannheim, Germany)
Anke Braunstein (Business School, University of Mannheim, Mannheim, Germany)

Journal of Workplace Learning

ISSN: 1366-5626

Article publication date: 21 March 2023

Issue publication date: 18 December 2023

2863

Abstract

Purpose

This study aims to support researchers and practitioners in finding suitable instruments for future research studies and organizational quality assessments.

Design/methodology/approach

Employees’ success of learning at work is strongly influenced by the quality of the workplace learning environment. In the recent decades growing effort has been given to the development of surveys to measure the quality of workplace learning, resulting in a large number of available survey instruments. This study conceptually draws on a 3-P model and uses a qualitative metasynthesis to collect and categorize n = 94 surveys that intend to measure the quality of workplace learning (WPL).

Findings

The results underline that research on WPL environments is a highly interdisciplinary endeavor, where every discipline enriches the field by a new perspective and own foci. Overall, this study finds a focus on learning culture and working conditions, on social and functional inclusion of the learner and on support and feedback during training. Products of WPL such as professional competences or career aspirations play a minor role.

Originality/value

With the integration of quality measurement instruments from various research studies, this study produces an interactive online instrument map that gives a broad, yet organized overview of available quality measures in the WPL field.

Keywords

Citation

Deutscher, V. and Braunstein, A. (2023), "Measuring the quality of workplace learning environments – a qualitative meta synthesis of employee questionnaires", Journal of Workplace Learning, Vol. 35 No. 9, pp. 134-161. https://doi.org/10.1108/JWL-06-2022-0074

Publisher

:

Emerald Publishing Limited

Copyright © 2023, Viola Deutscher and Anke Braunstein.

License

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial & non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode


1. Premise

Learning through observation and imitation, as typical for workplace learning (WPL), can be regarded as the natural and historically earliest way of knowledge acquirement (see the concept of mimesis, Billett, 2014). Yet, traditionally, most research activities in the learning sciences have been directed toward learning at schools and universities, eventually for work but outside of the workplace, rather than on workers’ learning processes through everyday work activities. The rise of knowledge economies and the concept of lifelong learning have rendered the neglect of workplaces as potential learning sites apparent and currently spur a vivid rediscovery of workplaces as rich learning environments. Policy agendas hence call for the recognition of non-formal and informal learning at workplaces [e.g. Organisation for Economic Co-operation and Development (OECD), 2010] – often based on considerations of social justice but also as an argument of economic efficiency. As a result, to mobilize the available workforce, many countries are now considering WPL as a source of untapped potential for human prosperity (Werquin, 2009).

Contemporary WPL research has, on the one hand, indeed shown that learning occurs at workplaces and that it is an adjuvant in developing specific knowledge, abilities and generic working-life competences (Kyndt et al., 2014; Klotz et al., 2015; Deutscher and Winther, 2018), intuition and expertise (Harteis and Billett, 2013) and positive personal traits and attitudes toward work such as a vocational identity (Billett, 2007; Klotz et al., 2015). On the other hand, large differences have become apparent in the extent to which workplaces are indeed rich learning environments (Negrini et al., 2016) affecting competence development and drop-out rates from educational WPL programs.

The fact that not every workplace is per se a rich learning environment has led to political discussions about learning-related work conditions and international agendas aiming to improve WPL conditions (OECD, 2020). These differences between workplace learning environments have also spurred a growing interest among employers, policymakers and researchers into what comprises learning at the workplace and how it can be facilitated by adequate conditions. The growing interest in and awareness of the quality of WPL is reflected in various research disciplines and research areas ranging from Economics and Business (e.g. educational economics, human resource management and development) to Psychology (e.g. cognitive learning psychology, work and organizational psychology), Sociology (e.g. educational sociology, sociology of work) and, of course, Education (e.g. adult education, vocational education); in all of these areas, learning at and through work nowadays has become a major theme.

A crucial precondition for research on learning through work is a valid description and measurement of WPL environments. The interdisciplinary WPL research community has already generated a considerable range of measurement instruments, as numerous studies deal with the operationalization and analysis of the quality of WPL environments. Nevertheless, the search for and the development of adequate measurement instruments continues. Because of different theoretical scopes, discipline-specific vocabulary and varying (also regional) accessibility and context, the state of research on WPL characteristics can hardly be called interconnected. This disjointedness seems problematic on the grounds of three interrelated phenomena detrimental to the progress of WPL research:

  1. the existence of nonidentical but highly interrelated theoretical concepts;

  2. an unmanageable number of measurement instruments in various research disciplines, often resulting in a (re)construction of already existing scales; and

  3. a low comparability of empirical study results, which could, in part, be justified by different vocational fields and varying characteristics of workplaces but could also be attributed to different measurement approaches and instruments.

Those deficiencies – which are of naturally not exclusive to the WPL field but largely denote the WPL research landscape – might also explain why WPL research so far has mostly missed out on comprehensive meta-analytic contemplations (for rare exceptions, see e.g. Blume et al., 2010; Jones et al., 2016; Kleine et al., 2019) or at least comparative studies between different vocations (Kyndt and Beausart, 2017) [1].

We argue that it is key to the progress of WPL research to gain insight into the existing measurement instruments over different scientific disciplines. This insight is, in particular, key in gaining an interdisciplinary understanding of the effectiveness and efficiency of “workplaces” for learning processes. To this end, we provide an interdisciplinary overview of existing scales and items in survey instruments measuring characteristics of WPL environments. We focus here on the learner perception of WPL characteristics. To fulfill this goal, we identify and categorize with respect to their content accessible standardized test instruments from diverse disciplines and developed for different vocational settings. We then range the measures include in the instruments in a framework model for WPL learning (Tynjälä, 2013). Subsequently, we discuss the current measurement landscape with respect to common core areas and the statistical quality of the developed instruments (validation studies). The practical significance of this study lies in its support for researchers and practitioners in efficiently finding suitable instruments and avoiding unnecessary redevelopments when conducting new WPL studies. If, via such an overview, in form of a research instrument map, more instruments are used that systematically relate to the existing state of research, this use should, in the longer run, heighten the general comparability of study results and advance the feasibility of quantitative meta-analyses in the field.

2. Defining the quality of workplace learning environments

Defining the quality of WPL environments inevitably requires defining the three concepts of WPL, learning environments and quality.

  1. We define WPL as the acquirement of professional competence (domain-generic and -specific knowledge, skills and attitudes), allowing for proficient performance at a workplace. The different phases or levels of action in which skills are upgraded and knowledge and attitudes are acquired may reach from observation to planning and execution and finally to reflection of work tasks (Benner, 2004). Such a broad definition of WPL includes advanced workers’ learning as well as learning as preparing to enter an occupation. However, in contrast to the concept of work-based learning, where learning takes place “either in the workplace, in settings simulating the workplace or outside the workplace” (European Centre for the Development of Vocational Training [CEDEFOP], 2015, p. 7), WPL always refers to learning processes at, but not outside, the workplace (e.g. in school-based or university-based vocational educational settings). With respect to its nature, WPL occurs through everyday practices accompanied by work-related interactions resulting in the learning of the individual employee and potentially also in the learning of the organization as a whole (Fenwick, 2008a; Collin et al., 2011). It is therefore noticeable that WPL takes place in the social context of a “community of practice” (Lave and Wenger, 1991), grounded on the dynamics between “individual actors” and “collectives” (Fenwick, 2008b).

  2. For an understanding of workplaces as learning environments, a basic distinction between how workplaces afford or hinder opportunities for learning and how individuals elect to engage in activities and with the support and guidance provided by the workplace is crucial (Billett, 2001). This duality is the key determinant of the quality of learning in both formal and informal learning environments [2]. Workplaces differ substantially in how they support learning: “In particular, the readiness of the workplace to afford opportunities for individuals to participate in work activities and access direct and indirect support are key determinants in the quality of learning that arises from that participation” (Billett, 2001, p. 209). Such affordances of a WPL environment do not determine learning per se; they are mediated by the learners’ perception and elected use of the provided work environment. This perception is, in turn, premised on diverse individual learner characteristics (e.g. prior knowledge, individuals’ personal initiative, self-efficacy) (Hetzner et al., 2015).

  3. In recent years, a growing research effort has been put into identifying and describing central characteristics supportive of WPL (for an overview see Kyndt and Baert, 2013). At the same time, the relation between learning conditions in the workplace and learning outcomes has been increasingly empirically analyzed (Janssens et al., 2017), allowing a deeper insight into the genesis and quality of WPL. “Quality” in the context of WPL is, however, characterized by a lack of conceptual clarity (Van den Berghe, 1997) and different connotations of the term prevail. With this study’s aim of describing the quality of WPL environments, a functional quality approach seems preferable over a notion of quality as excellence (which according to Harvey and Green (1993) is exceptional and relatively difficult to attain). This rather neutral approach allows for “good” as well as for “bad” aspects of a workplace to be studied, therefore allowing the nature of WPL to be fully grasped. But, even with a functional definition as such, an operationalization of the term “quality” is still problematic, as it is always a matter of perspective. It can be argued that quality conceptions are likely to differ between stakeholders. For instance, in the workplace context, employees’ categories for describing the quality of WPL environments might differ from those of their employers. We focus on the presumed perspective of the learner, as WPL learning is primarily a process within the learner and because we wish to acknowledge the individually mediated nature of WPL. Hence, when operationalizing workplace conditions, the judgment of the individual seems crucial. This importance of the learner’s evaluation and interpretation component is, inter alia, expressed in Tynjälä (2013) adaption of the 3-P Model of quality (Biggs, 1999), which stresses individual perception (Figure 1). To categorize and rank various test instruments related to the quality of WPL environments, we draw on that model, which differentiates WPL quality based on presage, process and product factors. Such sequential trinity for describing quality is internationally and interdisciplinary acknowledged (Biggs, 1999; Marsick et al., 2011; Ballantine, 2002; De la Fuente and Justicia, 2007; Anselmann, 2022) and, most importantly, widely accepted for the field of WPL.

The 3-P framework model by Tynjälä (2013) conceptualizes WPL by focusing on the “perceived quality” of an individual and therefore fits the duality notion of WPL learning research best. Both the learner factors and the learning context are represented within the presage dimension. The process dimension covers the description of WPL characteristics during interactional processes, including the structure and performance of work tasks or the learner’s interaction with others. Finally, within the product dimension, all learning-related outcomes are summarized; these are mainly focused on the individual’s personal and professional development (Tynjälä, 2013).

Summing up, for this paper we define the quality of WPL environments as the subjective perception of presage and process characteristics of the workplace (affordances) that, in their interaction with individual learner characteristics, are conducive to certain learning outcomes (products). Given this broad definition and framework, it is impossible to cover the field of quality research in its entirety. Instead, this paper provides a general overview of current research on workplace environments and their presage, process and product dimensions from the learner point of view. Accordingly, the measurement instruments related to individual learner characteristics (e.g. prior knowledge, agency) are excluded – albeit acknowledging that, for empirical studies, it is crucial to model the influences of the individual learner characteristics and traits that are relevant for the respective study purpose and sample.

3. Method

While there is generally broad consensus about the distinction between presage, process and product dimensions for describing WPL environments, the key characteristics specifying those dimensions vary greatly between studies and their respective measurement instruments. To list and categorize existing instruments capturing characteristics of the WPL environments, the method of qualitative metasynthesis seems particularly suitable. It combines a systematic literature search with a qualitative content analysis, thus allowing research to be systematically and fully integrated. Qualitative metasynthesis, sometimes also referred to as qualitative meta-analysis, represents a relatively new and, to date, rarely chosen form of analysis (Eisend, 2014) that is, however, spreading to more and more research areas. Like all meta-analyses, it can be described as a systematic summary of published studies with the help of a set of fixed rules (Leary and Walker, 2018). Following Lipsey and Wilson (2001), similarly to conducting a quantitative meta-analysis, the findings of several studies are amalgamated to develop a comprehensive overall new model (see also Leary and Walker, 2018), only that, in this case, the database is qualitative in nature and – unlike in a regular systematic review – is extended through qualitative content analysis and the formation of categories. Hence, this procedure exceeds a classic systematic literature review by not just providing a summary and possibly discussion of the state of research; it contains a systematic qualitative content analysis aiming to inductively generate a novel and integrative model.

For our study, the qualitative data base consists of the test items used in employee surveys. In accordance with the process logic of metasynthesis (Jensen and Allen, 1996), the first two steps comprise literature research and literature selection. The core of a qualitative metasynthesis then consists of inductively determining qualitative categories (item analysis and categorization). Finally, an integrative model of categories is built as a result of this methodological process (model building). These steps structure the following paragraphs of our methods section and will be described in greater detail. Where possible within the qualitative metasynthesis framework, these steps were performed in accordance with the ‘Preferred Reporting Items for Systematic Reviews and Meta-Analyses‘ (PRISMA) recommendations (Liberati et al., 2009) to make the four blocks reproducible.

3.1 Literature research

To summarize the state of research into the measurement of WPL learning environments, we conducted a systematic literature search targeting various key words indicating learning at the workplace in combination with search terms indicating measurement instruments, resulting in 36 search combinations, displayed in Figure 2.

To exclude as few studies as possible (Jensen and Allen, 1996) to analyze a broad spectrum of WPL research, no limitations regarding publication type, profession, industry or country were set. We did, however, establish two general eligibility criteria. First, for the purpose of assembling an international online repository, studies and test instruments had to be published in the English language. Second, since the systematic recording of WPL using written survey instruments only began in the mid-1990s, the literature search was limited to publications from 1990 onwards. In addition, before the 1990s, WPL was often understood to involve only formal learning (Kyndt et al., 2014), leading to a limited focus of the developed instruments. With respect to the chosen information sources, this search strategy was implemented interdisciplinarily in five large databases in the fields of Education, Psychology, Economics and Business and Sociology [Education Resource Information Center (ERIC), PsycINFO, Business Source Premier, EconLit and Social Sciences Citation Index (SSCI)]. Following Walsh and Downe (2005), the keyword search was complemented by a snowballing procedure, marking relevant references of the identified studies, so as not to overlook relevant studies.

3.2 Literature selection

The initial search generated 69,514 results (including duplicates). A title and abstract analysis led to the majority of these hits being eliminated because they were related to a non-WPL context. After this elimination, 558 records were identified as fitting to the WPL context and another 29 records were added through snowballing (leading to a total of 587 records). The list of potential records was further reduced by removing duplicate texts, from both within and across the databases, resulting in 465 remaining records. For 31 studies, neither an abstract nor a full paper was accessible. This left 434 studies for the full text screening process. After screening the full articles, another 74 studies were excluded because of a misfit to the WPL context (360 studies remaining). In a second step, studies were eliminated if they either were only theoretically or conceptually founded or used no written survey (e.g. observation studies or interviews), focused solely on classroom or university settings instead of WPL focused on a point of view other than that of the employees – for instance, the perspective of training personnel – or focused exclusively on individuals’ personal learning characteristics at the workplace (e.g. agency, identity, learning types, mental or physical limitations) instead of the WPL environment. These exclusion criteria reduced the number of records to 360 studies. The reduction of search results to the included studies is comparatively drastic for two reasons. First, our original search string was broad to avoid overlooking relevant work in an interdisciplinary field. Second, most of the excluded studies mentioned the workplace but were not conducted in a WPL context (eligibility criterion 1) or did not contain empirical work in the form of a survey instrument (eligibility criterion 2).

Then the availability of the full study as well as the availability of the measurement instruments was checked for. For 77.22% of the studies, the underlying test instrument was neither included in an Appendix nor retrievable from supplemental materials or online resources. In such cases, the authors of the respective studies were contacted and asked if they wish to provide the test instruments for the analysis and/or the online repository. After a return time of eight weeks, 21.28% of the contacted authors responded by providing their instruments. Prior to the content analysis, it was necessary to identify instruments that had been used in more than one study to avoid multiple counting, which would have distorted the results. By checking for multiple uses, 92 different studies with 94 different test instruments could be extracted and formed the basis for the subsequent qualitative content analysis (Appendix Table A1 contains an overview of the included studies). The fact that there are more instruments than studies, even though some studies fall back on the same questionnaire, is because of the circumstance that many studies use more than one measurement instrument to describe the workplace environment.

3.3 Item analysis and categorization

To develop the WPL measurement model and the list of measurement instruments, all items of all identified measurement instruments were recorded in an Excel spreadsheet. This data collection resulted in 3,688 test items. These items were connected and synthesized by inductively determining categories based on Mayring’s (2000) qualitative content analysis, including generalization, selection and bundling. First, all items were collected from the measurement instruments in tabular form. Second, since the focus of this metasynthesis lies in measuring WPL environments, all irrelevant test items were excluded. This means that particularly individual factors (personal data or demographic factors, personal biography or personal traits) were excluded because they are not part of the learning environment itself, even though they almost certainly influence WPL and are collected in almost all measurement instruments. Then, every item was categorized separately on the basis of specific contents and classified within a general dimension of the 3-P model (Tynjälä, 2013). Within these dimensions, items with similar contents were grouped and subcategories were formed. Subsequently, keywords were assigned to categories, allowing a reasonable summary of the contents. A codebook for the keywords and categories was developed. To ensure and indicate the reliability of the item mapping the data was coded twice by two independent individuals[3] (intercoder reliability = 0.934). An exemplary assignment is shown in Appendix Table A2.

4. Results

The distribution of original hits to finally available test instruments can be retrieved from Table 1, reported separately for the different scientific disciplines.

The instrument output was highest for the disciplines of Education (ERIC) and Psychology (PsychINFO) (46 instruments each), followed by a significantly lower output for the Economics and Business database (BSP and EconLit) (13 instruments) and the Sociology database (SSCI) (12 instruments). Regarding the quality of the identified instruments, 81.08% of the instruments report reliability analyses (most often through a reporting of Cronbach’s alpha), while a lower percentage (63.06%) of the identified measurement instruments was additionally validated structurally through factor analysis. Advanced factor analyses were most common for studies identified through Sociology (100% within this database), followed by Psychology (71.79%) and Economic and Business databases (61.54%). In ERIC, 45.45% of the identified test instruments were structurally validated through factor analysis. Over all disciplines, in total, 94 test instruments containing 3,688 items could be analyzed. The resulting map consists of three dimensions, seven content areas and 22 categories. For space/design reason, the keyword catalog contains another 66 keywords. The created content areas and categories can be found in Figure 3, and the keyword catalogue is displayed in the Appendix Table A3 [4].

The presage dimension comprises the employees’ perception of general factors of their work environment that existed prior to or independent of their interaction in the company. Usually organizational conditions are largely determined for the employee, although they may significantly influence the further individual processes and outcomes of WPL. For the purpose of our analysis, it was useful to distinguish within the content area of Organizational Conditions between general Framework Conditions (e.g. number of employees, industry or city, profitability), Working Conditions (e.g. work status, vacation days, payments and rewards policy, safety standards) and Learning Culture (e.g. perception of learning atmosphere, shared views on learning, enforcement of common learning standards).

The process dimension targets aspects that shape WPL through an interaction of employees with their work and with others. Here, work tasks can be complex and meaningful in different ways. Furthermore, social interaction through collegiality and cooperation as well as pedagogical inclusion through support and feedback may vary and belong to this dimension. Correspondingly, the analysis of item content for the process dimension yielded the three major content areas Work Tasks, Social Interaction and Pedagogic Interaction. Within the first area, the following categories were determined as intuitive in assigning the items: Workload (e.g. time pressure, physical stress), Variation of Tasks (e.g. degree of reiteration of tasks, possibility of knowledge expansion), Autonomy of Tasks (e.g. decision-making freedom, scope of action), Meaningfulness of Tasks (e.g. relevance for the company, relevance beyond the company), Complexity of Tasks (e.g. difficulty of tasks, information availability) and the Work Requirements (e.g. skill fit for tasks, degree of challenge). The Social Interaction content area was subdivided into the categories Involvement in Expert Culture (e.g. appreciation and codetermination among colleagues), Functional Inclusion (e.g. collaboration on tasks) and Social Inclusion (e.g. friendships, sense of belonging). The third content area comprised Methods/Activities (e.g. learning and training methods such as e-learning), Training Resources (e.g. fixed learning times, learning budget), Support (e.g. Coaching, Mentoring) and Feedback (e.g. reflection, performance appraisal).

Finally, all items that could be interpreted, as the results of WPL were included in the product dimension. The subdivision of the dimension was adapted according to a conceptual suggestion by Kyndt et al. (2014). Correspondingly, a distinction was made between Generic, Job-Specific and Organizational-level outcomes of WPL. The categories describe the effects of employees’ learning experiences and mainly deal with the results employees expect in terms of their lives, their jobs and their companies. Generic outcomes can refer to acquiring Professional Competences (e.g. generic knowledge, occupational skills) or the employees Satisfaction with Learning Opportunities, sometimes in the sense of self-fulfillment. Job-specific outcomes may refer to Occupational Identity or to Occupational Career Aspirations (e.g. further education). The category Organizational-level outcomes target employees’ Organizational Identity and Organizational Career Aspirations (e.g. promotion). Via this identified empirical model, readers can be guided online to the respective measurement instruments covering their category or categories of interest (see Excel File). Here, they can find possible items of interest, the instrument name and bibliographical information of the studies that developed or used the respective item. Moreover, a search and filter of the WPL measurement inventory by dimension, by content area, by category, by study or finally by instrument is available.

Within this model, the frequency of study coverage of the different categories can be illustrated. Figure 4 gives an overview of the frequency of category usage in the different studies.

Here, clear foci are identifiable. For example, with respect to the presage dimension, there seems to be a strong focus on Learning Culture and on Working Conditions whereas in the process dimension, most attention is given to Functional and Social Inclusion while the analysis of the Meaningfulness of Tasks and Training Resources is relatively underrepresented. In the product dimension, the category of Occupational Identity is most prevalent while Occupational Career Aspirations and Professional Competences play a minor role.

Moreover, there seem to be differences in the coverage of categories dependent on the scientific discipline. Appendix Table A4 gives a numerical overview of the category coverage for studies stemming from the four different disciplinary contexts. The results display how the different disciplines take different interests in the topic of WPL. Educational studies take a particular interest in the concept of Learning Culture, Involvement in Expert Culture and in Methods and Activities during the learning process. Psychological studies show less interest in the presage dimension, taking instead a stronger interest in Social Inclusion and in the Work Requirements, Satisfaction with Learning Opportunities and Occupational Career Aspirations. Meanwhile, the focus in the discipline of Economics and Business is primarily on Working Conditions and professional outputs such as Professional Competences and Organizational Identity. Finally, Sociological studies are mostly concerned with Workload and Work Tasks, as well as with Functional Inclusion, Occupational Identity and Career Aspirations.

5. Discussion

The study collects and categorizes n = 94 measurement instruments on the quality of WPL environments from the perspective of the learner via a qualitative metasynthesis and integrates the identified 3,688 items into a research instrument map. This map consists of three dimensions, seven content areas, 22 categories and 66 keywords that characterize contemporary efforts to measure the quality of WPL environments. The categories and contents identified through our instrument analysis resemble the content dimensions identified in an integrative literature review on informal workplace learning by Jeong et al. (2018), who were looking empirically at what factors influence informal learning in the workplace. An analysis of our research instrument map allows insights into the quality and foci of WPL research instruments, differentiated for the research disciplines investigated. Here, first of all, it becomes apparent that the research on the quality of WPL environments must be characterized as a highly interdisciplinary endeavor, where every discipline enriches the field by a new perspective: while Education takes a didactic perspective with a focus on Learning Cultures, Methods and Activities, Psychology puts a focus on learner-side dispositions that interact with the work environment. The discipline of Economics and Business further contributes to the field by taking a relatively institutional perspective, targeting amongst other things Working Conditions and outputs that are of particular importance from an institutional perspective. This finding of a difference in perspectives is in line with the results of a scoping review by Brandi and Iannone (2021). It suggests that studies from the Human Resources field have chiefly focused on workplace learning (as a strategic means) for developing workforce abilities, as a form of capital to maximize business performance, while WPL studies tend to take a relatively deep interest in understanding the mechanisms between workplaces and different forms of learning processes (as an end). Sociology finally adds to the shared research field by analyzing socialization processes in and through work environments with a particular focus on the nature of Work Tasks and on influences on Occupational Identities and biographies. When analyzing the quantity and quality of the identified instruments, it is noteworthy that the disciplines of Education and Psychology have produced the highest output of measurement instruments in quantitative terms while instruments stemming from Sociology most often administer advanced validation procedures to report on the quality of the developed measures, followed by studies from Psychology and Economics and Business. This result may be explained by different reporting customs in the respective disciplinary contexts.

A glance at the research foci over all disciplines shows that in the input dimension there seems to be a strong focus on Working Conditions and Learning Culture whereas, in the process dimension, most attention is given to forms of Functional and Social Inclusion, to Involvement in Expert Culture and to the categories Support and Feedback. Meanwhile the analysis of tasks (meaningfulness and complexity) and Training Resources are underrepresented. These finding can be in part explained by the greater number of studies from the disciplines of Education and Psychology, compared to Sociology and Economics and Business. It is moreover noteworthy that the coverage of the whole product dimension (with the exception of the concept of Occupational Identity) is underrepresented. Here, the results are in line with – or may indeed explain – the phenomenon that there are comparatively few research findings on the learning outcomes of WPL (Janssens et al., 2017; Kyndt et al., 2014). This dearth can be attributed to a low number of studies and measurement instruments that cover this quality dimension of WPL, as our results show. While the disciplines of Sociology and Economics and Business are relatively active regarding the product dimension, these disciplines take a smaller share in WPL research compared to that of Education and Psychology, where the focus seems to lie primarily in the presage and process dimension.

Another aspect that came up when analyzing the product dimension is that quantifiable outputs of workplace learning, such as financial income, are largely neglected as outputs even though they might play a crucial role in the learning process. This aspect indicates that our overview can hardly show what is obscure in the research on the quality of WPL environments. Therefore, the study presented here is constrained by the identification, selection and aggregation of WPL characteristics that constitute former research activities. The generated research map cannot – and does not intend to – claim normative completeness. Moreover, since this paper aimed to operationalize WPL, no assumptions can be made about the influence of one factor on another. For example, despite the large number of assigned test items in the category Learning Culture, it cannot be concluded that a good learning climate and culture automatically lead to increased WPL. Such assumptions must be confirmed by quantitative meta-analyses. The large number of items in this category only reflects the fact that, within the 94 different test instruments, there are many test items related to Learning Culture. Here, the created measurement model of WPL in environments can serve as an overview and measurement instrument repository for future research to investigate relationships between the different identified categories of WPL environments.

Further, the results are influenced by several methodological issues we encountered during data collection and analysis. Collecting the different test instruments was time consuming, taking 26 months for one scientific assistant along with two student assistants. This length of time was mostly because of a large number of search results containing numerous duplicates of studies and test instruments and because of unavailability of instruments. Even though, where necessary, we contacted authors to retrieve as many instruments as possible, the studies and/or the test instruments were not always made available so that our results only depict the accessible instrument landscape. Moreover, the focus of this work was on written forms of employee questionnaires, which constitute the most frequent study approach in WPL (Berings et al., 2006). Therefore, no fundamental statements on the operationalization of learning conditions in the context of WPL for other research methods (e.g. via interviews) and actors other than the learner (e.g. training personnel) can be deduced from the analysis presented here.

Moreover, our analysis, which aimed at a quality and instrument map from the learner perspective (learner surveys), is bound to only one perspective on WPL quality. A metasynthesis on employers’ or other stakeholders’ perceptions of quality will possibly identify different measurement instruments and potentially also other quality categories. Here, we wish to note that quality aspects of workplace learning and particularly learning outcomes should ideally also be measured through multiple stakeholders, particularly to avoid single-source biases in cross-sectional and longitudinal analyses of learning processes. Additionally, wherever feasible, objective measurement approaches, e.g. tests for assessing competences, should be used to contrast self-report measures.

Finally, because of the focus of this paper, which is on in-company learning conditions, the individual learner characteristics and traits, such as intelligence quotient (IQ) or motivation, were excluded. However, during the categorization of the test instruments, it became apparent that, while differentiating between an individual’s traits and the workplace surrounding is theoretically straight forward, this simplicity is not always reflected when formulating the items. Instead, one item may contain individual traits as well as an evaluation of the work surrounding to provide a relative assessment of the work environment. Those items were then still included in the inventory and occur particularly often for the category Work Requirements, where learners were often asked for the work requirements in relation to their personal ability level. This combination can be interpreted as an instance where the individual and his or her perception of the surroundings are, in practice, closely aligned. This and the fact that previous research has shown that motivation influences learning outcomes and that, conversely, the organization can influence motivation (Wielenga-Meijer et al., 2010) indicate that including personal learner characteristics seems absolutely essential to fully understand the interactions between working individuals and their work environment. Furthermore, this interaction is less linear than depicted in the research map presented here, where the visualization has been simplified for the sake of a clearer overview.

Acknowledging these limitations, we nevertheless presume that the generated research map displaying the instrument landscape gives a broad and interesting overview of current measurement instruments operationalizing the survey research on workplace learning quality. We moreover believe that it will prove useful for several purposes. First, the sheer amount of measurement instruments from various research disciplines now seems manageable via this metatool. The ensuing map helps researchers in finding suitable instruments when conducting new studies in this field, allowing them to avoid a (re)construction of already existing scales and items as well as facilitating a later comparison of empirical study results through the (interdisciplinarily) shared instruments. Concurrently, by providing an online version of the research instrument map as an interactive tool, practitioners (e.g. employers, staff managers, learning managers) can use the developed tool when assessing the quality of WPL in their organizations. For these purposes, the developed tool gives wide and organized access to the identified research instruments, showing the item content as well as leading the user back to the links where the original instruments can be found. Here, it is important to point out that single items should not be taken from the resulting item list without considering their associated scale. Within the test instruments, the test items often refer to or are based on each other. Researchers should therefore use the generated research instrument map as a systematic starting point for their instrument assembly and also always retrieve and engage with the corresponding original studies to avoid misinterpreting items and scales and to get a better grasp of the theoretical rationale underlying the respective measurement scales.

Supplementary material

The supplementary material for this article can be found online at www.bwl.uni-mannheim.de/media/Lehrstuehle/bwl/Deutscher/Interactive_Item_Map.xlsx

Figures

Three-p model of workplace learning

Figure 1.

Three-p model of workplace learning

Search string

Figure 2.

Search string

Measurement model of workplace learning environments

Figure 3.

Measurement model of workplace learning environments

Frequency of category usage throughout the identified studies

Figure 4.

Frequency of category usage throughout the identified studies

Literature and test instrument selection process and results

Database Hits Records Full paper Available relevant studies
with test instruments
Available relevant
test instruments
Education 22,032 192 184 39 46
Psychology 14,464 173 164 34 46
Economics and business 24,358 83 75 9 13
Sociology 8,631 110 101 12 12
Snowballing 29 29 29 25 25
Σ 69,514 587 553 119 142
Note:

The results of this table contain duplicates due to the utilisation of multiple databases

Overview of the included studies

Year ID Study Test instrument
1974 001 Hackman, J. R., and Oldham, G. R. (1974). The Job Diagnostic Survey: An Instrument for the Diagnosis of Jobs and the Evaluation of Job Redesign Projects. Yale University, Department of Administrative Sciences, New Haven Job Diagnostic Survey (JDS)1, a
1979 002 House, J. S., Wells, J. A., Landerman, L. R., McMichael, A. J., and Kaplan, B. H. (1979). Occupational Stress and Health among Factory Workers. Journal of Health and Social Behavior, 20(2), 139-160 Measures of Stressa
003 Mowday, R. T., Steers, R. M., and Porter, L. W. (1979). The Measurement of Organizational Commitment. Journal of Vocational Behavior, 14(2), 224-247 Organizational Commitment Questionnaire (OCQ)a,b
1985 004 Herold, D. M., and Parsons, C. K. (1985). Assessing the Feedback Environment in Work Organizations: Development of the Job Feedback Survey. Journal of Applied Psychology, 70(2), 290-305 Feedback Culturea,b
1989 005 Ashford, S. J., Lee, C., and Bobko, P. (1989). Content, Causes, and Consequences of Job Insecurity: A Theory-Based Measure and Substantive Test. The Academy of Management Journal, 32(4), 803-829 Job Insecurity Scale (JIS)a
1991 006 Owens, T., and Cohen, C. (1991). Northwest Entry-Level Worker Study. Portland: Northwest Regional Education Laboratory Northwest Entry-Level Worker Survey Tabulation
1993 007 Oregon State Economic Development Department (1993). Oregon Works: Assessing the Worker Training and Work Organization Practices of Oregon Employers Oregon Employer Survey
008 Morton, S. T. (1993). Socialization-related learning, job satisfaction and commitment for new employees in a federal agency. Dissertation, Virginia Polytechnic Institute and State University, Blacksburg Survey of new employee's experiences on the joba,b
1994 009 Taormina, R. J. (1994). The Organizational Socialization Inventory. International Journal of Selection and Assessment, 2(3), 133-145 The Organizational Socialization Inventory (OSI)a,b
010 Krueger, A., and Rouse, C. (1994). New Evidence on Workplace Education (NBER Working Paper 4831). Cambridge: National Bureau of Economic Research Skills Classes Survey
1996 011 Martineau, J. W. (1996). A contextual examination of the effectiveness of a supervisory skills training program. Dissertation, Pennsylvania State University, State College Frontline Leadership Studya,b
1997 012 Reio, T. G. Jr. (1997). Effects of curiosity on Socialization-related Learning and Job Performance in Adults. Dissertation, Virginia Polytechnic Institute and State University, Blacksburg Job Performance Surveya +
Workplace Adaptation Questionnaire (WAQ)a,b
1999 013 Edmondson, A. C. (1999). Psychological Safety and Learning Behavior in Work Teams. Administrative Science Quarterly, 44(2), 350-383 Learning Behavior in Work Teamsa,b
014 Mikkelsen, A., and Grønhaug, K. (1999). Measuring Organizational Learning Climate: A Cross-National Replication and Instrument Validation Study Among Public Sector Employees. Review of Public Personnel Administration, 19(4), 31-44 Learning Climate Questionnaire (LCQ)a,b
2001 015 Washington State Workforce Training and Education Coordinating Board (2001). Workforce Training Results: An Evaluation of Washington State's Workforce Development System. Olympia: Workforce Training and Education Coordinating Board Workforce Training: Needs and Practices of Employers in Washington State (1999)
016 Strickland, A., Simons, M., Harris, R., Robertson, I., Harford, M., and Edwards, A. (2001). Evaluating on- and off-the-job approaches to learning and assessment in apprenticeships and traineeships. Leabrook: NCVER Learning and assessment in traineeships and apprenticeships
2002 017 Smith, E., and Wilson, L. (2002). Learning and training in school-based new apprenticeships. Leabrook: NCVER Learning and training in school-based apprenticeships and traineeships
018 Ridoutt, L., Dutneall, R., Hummel, K., and Smith, C. S. (2002). Factors influencing the implementation of training and learning in the workplace. Leabrook: NCVER Implementation of learning and training in selected industries
2003 019 Kirby, J. R., Delva, M. D., Knapper, C., and Birtwhistle, R. V. (2003). Development of the approaches to work and workplace climate questionnaires for physicians. Evaluation and The Health Professions, 26(1), 104-121 Workplace Climate Questionnaire for physiciansa,b
020 Kirby, J. R., Knapper, C. K., Evans, C. J., Carty, A. E., and Gadula, C. (2003). Approaches to learning at work and workplace climate. International Journal of Training and Development, 7(1), 31-52 Workplace Climate Questionnaire (WCQ)2,a,b
021 Marsick, V. J., and Watkins, K. E. (2003). Demonstrating the Value of an Organization's Learning Culture: The Dimensions of the Learning Organization Questionnaire. Advances in Developing Human Resources, 5(2), 132-151 Dimensions of the Learning Organization Questionnaire (DLOQ)3,a,b
022 Van Yperen, N. W., and Hagedoorn, M. (2003). Do High Job Demands Increase Intrinsic Motivation or Fatigue or Both? The Role of Job Control and Job Social Support. The Academy of Management Journal, 46(3), 339–348 Job Demands and Motivationa,b
2004 023 Steelman, L. A., Levy, P. E., and Snell, A. F. (2004). The Feedback Environment Scale: Construct Definition, Measurement, and Validation. Educational and Psychological Measurement, 64(1), 165-184 Feedback Environment Scale (FES)a,b
2005 024 Clarke, N. (2005). Workplace Learning Environment and its Relationship with Learning Outcomes in Healthcare Organizations. Human Resource Development International, 8(2), 185-205 National survey of specialist palliative care healthcare organizations (hospices) in the UK
025 Lohman, M. C. (2005). A survey of factors influencing the engagement of two professional groups in informal workplace learning activities. Human Resource Development Quarterly, 16(4), 501-527 Informal Workplace Learning Survey4,a
2006 026 Lohman, M. C. (2006). Factors influencing teachers' engagement in informal learning activities. Journal of Workplace Learning, 18(3), 141-156 Informal Workplace Learning Survey4,a
027 Morgeson, F. P., and Humphrey, S. E. (2006). The Work Design Questionnaire (WDQ): Developing and Validating a Comprehensive Measure for Assessing Job Design and the Nature of Work. Journal of Applied Psychology, 91(6), 1321-1339 Work Design Questionnaire (WDQ)a,b
028 Kersley, B., Alpin, C., Forth, J., Bryson, A., Bewley, H., Dix, G., and Oxenbridge S. (2006). Inside the workplace: Findings from the 2004 Workplace Employment Relations Survey. London: Routledge Workplace Employment Relations Survey (WERS)
2007 029 Chang, S.-C., and Lee, M.-S. (2007). A study on relationship among leadership, organizational culture, the operation of learning organization and employees' job satisfaction. The Learning Organization, 14(2), 155-185 Learning Organizationa,b
030 Hicks, E., Bagg, R., Doyle, W., and Young, J. D. (2007). Canadian accountants: Examining workplace learning. Journal of Workplace Learning, 19(2), 61-77 Learning Barriers Survey +
Learning Facilitators Survey
031 Schmidt, S. W. (2007). The Relationship Between Satisfaction with Workplace Training and Overall Job Satisfaction. Human Resource Development Quarterly, 18(4). 481-498 Job Training and Job Satisfaction Survey (JTJSS)a,b
032 Jennings, M. E. (2007). Adult learning in a workplace setting: Key Factors Associated with the Development of Performance and Efficacy. Dissertation, Ann Arbor: ProQuest LLC Job Satisfaction Scalea,b
2008 033 Chiva, R., and Alegre, J. (2008). Emotional intelligence and job satisfaction: The role of organizational learning. Personnel Review, 37(6), 680-701 Organizational Learning Capability (OLC) Questionnairea,b
034 Vazsonyi, A. T., and Snider, B. J. (2008). Mentoring, competencies, and adjustment in adolescents: American part-time employment and European apprenticeships. International Journal of Behavioral Development, 32(1), 46-55 Mentoring Behaviora,b
035 Doornbos, A. J., Simons, R.-J., and Denessen, E. (2008). Relations Between Characteristics of Workplace Practices and Types of Informal Work‐Related Learning: A Survey Study Among Dutch Police. Human Resource Development Quarterly, 19(2), 129-151 Learning from Police Work Questionnaire (LPWQ)a,b
036 Berg, S. A., and Chyung, S. Y. (2008). Factors that influence informal learning in the workplace. Journal of Workplace Learning, 20(4), 229-244 Job Tasks +
Informal Workplace Learning Survey4,a,b +
Dimensions of the Learning Organization Questionnaire (DLOQ)3,a,b
037 Cooney, R., and Long, M. (2008). Inter-firm cooperation in training. A National Vocational Education and Training Research and Evaluation Program Report. Adelaide: NCVER Inter-firm Cooperation in Training
038 Deilkås, E. T., and Hofoss, D. (2008). Psychometric properties of the Norwegian version of the Safety Attitudes Questionnaire (SAQ), Generic version (Short Form 2006). BMC Health Services Research, 8(1), 191 Safety Attitudes Questionnaire (SAQ) Generic version (Short Form 2006)a,b
2009 039 Burgard, S. A., and Ailshire, J. A. (2009). Putting Work to Bed: Stressful Experiences on the Job and Sleep Quality. Journal of Health and Social Behavior, 50(4), 476-492 Job Strains and Sleep Quality
040 Taris, T. W., and Schreurs, P. J. G. (2009). Explaining worker strain and learning: How important are emotional job demands? Anxiety, Stress and Coping, 22(3), 245-262 Emotional Job Demandsa,b
041 Kyndt, E., Dochy, F., and Nijs, H. (2009). Learning conditions for non-formal and informal workplace learning. Journal of Workplace Learning, 21(5), 369-383 Questionnaire on learning conditions for non-formal and informal workplace learninga,b
042 DeSpain, L. (2009). The Relationship between Police Supervisor Training and Job Satisfaction Levels as reported by Patrol Officers. Dissertation, Ann Arbor: ProQuest LLC Job in General Scale (JIG) (1997 Revision)5,a +
Job Descriptive Index (JDI)6,a
043 Schalk, R. and van Woerkom, M. (2009). Does age influence relationship between learning opportunities at work and employee wellbeing and mobility? Proceedings of the ECER VETNET Conference 2009: Papers presented for the VETNET programme of ECER 2009 “Theory and Evidence in European Educational Research”, Vienna, September 28–30 Learning Opportunities
2010 044 Yamazaki, Y., and Kayes, D. C. (2010). Learning and work satisfaction in Asia: A comparative study of Japanese, Chinese and Malaysian managers. The International Journal of Human Resource Management, 21(12), 2271-2289 Index of Job Satisfaction (IJS)7,a
045 European Social Survey ERI. (2010). European Social Survey (ESS). London: ESS ERIC Headquarters European Social Surveya,b
2012 046 Carpita, M., and Golia, S. (2012). Measuring the quality of work: The case of the Italian social cooperatives. Quality and Quantity, 46, 1659-1685 Indagine sulle Cooperative Sociali Italiane (ICSI, 2007)b
047 Liu, Y. C., Huang, Y.-A., and Lin, C. (2012). Organizational factors’ Effects on the Success of E-Learning Systems and Organizational Benefits: An Empirical Study in Taiwan. The International Review of Research in Open and Distance Learning, 13(4), 130-151 E-Learning in Organizational Contextsa,b
048 Martins, H., and Proença, T. (2012). Minnesota Satisfaction Questionnaire: Psychometric Properties and Validation in a Population of Portuguese Hospital Workers (FEP Working Papers 471). Porto: FEP Minnesota Satisfaction Questionnaire (MSQ) short version8,a,b
049 Westover, J. H. (2012). Comparative international differences in intrinsic and extrinsic job quality characteristics and worker satisfaction, 1989-2005. International Journal of Business and Social Science, 3(7), 1-15 General Social Survey (GSS)
2013 050 Kyndt, E., Raes, E., Dochy, F., and Janssens, E. (2013). Approaches to Learning at Work: Investigating Work Motivation, Perceived Workload, and Choice Independence. Journal of Career Development, 40(4), 271-291 Workplace Climate Questionnaire (WCQ)2,a,b
051 Hoekstra, B. (2013). Relating Training to Job Satisfaction: A Survey of Online Faculty Members. Dissertation, Northcentral University, San Diego Index of Job Satisfaction (IJS)7,a
052 Molino, M., Ghislieri, C., and Cortese, C. G. (2013). When work enriches family‐life: The mediational role of professional development opportunities. Journal of Workplace Learning, 25(2), 98-113 Professional Developmenta,b
053 Masier, D. J. (2013). An Exploratory Study of the Relationship between Self-Directed Learning and Senge's Five Disciplines Necessary to Become a Learning Organization: In a High-Tech Company. Dissertation, Ann Arbor: ProQuest LLC Organizational Self-directed Learning and the Learning Organizationa
054 Ariga, K., Kurosawa, M., Ohtake, F., Sasaki, M., and Yamane, S. (2013). Organization adjustments, job training and productivity: Evidence from Japanese automobile makers. Journal of the Japanese and International Economies, 27(C), 1-34 Skills Development in the Workplace
055 Hosie, P., Jayashree, P., Tchantchane, A., and Lee, B. S. (2013). The effect of autonomy, training opportunities, age and salaries on job satisfaction in the South East Asian retail petroleum industry. The International Journal of Human Resource Management 24(21), 3980-4007 Job Characteristics and Job Satisfactiona
2014 056 Nikolova, I., Van Ruysseveldt, J., De Witte, H., and Syroit, J. (2014) Work-based learning: Development and validation of a scale measuring the learning potential of the workplace (LPW). Journal of Vocational Behavior, 84(1), 1-10 Task-based Workplace Learning Scale10,a,b
057 Jason, E. (2014). Factors Affecting Employee Trust in Leadership. Dissertation, Ann Arbor: ProQuest LLC Brief Index of Affective Job Satisfaction (BIAJS)a +
Organizational Culture Survey (OCS)a
058 Hobdy, A. K. (2014). Workplace Learning: Relationships Among Perceived Learning Opportunities, Job Satisfaction and Commitment in Small Business. Dissertation, Ann Arbor: ProQuest LLC Dimensions of the Learning Organization Questionnaire (DLOQ)3,a,b+
Staying or Leaving Index (SLI)a,b +
Job Satisfaction Survey (JSS)a,b +
Training and Learning Opportunitiesa,b
059 Kim, M. K., Kim, S. M., and Bilir, M. K. (2014). Investigation of the Dimensions of Workplace Learning Environments (WLEs): Development of the WLE Measure. Performance Improvement Quarterly, 27(2), 35-57 Dimensions of the Learning Organization Questionnaire (DLOQ)3,a,b+ Workplace Learning Environments (WLEs)a,b
2015 060 Fang, P., Luo, Z., and Fang Z. (2015). What is the job satisfaction and active participation of medical staff in public hospital reform: A study in Hubei province of China. Human Resources for Health, 13, 34 Minnesota Satisfaction Questionnaire (MSQ)8,a,b
061 Preenen, P., Verbiest, S. Van Vianen, A., and Van Wijk, E. (2015). Informal learning of temporary agency workers in low-skill jobs: The role of self-profiling, career control, and job challenge. Career Development International, 20(4), 339-362 Job Challenge and Informal Learninga,b
062 Steffgen, G., Kohl, D., Reese, G., Happ, C., and Sischka, P. (2015). Quality of Work: Validation of a New Instrument in Three Languages. International Journal of Environmental Research and Public Health, 12, 14988-15006 The Measuring Quality of Work questionnaire (MQW)a,b
063 Maher, K. (2015). The Role of the Personality in the Advisory Relationship. Dissertation, Ann Arbor: ProQuest LLC Advisory Working Alliance Inventory (AWAI-S)a
064 Weinberg, F. J., Mulki, J. P., and Lankau, M. J. (2015). The impact of effort-oriented epistemological beliefs on mentoring support. Journal of Workplace Learning, 27(5), 345-365 Mentoring Supporta,b
065 Messmann, G., and Mulder, R. H. (2015). Conditions for apprentices’ learning activities at work. Journal of Vocational Education and Training, 67(4), 578-596 Apprentices Learning Activities at Work and School–Work Alignmenta,b
066 Yeh, H.-J. (2015). Job Demands, Job Resources, and Job Satisfaction in East Asia. Social Indicators Research, 121(1), 47-60 International Social Survey Programme (ISSP) Work Orientations III questionnaire (2005)9,a
067 Van Dellen, T., and Heidekamp, I. (2015). How Dutch employees experience freedom of learning for work. International Review of Education, 61(6), 735-759 Freedom of Learning for Worka,b
068 Zumrah, A. R. (2015). Examining the relationship between perceived organizational support, transfer of training and service quality in the Malaysian public sector. European Journal of Training and Development, 39(2), 143-160 Perceived Organizational Support and Transfer of Training Outcomesa,b
069 Beverborg, A. O. G., Sleegers, P. J. C., and Van Veen, K. (2015). Fostering teacher learning in VET colleges: Do leadership and teamwork matter? Teaching and Teacher Education, 48, 22-33 Leadership and Teamworka,b
070 Kim, J. (2015). The relationships among the learning transfer system, managers' creative learning transfer, and job performance. Dissertation, Texas A and M University, College Station Learning Transfer and Job Performancea,b
071 Fontana, R. P., Milligan, C., Littlejohn, A., and Margaryan, A. (2015). Measuring self-regulated learning in the workplace. International Journal of Training and Development, 19(1), 32-52 Self-Regulated Learning in the Workplace Questionnaire (SRLWQ)11,a,b
072 Milligan, C., Fontana, R. P., Littlejohn, A., and Margaryan, A. (2015). Self-regulated learning behaviour in the finance industry. Journal of Workplace Learning, 27(5), 387-402 Self-Regulated Learning in the Workplace Questionnaire (SRLWQ)11,a,b
2016 073 Tseng, L.-M., and Yu, T.-W. (2016). How can managers promote salespeople’s person-job fit? The effects of cooperative learning and perceived organizational support. The Learning Organization, 23(1), 61-76 Cooperative Learning and Support in the Organizationa,b
074 Suzan, Z. (2016). The Relationships among Job Satisfaction, Length of Employment, and Mentoring of Nursing Faculty. Dissertation, Walden University, Minneapolis Job Descriptive Index (JDI)6,a +
Job in General Scale (JIG) (1997 Revision)5,a
075 Hasson, H., von Thiele Schwarz, U., Holmstrom, S. Karanika-Murray, M., and Tafvelin, S. (2016). Improving organizational learning through leadership training. Journal of Workplace Learning, 28(3), 115-129 Dimensions of the Learning Organization Questionnaire (DLOQ) short version3,a,b
076 Yoo, S. J., and Huang, W. D. (2016). Can e-learning system enhance learning culture in the workplace? A comparison among companies in South Korea. British Journal of Educational Technology, 47(4), 575-591 Dimensions of the Learning Organization Questionnaire (DLOQ) short version3,a,b
2017 077 Naidoo, K. L., Van Wyk, J. M., and Adhikari, M. (2017). The learning environment of paediatric interns in South Africa. BMC Medical Education, 17(1), q Postgraduate Hospital Educational Environment Measure (PHEEM)a,b
078 Smith, E., Smith, A., Tuck, J., and Callan, V. (2017). Continuity and change: Employers' training practices and partnerships with training providers. Adelaide: NCVER Employer Survey
079 Janssens, L., Smet, K., Onghena, P., and Kyndt, E. (2017). The relationship between learning conditions in the workplace and informal learning outcomes: A study among police inspectors. International Journal of Training and Development, 21(2), 92–112 Informal Workplace Learning Outcomes Questionnairea,b +
Learning Conditions Questionnairea,b
080 Nafukho, F. M., Alfred, M., Chakraborty, M., Johnson, M., and Cherrstrom, C. A. (2017). Predicting workplace transfer of learning: A study of adult learners enrolled in a continuing professional education training program. European Journal of Training and Development, 41(4), 327-353 Transfer of Learning Instrumenta,b
081 Serrat, O. (2017). Building a Learning Organization. In O. Serrat (Ed.), Knowledge solutions: Tools, Methods, and Approaches to Drive Organizational Performance (pp. 57-67). Singapore: Springer Singapore Learning Organization Model
082 Jutz, R., Scholz, E., and Braun, M. (2017). International Social Survey Programme: ISSP 2015 - Work Orientations IV; Questionnaire Development. (GESIS Papers, 2017/7). Köln: GESIS International Social Survey Programme (ISSP) Work Orientations III questionnaire (2005)9
083 Kyndt, E., and Beausaert, S. (2017). How do conditions known to foster learning in the workplace differ across occupations? In J. E. Ellingson and R. A. Noe (Ed.), Autonomous learning in the workplace. (pp. 201–218). London: Routledge Informal Learning Conditions Questionnairea,b
084 Ferreira, A. I., Martinez, L. F., Lamelas, J. P., and Rodrigues, R. I. (2017). Mediation of job embeddedness and satisfaction in the relationship between task characteristics and turnover: A multilevel study in Portuguese hotels. International Journal of Contemporary Hospitality Management, 29(1), 248-267 Job Diagnostic Survey (JDS)1, a, b +
Job Embeddednessa,b +
Job Satisfactiona,b +
Turnover Intentionsa,b
2018 085 Jabeen, F., Friesen, H. L., and Ghoudi, K. (2018). Quality of work life of Emirati women and its influence on job satisfaction and turnover intention: Evidence from the UAE. Journal of Organizational Change Management, 31(2), 352-370 Quality of Work Life (QoWL)a,b
086 Malik, M. S., and Kanwal, M. (2018). Impacts of organizational knowledge sharing practices on employees’ job satisfaction: Mediating roles of learning commitment and interpersonal adaptability. Journal of Workplace Learning, 30(1), 2-17 Impacts of Organizational Knowledge-Sharing Practices (KSP)a,b
087 Hussain, S., and Soomro, F. Q. (2018). Role of Employee Training in Enhancing Perceived Performance through competencies in Services Industry: A Study of Pakistani Banking Sector. Pakistan Business Review, 20(1), 122-136 Role of Employee Training in Enhancing Employee Performance in Banking Sector of Pakistana
2019 088 Decius, J., Schaper, N., and Seifert, A. (2019). Informal workplace learning: Development and validation of a measure. Human Resource Development Quarterly, 30(4), 495-535 Informal Workplace Learning scale (IWL)a,b
089 Battistelli, A., Odoardi, C., Vandenberghe, C., Napoli, G. Di., and Piccione, L. (2019). Information Sharing and Innovative Work Behavior: The Role of Work-Based Learning, Challenging Tasks, and Organizational Commitment. Human Resource Development Quarterly, 30(3), 361-381 Challenging Tasks and Goal Orientationa,b+
Commitment Oriented Work Systemsa,b+
Commitment to Organizationsa,b+
Task-based Workplace Learning Scale10,a,b
090 Kortsch, T. and Kauffeld, S. (2019). Validation of a German Version of the Dimensions of the Learning Organization Questionnaire (DLOQ) in German Craft Companies. Zeitschrift für Arbeits- und Organisationspsychologie, 63(1), 15-31 Dimensions of the Learning Organization Questionnaire (DLOQ)3,a,b
091 Lewis, S. (2019). A Quantitative Explanatory Examination of Job Training, Job Satisfaction, and Turnover Intentions among U.S. Retail Grocery Employees. Dissertation, Capella University, Minneapolis Job Satisfaction Scale (JSS)a,b +
Turnover Intentions Scalea,b
092 Zahrah, N., Aziz, A., and Hamid, S. N. A. (2019). Workload and Work Engagement among Nurses in Public Hospitals: Moderating Role of Religious Spirituality. Asian Journal of Miltidisciplinary Studies, 7(6), 1–10 Workload and Work Engagementa,b
Notes:

1–11 test instruments used multiple times. aCarrying out a reliability test; bcarrying out a structural factor validity

Exemplary assignment of test items

Item Dimension Content area Category Keyword
How many hours a week do you generally work at this company? Presage Framework Job-specific conditions Working time/
overtime
Overall the climate of my workplace helps me learn Presage Learning
environment
Work Climate Working
atmosphere
In my organization, leaders share up-to-date information with employees about competitors, industry trends and organizational directions Process Social
interaction
Functional
inclusion
Transparency
I received a lot of guidance from experienced company members Process Pedagogic interaction Support Support from
superior/coworker
Do the skills you learned in class help you with your current job and/or would they help with future jobs at this company? Product Generic Professional competences Job-specific acquired skills/knowledge

Keyword catalog

Dimension Content area Category Keywords
Presage Organizational conditions Framework conditions • Number of employees
• Industry
• Subsidiaries
• City/county/state
• Business form
• Corporate success
• Trade union or staff association
Working conditions • Working time/overtime
• Leisure/holiday
• Payments/rewards
• Work status
• Job security
• Health, safety
Learning culture • Learning climate
• Learning culture
• Organization of learning
Process Work tasks Workload • Negative emotions
• Positive emotions
• Physical stress
• Psychological stress
• Time pressure
Variation of tasks • Variety in tasks and skills
• Possibility of knowledge expansion
Autonomy of tasks • Decision-making freedom
• Scope of action
• Freedom in method
• Freedom in planning
• Freedom in implementation
• Self-reliance
Meaningfulness of tasks • Responsible tasks
• Relevance for overall business, other departments, beyond the company
Complexity of tasks • Simplicity/difficulty of tasks
• Specialization
• Information availability
Work requirements • Fitting of abilities/skills
• Challenge
Social interaction Involvement in expert culture • Appreciation
• Codetermination
Functional inclusion • Work organization
• Transparency
• Collaboration
• Accountability
Social inclusion • Attention
• Open-mindedness
• Friendship/collegiality
• Sense of belonging
Pedagogic interaction Methods/activities • Learning and training methods/activities
• E. g. E-learning, on-the-job learning, off-the-job learning, learning companies, …
Training resources • Materials
• Time for learning
• Money for learning
Support • Support from superior/coworker
• Handling errors
• Coaching/mentoring
Feedback • Performance appraisal
• Self-reflection
• Reflection with others
Product Generic Professional competences • Generic acquired skills/knowledge
• Job specific acquired skills/knowledge
Satisfaction with learning opportunities • Summarizing judgement on learning opportunities
Job specific Occupational identity • Fit with the profession
• Interests
• Boredom
• Fun
• Satisfaction
• Re-election of the profession
Occupational career aspirations • Further education and career
• Influence of family/friends
• Desired occupation
Organizational level Organizational identity • Fit to the operation
• Re-election of the company
Organizational career aspirations • Promotion/advancement
• Continuing education/study
• Plans for the future
• Goals/wishes

Frequency and number of items, test instruments and studies per dimension, content area and category

Dimen-sion Content area Category A B C D E F G H
Presage Organi-zational conditions Framework conditions 133 3.61 21 22.34 25 27.17 E: 9 32.14
P: 5 18.52
E&B: 2 22.22
S: 1 8.33
Working conditions 298 8.08 44 46.81 47 51.09 E: 15 53.57
P: 14 51.85
E&B: 5 55.56
S: 7 58.33
Learning culture 416 11.28 39 41.49 42 45.65 E: 16 57.14
P: 10 37.04
E&B: 4 44.44
S: 5 41.67
Process Work tasks Workload 109 2.96 30 31.91 32 34.78 E: 7 25.00
P: 9 33.33
E&B: 4 44.44
S: 7 58.33
Variation of tasks 81 2.20 33 35.11 36 39.13 E: 10 35.71
P: 12 44.44
E&B: 2 22.22
S: 6 50.00
Autonomy of tasks 140 3.80 38 40.43 44 47.83 E: 11 39.29
P: 14 51.85
E&B: 4 44.44
S: 7 58.33
Meaningfulness of tasks 40 1.08 16 17.02 18 19.57 E: 3 10.71
P: 3 11.11
E & B: 2 22.22
S: 4 33.33
Complexity of tasks 67 1.82 28 29.79 30 32.61 E: 8 28.57
P: 8 29.63
E&B: 2 22.22
S: 5 41.67
Work requirements 119 3.23 37 39.36 39 42.39 E: 10 35.71
P: 14 51.85
E&B: 3 33.33
S: 5 41.67
Social interaction Involvement in expert culture 124 3.36 39 41.49 47 51.09 E: 16 57.14
P: 13 48.15
E&B: 4 44.44
S: 6 50.00
Functional inclusion 261 7.08 50 53.19 55 59.78 E: 17 60.71
P: 13 48.15
E&B: 5 55.56
S: 9 75.00
Social inclusion 223 6.05 50 53.19 54 58.70 E: 14 50.00
P: 19 70.37
E&B: 5 55.56
S: 8 66.67
Pedagogic interaction Methods/activities 345 9.35 36 38.30 37 40.22 E: 18 64.29
P: 7 25.93
E&B: 5 55.56
S: 1 8.33
Training resources 94 2.55 18 19.15 24 26.09 E: 11 39.29
P: 5 18.52
E&B: 2 22.22
S: 1 8.33
Support 180 4.88 40 42.55 48 52.17 E: 12 42.86
P: 16 59.26
E&B: 4 44.44
S: 7 58.33
Feedback 280 7.59 45 47.87 50 54.35 E: 13 46.43
P: 14 51.85
E&B: 2 22.22
S: 7 58.33
Product Generic Professional competences 104 2.82 15 15.96 15 16.30 E: 5 17.86
P: 4 14.81
E&B: 3 33.33
S: 0 0.00
Satisfaction with learning opportunities 63 1.71 22 23.40 22 23.91 E: 7 25.00
P: 7 25.93
E&B: 2 22.22
S: 2 16.67
Job specific Occupational identity 198 5.37 36 38.30 35 38.04 E: 10 35.71
P: 10 37.04
E&B: 3 33.33
S: 5 41.67
Occupational career aspirations 5 0.14 5 5.32 5 5.43 E: 0 0.00
P: 2 7.41
E&B: 0 0.00
S: 1 8.33
Organi-zational level Organizational identity 86 2.33 22 23.40 19 20.65 E: 2 7.14
P: 3 11.11
E&B: 2 22.22
S: 3 25.00
Organizational career aspirations 67 1.82 25 26.60 27 29.35 E: 6 21.43
P: 9 33.33
E&B: 2 22.22
S: 5 41.67
Notes:

A = Number of items per category (Ni = 3,688); B = Percentage of items per category (Ni = 3,688); C = Number of test instruments with recourse to category (Nt = 94); D = Percentage of test instruments with recourse to category (Nt = 94); E = Number of studies with recourse to category (Ns = 92); F = Percentage of studies with recourse to category (Ns = 92); G = Assignment of studies to databases with recourse to category; H = Percentage of studies per discipline

Notes

1.

Disputable is whether it is valid to compare workplace learning across different vocations and industries, given varying specific occupational and workplace-related characteristics. However, Janssens et al. (2017) argue that research in one area can give at least an idea of what it might look like in another area. While, Kyndt and Beausaert (2017) find in a study with nurses and military employees that workplace learning can be researched across different areas, differences must be expected.

2.

Formal learning is highly organized in terms of learning content, learning objectives, time and funding and is also intentional and usually leads to certification while informal learning tends not to be organized and is unintentional on the part of the learners (OECD, 2010).

3.

We would like to thank two student assistants of the project (Lisa Müller and Herbert Thomann) for their continuing help in the project during data collection and analysis.

4.

While some categories have up to seven keywords (e.g. framework conditions), others have only one (e.g. meaningfulness of tasks). This was primarily necessary to ensure an unambiguous assignment of items.

References

Anselmann, S. (2022), “Trainers’ learning conditions, informal and formal learning and barriers to learning”, Journal of Workplace Learning, Vol. 34 No. 8, pp. 742-764.

Ballantine, J. (2002), “Sociology of education: Open systems approach”, in Levinson, D.L., Cookson P.W. Jr. and Sadovnik, A.R. (Eds), Education and Sociology: An Encyclopedia, RoutledgeFalmer, New York, NY, pp. 599-604.

Benner, P. (2004), “Using the dreyfus model of skill acquisition to describe and interpret skill acquisition and clinical judgement in nursing practice and education”, Bulletin of Science, Technology and Society, Vol. 24 No. 3, pp. 188-199, doi: 10.1177/0270467604265061.

Berings, M.G., Doornbos, A.J. and Simons, P.-J. (2006), “Methodological practices in on-the-job learning research”, Human Resource Development International, Vol. 9 No. 3, pp. 333-363, doi: 10.1080/13678860600893557.

Biggs, J. (1999), Teaching for Quality Learning at University: What the Student Does, Society for Research into Higher Education and Open University Press, Buckhingham.

Billett, S. (2001), “Learning through work: Workplace affordances and individual engagement”, Journal of Workplace Learning, Vol. 13 No. 5, pp. 209-214, doi: 10.1108/EUM0000000005548.

Billett, S. (2007), “Exercising self through working life: Learning, work and identity”, in Brown, A. Kirpal, S. and Rauner, F. (Eds), Identities at Work. Technical and Vocational Education and Training: Issues, Concerns and Prospects, Vol5, pp. 183-210, Springer, Dordrecht, NL, doi: 10.1007/978-1-4020-4989-7_7

Billett, S. (2014), “Mimesis: Learning through everyday activities and interactions at work”, Human Resource Development Review, Vol. 13 No. 4, pp. 462-482, doi: 10.1177/1534484314548275.

Blume, B.D., Ford, J.K., Baldwin, T.T. and Huang, J.L. (2010), “Transfer of training: a meta-analytic review”, Journal of Management, Vol. 36 No. 4, pp. 1065-1105.

Brandi, U. and Iannone, R.L. (2021), “Approaches to learning in the context of work – workplace learning and human resources”, Journal of Workplace Learning, Vol. 33 No. 5, pp. 317-333.

Collin, K., Sintonen, T., Paloniemi, S. and Auvinen, T. (2011), “Work, power and learning in a risk filled occupation”, Management Learning, Vol. 42 No. 3, pp. 301-318, doi: 10.1177/13507610394411.

De la Fuente, J. and Justicia, F. (2007), “The dedepro model of regulated teaching and learning: Recent advances”, Electronic Journal of Research in Educational Psychology, Vol. 5 No. 3, pp. 535-564.

Deutscher, V. and Winther, E. (2018), “Instructional sensitivity in vocational education”, Learning and Instruction, Vol. 53, pp. 21-33, doi: 10.1016/j.learninstruc.2017.07.004.

Eisend, M. (2014), Metaanalyse, Hampp, München, DE, doi: 10.1688/9783866189751.

European Centre for the Development of Vocational Training (CEDEFOP) (2015), Work-Based Learning in Continuing Vocational Education and Training: Policies and Practices in Europe, Publications Office of the European Union, Luxembourg.

Fenwick, T. (2008a), “Understanding relations of individual-collective learning in work: a review of research”, Management Learning, Vol. 39 No. 3, pp. 227-243, doi: 10.1177/1350507608090875.

Fenwick, T. (2008b), “Workplace learning: Emerging trends and new perspectives”, New Directions for Adult and Continuing Education, Vol. 119, pp. 17-26, doi: 10.1002/ace.302.

Harteis, C. and Billett, S. (2013), “Intuitive expertise: theories and empirical evidence”, Educational Research Review, Vol. 9, pp. 145-157, doi: 10.1016/j.edurev.2013.02.001.

Harvey, L. and Green, D. (1993), “Defining quality”, Assessment and Evaluation in Higher Education, Vol. 18 No. 1, pp. 9-34, doi: 10.1080/0260293930180102.

Hetzner, S., Heid, H. and Gruber, H. (2015), “Using workplace changes as learning opportunities. Antecedents to reflection in professional work”, Journal of Workplace Learning, Vol. 27 No. 1, pp. 34-50.

Janssens, L., Smet, K., Onghena, P. and Kyndt, E. (2017), “The relationship between learning conditions in the workplace and informal learning outcomes: a study among police inspectors”, International Journal of Training and Development, Vol. 21 No. 2, pp. 92-112, doi: 10.1111/ijtd.12095.

Jensen, L.A. and Allen, M.N. (1996), “Meta-synthesis of qualitative findings”, Qualitative Health Research, Vol. 6 No. 4, pp. 553-560, doi: 10.1177/104973239600600407.

Jeong, S., Han, S.J., Lee1, J., Sunalai, S. and Yoon, S.W. (2018), “Integrative literature review on informal learning: Antecedents, conceptualizations, and future directions”, Human Resource Development Review, Vol. 17 No. 2, pp. 128-152.

Jones, R.J., Woods, S.A. and Guillaume, Y.R.F. (2016), “The effectiveness of workplace coaching: a meta-analysis of learning and performance outcomes from coaching”, Journal of Occupational and Organizational Psychology, Vol. 89 No. 2, pp. 249-277, doi: 10.1111/joop.12119.

Kleine, A.-K., Rudolph, C.W. and Zacher, H. (2019), “Thriving at work: a meta-analysis”, Journal of Organizational Behavior, Vol. 40 Nos 9/10, pp. 973-999, doi: 10.1002.job.2375.

Klotz, V.K., Billett, S. and Winther, E. (2015), “Promoting workforce excellence: formation and relevance of vocational identity for vocational educational training”, Empirical Research in Vocational Education and Training, Vol. 6, pp. 1-20, doi: 10.1186/s40461-014-0006-0.

Kyndt, E. and Baert, H. (2013), “Antecedents of employees’ involvement in work-related learning: a systemativ review”, Review of Educational Research, Vol. 83 No. 2, pp. 273-313, doi: 10.3102/0034654313478021.

Kyndt, E. and Beausart, S. (2017), “How do conditions known to foster learning in the workplace differ across occupations?”, in Ellingson, J.E. and Noe, R.A. (Eds), Autonomous Learning in the Workplace, Routledge, New York, NY, pp. 201-218. doi: 10.4324/9781315674131

Kyndt, E., Govaerts, N., Verbeek, E. and Dochy, F. (2014), “Development and validation of a questionnaire on informal workplace learning outcomes: a study among socio-educational care workers”, British Journal of Social Work, Vol. 44 No. 8, pp. 2391-2410, doi: 10.1093/bsjw/bct056.

Lave, J. and Wenger, E. (1991), Situated Learning: Legitimate Peripheral Participation, Cambridge University Press, Cambridge, UK, doi: 10.1017/CBO9780511815355.

Leary, H. and Walker, A. (2018), “Meta-analysis and meta-synthesis methodologies: rigorously piecing together research”, TechTrends, Vol. 62 No. 5, pp. 525-534, doi: 10.1007/s11528-018-0312-7.

Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., Gøtzsche, P.C., Ioannidis, J.P.A., Clarke, M., et al. (2009), “The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration”, BMJ, Vol. 339, doi: 10.1136/bmj.b2700.

Lipsey, M.W. and Wilson, D.B. (2001), Practical Meta-Analysis, Sage, Thousand Oaks.

Marsick, V.J., Watkins, K.E. and O'Connor, B.N. (2011), “Researching workplace learning in the United States”, in Malloch, M. Cairns, L. Evans, K. and O'Connor, B. N. (Eds), The Sage Handbook of Workplace Learning, Sage, Los Angeles, CA, pp. 198-209.

Mayring, P. (2000), “Qualitative content analysis”, Forum: Qualitative Social Research, Vol. 1 No. 2, p. 20.

Negrini, L., Forsblom, L., Gurtner, J.-L. and Schumann, S. (2016), “Is there a relationship between training quality and premature contract terminations in VET? ”, Vocations and Learning, Vol. 9 No. 3, pp. 361-378, doi: 10.1007/s12186-016-9158-3.

OECD (2020), “Improving evidence on VET: comparative data and indicators (OECD social”, Employment and Migration Working Papers 250). Paris, Organisation for Economic Co-operation and Development, avilable at: www.oecd-ilibrary.org/docserver/d43dbf09-en.pdf?expires=1604404367&id=id&accname=guest&checksum=B088620B8C1089E265566E85E5AFC853

Organisation for Economic Co-operation and Development (OECD) (2010), Learning for Jobs: Synthesis Report of the OECD Reviews of Vocational Education and Training, OECD, Paris.

Tynjälä, P. (2013), “Toward a 3-P model of workplace learning: a literature review”, Vocations and Learning, Vol. 6 No. 1, pp. 11-36, doi: 10.1007/s12186-012-9091-z.

Van den Berghe, W. (1997), Indicators in Perspective. The Use of Quality Indicators in Vocational Education and Training, European Centre for the Development of Vocational Training (CEDEFOP), Thessaloniki.

Walsh, D. and Downe, S. (2005), “Meta-synthesis method for qualitative research: a literature review”, Journal of Advanced Nursing, Vol. 50 No. 2, pp. 204-211, doi: 10.1111/j.1365-2648.2005.03380.x.

Werquin, P. (2009), “Recognition of non-formal and informal learning in OECD countries: an overview of some key issues”, REPORT Zeitschrift Für Weiterbildungsforschung, Vol. 3, pp. 11-23, doi: 10.3278/REP0903W011.

Wielenga-Meijer, E.G., Taris, T.W., Kompier, M.A. and Wigboldus, D.H. (2010), “From task characteristics to learning: a systematic review”, Scandinavian Journal of Psychology, Vol. 51 No. 5, pp. 363-375, doi: 10.1111/j.1467-9450.2009.00768.x.

Xu, Y. (2008), “Methodological issues and challenges in data collection and analysis of qualitative meta-synthesis”, Asian Nursing Research, Vol. 2 No. 3, pp. 173-183, doi: 10.1016/S1976-1317(08)60041-9.

Further reading

Sandelowski, M., Docherty, S. and Emden, C. (1997), “Qualitative metasynthesis: issues and techniques”, Research in Nursing and Health, Vol. 20 No. 4, pp. 365-371, doi: 10.1002/(sici)1098-240x(199708)20:4<365::aid-nur9>3.0.co;2-e.

Acknowledgements

This research was funded by the German Research Foundation (DFG), within the project ‘Competence Development through Enculturation’ (KL 3076/2-1).

Corresponding author

Viola Deutscher can be contacted at: viola.deutscher@uni-mannheim.de

Related articles