Adopting a socio-technical perspective to rethink the use of ICT in VNFIL

Abstract

Purpose

This paper aims to draw on a socio-technical perspective to explore how information and communication technology (ICT) supports the validation of non-formal and informal learning (VNFIL) in specific practices and arrangements.

Design/methodology/approach

The authors use qualitative content analysis to analyse 43 country reports of the “European inventory on validation” published by the European Centre for the Development of Vocational Training (Cedefop) in 2019.

Findings

This study conceptualises ICT supported VNFIL practices and arrangements from a socio-technical perspective. Given this, the paper finds that ICT is used mainly to provide information on validation practices and arrangements, store qualification registers online and identify and document learning outcomes. The use of ICT to assess and certify learning outcomes remains limited.

Originality/value

VNFIL is very rarely seen in a technical context. This paper contributes to a theoretical perspective and highlights the mutual interdependence of social and technical components. Furthermore, this study provides an overview of inasmuch ICT is currently used to support VNFIL practices and arrangements. Based on the results, validation researchers and practitioners can get inspiration on how to develop ICT supported VNFIL practices and arrangements further.

Citation

Fahrenbach, F. and Luomi-Messerer, K. (2022), "Adopting a socio-technical perspective to rethink the use of ICT in VNFIL", European Journal of Training and Development, Vol. 46 No. 1/2, pp. 1-21. https://doi.org/10.1108/EJTD-06-2020-0097

1. Introduction

Lifelong learning plays an essential role for policymakers (World Bank, 2003) and researchers alike (Brine, 2006). Owing to fast-changing technologies and demands at the workplace, it is at one’s core to be and remain competitive in a knowledge economy (Brine, 2006; World Bank, 2003), and therefore a priority within the European Union since the early 2000s (Commission of the European Communities, 2000). To increase the lifelong learning of its citizens, European Union member states established several policies and tools such as the European Qualifications Framework (EQF) to make qualifications across the member states comparable or the validation of non-formal and informal learning (VNFIL). VNFIL is a mechanism by which learning outcomes are made visible and valuable, regardless of how they have been obtained. VNFIL is defined as a “process of confirmation by an authorised body that an individual has acquired learning outcomes measured against a relevant standard” (Council of the European Union, 2012, p. 5). In this context, learning outcomes are “statements of what a learner knows, understands and is able to do on completion of a learning process, which is defined in terms of knowledge, skills and competences” (Council of the European Union, 2012, p. 5). VNFIL can lead to a qualification, a “formal outcome of an assessment and validation process which is obtained when a competent body determines that an individual has achieved learning outcomes to given standards” (Council of the European Union, 2008, p. 4) but is also often conducted without leading to a qualification. According to Werquin (2010), a qualification obtained through VNFIL may help integrate migrants into the labour market, gives a second chance to those who dropped out of formal education and increases the recognised skill level of low-skilled workers and fosters equity within society. VNFIL is a supporting mechanism that puts lifelong learning into practice.

Even though the European Commission provides detailed recommendations to support the VNFIL in vocational education and training (VET), higher education (HE) and the tertiary sector (Council of the European Union, 2006, 2012, 2017), most countries lack nationwide arrangements and report a scattered validation landscape in which sectoral arrangements and bottom-up practices prevail (Cedefop, European Commission and ICF, 2017, 2019). To ensure the trustworthiness and legitimacy of the VNFIL, reliable and standardised practices are crucial (Cedefop, 2015). One of the means to standardise practices is through information and communication technology (ICT) allowing the “electronic input, storage, retrieval, processing, transmission and dissemination of information” (Cedefop, 2014, p. 113). Supporting VNFIL through ICT entails cost efficiency through scaling effects, time effectiveness through (partially) automated testing, the possibility to store results for a long time, the adaptation of practices to different target groups (e.g. for VET, HE or the tertiary sector) and an improvement of validity and reliability (Luomi-Messerer, 2019b, pp. 31–33) but also raises concerns about the security of personal data and unsuitability for specific target groups (e.g. less technology- or internet-savvy individuals) (Luomi-Messerer, 2019b, pp. 34–35). According to the European Centre for the Development of Vocational Training (Cedefop), standardised tools “mainstream processes increase awareness of validation” (Cedefop, European Commission and ICF, 2017, p. 75) and are “central to the quality, legitimacy and acceptance of the validation process” (Cedefop, European Commission and ICF, 2019, p. 38) in society. However, Cedefop also notes, the use of standardised tools to support the VNFIL is “not widespread” (Cedefop, European Commission and ICF, 2017, p. 75) and more can be done in the “standardisation of tools and the use of ICT” (Cedefop, European Commission and ICF, 2017, p. 20) to support the VNFIL. In this vein, only 50% (N = 28) of European countries report the use of ICT-tools to support VNFIL in vocational education and training (Cedefop, European Commission and ICF, 2019, pp. 40–41). As legal arrangements are in place and the advantages of standardised practices are well-known, the development of ICT-supported practices is encouraged. To guide the development of standardised and ICT-supported practices, we have to know the “state-of-the-art” or, in other words, how ICT is currently used to support VNFIL practices and arrangements. However, to the best of our knowledge, we are not aware of a study that explores how ICT supports VNFIL in specific practices and arrangements. Consequently, our research question is:

How do information and communication technologies support practices and arrangements in the validation of non-formal and informal learning processes?

This paper aims to fill this gap and answer the research question by qualitatively exploring how ICT is currently used to support practices and arrangements in VNFIL processes. More specifically, the authors conducted a qualitative content analysis (Mayring, 2015; Schreier, 2012) of the “European Inventory on Validation”. Published by Cedefop, this comprises a regularly updated overview (most recently in 2018) on current practices to make citizen’s non-formal and informal learning visible. We analysed 43 countries with 1,149 pages in total. We find that the socio-technical perspective well describes the interdependence of social (i.e. VNFIL related) and technical (i.e. ICT related) components. Furthermore, we find that while ICT is frequently used to support VNFIL practices and arrangements, this rings true only for certain parts and supporting processes.

The remainder of this paper is structured as follows. A theoretical background gives an overview of the VNFIL and ICT in Section 2. Subsequently, the method and more specifically, sample size and procedure, are outlined in more detail in Section 3. Section 4 provides the content analysis findings and discusses ICT use in VNFIL practices and arrangements. Section 5 discusses the results in light of the theoretical background, points at further research, provides limitations and concludes.

2. Theoretical background

Historically, the roots of VNFIL can be traced back to “lifelong education” which focussed on personal development (Faure, 1972) and “lifelong learning” that focussed on economic development and growth in the knowledge economy (OECD, 1996). Recent studies explore VNFIL and related approaches in educational programmes (Brown, 2001, 2002), its connection to learning (Breier, 2005), different forms of VNFIL for accreditation purposes (Heath, 2001; Houston et al., 1997; Howard, 1993; Scott, 2007), higher education (Castle and Attwood, 2001; Erhardt and Schmitt, 2018; Pilkinton-Pihko et al., 2019; Sandberg and Andersson, 2011), the health-care sector (Fearfull, 1997; Fejes, 2008; Heath, 2001; Houston et al., 1997; Pryor, 2012; Sandberg and Andersson, 2011; van Kleef and Werquin, 2013) and explores its technical context (Fahrenbach et al., 2019; Kitto et al., 2020; Volungevičiene˙ et al., 2020). However, none of these approaches is targeted towards exploring how ICT is currently used to support VNFIL practices and arrangements. For the purpose of this paper, we subsequently provide background on VNFIL, ICT and how ICT supports VNFIL practices and arrangements.

2.1 Validation of non-formal and informal learning

As stated above, the VNFIL is a process that defines how a competent authority should confirm that a person has acquired learning outcomes measured against a relevant standard. Usually, the VNFIL distinguishes four phases: identification, documentation, assessment and certification (Bjørnåvold, 2000a, 2000b; Council of the European Union, 2012). Firstly, learning outcomes are identified through a “dialogue of particular experiences of an individual” (Council of the European Union, 2012, p. 5), i.e. tacit experiences are made explicit (Nonaka, 1994). Secondly, learning outcomes are documented by making these individual experiences visible (Council of the European Union, 2012), often by putting together a portfolio (Baeten et al., 2008; Brown, 2011; McMullan et al., 2003). Thirdly, assessment entails either comparing documented learning outcomes or a performance assessment against a relevant standard of reference or both (e.g. a qualification standard referenced to the EQF via an NQF and stored in a qualifications register). In this context, assessment can have a formative and summative function (Colardyn and Bjørnåvold, 2004). While the formative assessment is “input-driven, centred on the education and training procedure and linked to educational standards” (Colardyn and Bjørnåvold, 2004, p. 79), summative assessment is “outcome-driven, centred on results achieved and linked to occupational standards (non-formal and informal learning) or to educational standards (formal learning)” (Colardyn and Bjørnåvold, 2004, pp. 79–80). As the assessment of learning outcomes can be input- or outcome-driven, the gold standard is “based on the triangulation of results from different assessment methods” (Cedefop, European Commission and ICF, 2017, p. 74). Fourthly, a competent body certifies the assessment results which may lead to a partial or full qualification or results in credits leading to a qualification (Council of the European Union, 2012, p. 5). While it is recommended that applicants be guided by a responsible person throughout the VNFIL process, in some cases learning outcomes have to be identified and documented independently by the applicant.

Learning outcomes are acquired in different learning settings (formal, non-formal and informal). Firstly, formal learning “occurs within an organised and structured context (formal education, in-company training, etc). and that is designated as learning” (Bjørnåvold, 2000b, p. 204). Secondly, non-formal learning is “embedded in planned activities that are not explicitly designated as learning, but which contain an important learning element” (Bjørnåvold, 2000b, p. 204). Thirdly, informal learning is a part of non-formal learning. It is “often referred to as experiential learning and can to a certain degree be understood as accidental learning” (Bjørnåvold, 2000b, p. 205). For the purpose of this paper, we focus on the latter two (Council of the European Union, 2012).

The recommendation on VNFIL (Council of the European Union, 2012) provides only a broad processual template to structure validation practices and arrangements. What practices and arrangements have in common is that they target at least one of the four phases outlined above. They target learning outcomes (i.e. knowledge skills and/or competences) and usually involve at least two stakeholder groups (applicants, validation providers and national authorities). Aside from these commonalities, they differ in certain respects, for example, where the country has an overarching legal frame or only sectoral arrangements supporting VNFIL.

2.2 Information and communication technology and its support of validation of non-formal and informal learning processes

ICT allows the “electronic input, storage, retrieval, processing, transmission and dissemination of information” (Cedefop, 2014, p. 113) and is researched within information systems. From a socio-technical perspective on information systems, social and technical components (i.e. humans and ICT systems) depend on each other (Bostrom et al., 2009). We can think of this relationship as a social network (Lamb and Kling, 2003). As depicted in Figure 1, social and technical components influence each other reciprocally with the goal of reaching a state of harmony or fit between them through a process of joint optimisation (Sarker et al., 2019). The joint optimisation of both components helps to achieve instrumental objectives (e.g. cost efficiency through scaling effects, time effectiveness through (partially) automated testing and the validity of instruments) and humanistic objectives (e.g. valuing non-formal and informal learning or self-efficacy of applicants). Optimising the technical component, a subject of information technology, is beyond the scope of this paper. Exploring the subjective human experience of information systems use is based on psychological theories such as the theory of planned behaviour (Ajzen, 1991; Ajzen and Fishbein, 1975) or the theory of reasoned action (Madden et al., 1992). How people use ICT (and whether its use is perceived as easy and provides perceived utility) is, amongst several other factors, extensively researched through the technology acceptance model and its further developments (Davis, 1989; Davis et al., 1989; Legris et al., 2003; Venkatesh and Bala, 2008; Venkatesh and Davis, 2000; Venkatesh et al., 2003; Venkatesh et al., 2012). Understanding the relation between technical and social components is crucial to design ICT systems that support the VNFIL. By design, we mean in this context, “changing existing situations into preferred ones” (Simon, 1996, p. 130). When we consider ICT-supported practices and arrangements in VFNIL processes, we want to highlight the mutual constitution of both the technological and social components without necessarily assuming that technology might “replace” the human effort in VNFIL processes. Rather, we see them as co-constituting each other.

ICT can be used to support each VNFIL phase individually. Firstly, ICT is used to support the identification phase. As the identification of learning outcomes is facilitated by a “Socratic” dialogue and between the applicant and a guiding person, ICT is mainly used as a means of communication to facilitate dialogue between them. Secondly, ICT is used to support the documentation phase. For example, portfolio templates such as the Europass [1] or the Profilpass [2] are available in an electronic format and support the documentation of learning outcomes. Thirdly, ICT can be used to support the assessment phase (Luomi-Messerer, 2019b), in both formative and summative assessment. ICT can support summative assessments, for example, through standardised psychological tests or knowledge tests. ICT can also support formative assessments, for example, through submission management systems (e.g. ScholarOne for submitting academic manuscripts) or learning management systems (e.g. “Moodle” for educational purposes), which are both instances of content management systems. Fourthly, ICT can be used to support the certification phase, for example, in preparing credentials and transferring them to applicants. Besides supporting each phase of VNFIL, ICT is also used to raise awareness and provide information on existing VNFIL opportunities. Firstly, ICT is used by the validation providers for awareness building. Awareness about opportunities to participate in VNFIL is often not given in society. ICT is used to create awareness in this respect, for example, through websites. Secondly, ICT is also used by national authorities to publish qualification registers and qualification standards online – which is the basis for validation providers (if the validation procedure aims at obtaining a qualification).

3. Research method

This study aims to answer the research question through qualitative content analysis (Krippendorff, 2009; Mayring, 2015; Schreier, 2012). It is defined as “a research technique for making replicable and valid inferences from texts (or other meaningful matter) to the contexts of their use” (Krippendorff, 2009, p. 18) or as a “method for systematically describing the meaning of qualitative material. It is done by classifying the material as instances of the categories of a coding frame” (Schreier, 2012, p. 1). It classifies qualitative data according to specific criteria and enables structured description and assignment of the material (Mayring, 2015, p. 24). A qualitative content analysis systematically follows a predefined sequence of steps and provides validity by iteratively adapting the coding frame to the material and research question. It departs from defining a research question, selecting material, developing and testing a coding frame, adapting the frame and subsequent analysis and interpretation of results. Coding and subsequent analysis of the materials were conducted with NVIVO.

3.1 Sample characteristics

The VNFIL is a field that is highly relevant in policymaking and for practitioners. Peer-reviewed research articles targeting the VNFIL offer valuable contributions to its distinct theoretical and epistemological grounding. However, they seldom describe practices and arrangements in such detail that this body of literature would promise to answer the research question sufficiently and satisfactorily. However, there are several valuable sources from international organisations and policy advisors that capture the state of the art of these practices and arrangements and their use of ICT. Cedefop, an advisory organisation of the European Union for VET, publishes a biannual report that synthesises the current state of the art in terms of validation arrangements. For this research endeavour, a European synthesis report (Cedefop, European Commission and ICF, 2019) and a thematic report on digital forms of assessment (Luomi-Messerer, 2019b) served as a departure point. Subsequently, 36 country specific reports (Akkök, 2019; Balica, 2019; Ball, 2019; Beleckiene, 2019; de Rick, 2019; Duda, 2019; Dumechin, 2019; Duvekot, 2019a; Dzhengozova, 2019; Gatt, 2019; Guimarães, 2019; Husted, 2019; Ieleja, 2019; Johnson, 2019; Karttunen, 2019; Kristensen, 2019; Lárusdóttir, 2019; Luomi-Messerer, 2019a; Manoudi, 2019a, 2019b; Mathou, 2019a; Murphy, 2019; Pavkov, 2019a, 2019b; Perulli, 2019; Popovic, 2019; Puukka, 2019; Richardson, 2019; Salini et al., 2019; Scott, 2019a, 2019b; Stalker, 2019; Tót, 2019; Ure, 2019; Vale, 2019; Vantuch, 2019) on which the synthesis report is based and seven additional reports (Danuza, 2019; Dimovska, 2019; Duvekot, 2019b; Endrodi, 2019; Mathou, 2019b; Spasovski, 2019) were analysed to gain a broader perspective and a deeper understanding of the issue at hand. While Cedefop publishes these synthesis reports to aggregate the state of the art for all country reports, these aggregations are of little use to describe specific approaches in sufficient detail and capture their qualitative and context-dependent differences to answer the research question. In other words, the synthesis reports are merely a quantitative summary. However, it is relatively easy to compare different country reports as they all follow the same structure (i.e. their table of content reveals the same structure for each report). It must be mentioned here that the aspect of “ICT-based assessment methods used for validation” was included as a specific point in the chapters on “validation methods” in the country reports for the 2018 update of the inventory, and a thematic report on this topic was also prepared (Luomi-Messerer, 2019b). Analysing these reports and the mentioned practices and arrangements in depth allowed for capturing and comparing their state of the art and how these are supported through ICT. As far as the authors are aware, this information could not have otherwise been obtained in such a systematic way.

3.2 Procedure

To capture the state of the art of practices and arrangements, all country reports published for the 2018 update of the Inventory were downloaded and saved on a hard disc [3]. This amounted to 39 countries from Europe and three non-European international case studies (Canada, Chile and Hong Kong). In total, the country reports amounted to 1,149 pages, which constituted the primary sample. In the first iteration of open coding, all 42 country reports were read by the author and openly coded without a specific focus. Overall, in this iteration, 459 references were created that generally described the validation situation in the respective country. This could be, for example, specific practices or tools but could also be laws and legal arrangements. These 459 references included 106 specific practices and arrangements (i.e. codes with distinct names). The first iteration of coding resulted in the creation of an initial coding frame which consisted of 12 dimensions:

Areas of practice or arrangement, content, cost, duration, form, its connection to learning outcomes, the name, outcomes for the participant, phases of the VNFIL covered, the purpose of the initiative, stakeholders involved and the reported use of ICT.

This initial coding frame was tested within the country reports of Austria, Germany and France. Based on the experiences from open coding and testing the coding frame, the coding frame was revised to ensure the following: a high consistency (i.e. reliability); validity (i.e. ensuring that the coding frame captures the material well); unidimensionality (i.e. one aspect of the material should be covered per the main category); mutual exclusiveness (i.e. subcategories exclude each other but a text segment can be assigned to different main categories); exhaustiveness (i.e. each relevant text segment can be assigned to one subcategory and the coding frame covers all relevant aspects of the meaning); and saturation (i.e. every category is used at least once). The revised coding frame was the departure point for the second iteration of coding. In this second iteration of axial coding, all 459 references including the 106 identified practices and arrangements were reread and if deemed helpful to answer the research question (i.e. reported the use of ICT), included in the second iteration of axial coding (which amounted to a total number of 68 practices and arrangements). The name for each practice or arrangement was identified and placed in the coding frame. The focus in the data analysis was on specific practices and arrangements currently used in different countries, different areas and different stakeholders. Rereading all of the codes from the first iteration was crucial as each country has different legal arrangements, and thus, each initiative works in its own context. This step led to 2,064 codes that place the 68 practices and arrangements into an axial coding frame. In the third iteration of coding (i.e. selective coding), we revised the coding frame and aimed to elicit the “underlying patterns” by restructuring the 68 practices and arrangements to the 15 dimensions that scaffold the use of ICT to support VNFIL practices and arrangements.

4. Findings

Firstly, we draw on the socio-technical perspective to report our findings and secondly draw two propositions concerning the primary use of ICT to support VNFIL practices and arrangements.

4.1 A socio-technical perspective on information and communication technology-supported validation of non-formal and informal learning practices and arrangements

As outlined in the theoretical background, we can see the ICT support of VNFIL practices and arrangements from a socio-technical perspective (social components, technical components, humanistic objectives and technical objectives), which provides a scaffold to present the results of the qualitative content analysis (Table 1, left column). The adapted coding frame consists of 15 dimensions (Table 1, middle column) based on this structure. The right column of Table 1 presents the definition of each dimension, as well as the codes and their frequency of occurrence in the data set (for each dimension). We decided against structuring practices and arrangements according to their country of occurrence and comparing practices and arrangements directly, as the context in which each practice and arrangement is used is highly diverse and does not allow us to compare these meaningfully. Cedefop frequently conducts a comparison of the regulations, frameworks and structures of VNFIL and how VNFIL is used in different countries in the “European Inventory on validation”.

We find that six dimensions can describe the social component: namely, the area of VNFIL (e.g. VET and HE); the target groups and stakeholders involved (e.g. elderly, youth and educational authorities); phases of VNFIL targeted (e.g. identification of learning outcomes and documentation of learning outcomes); learning outcomes targeted (e.g. occupational or transversal knowledge, skills and competences); the guidance or counselling offered, if at all (e.g. continuous and online); and the individualisation of a practice or arrangement (e.g. personal competence development plan and individual learning paths). We find that five dimensions can describe the technical component: namely, the outcome of practice or arrangement (e.g. competence portfolios and certificates); the form (e.g. described as website, web, e-portfolio, e-tool and online-tool); methods (e.g. interviews and debates); the access (e.g. login); the channel of communication (e.g. face-to-face, computer and telephone); and standards (e.g. qualification standards and occupational standards). We find two dimensions describe humanistic objectives: namely, the purpose of practice or arrangement (e.g. validation of occupational competences for the labour market and access to education and training courses) and outcomes for the participants (e.g. visibility of non-formal and informal learning and increase of motivation). Two dimensions describe technical objectives: namely, duration of practice or arrangement (e.g. hours, weeks and months) and cost of practice or arrangement for the individual (e.g. free of charge). Other technical objectives (e.g. effectiveness, cost efficiency and validity reliability) are not reported in the data set.

Based on these findings, we establish two propositions regarding the use of ICT to support VNFIL practices and arrangements and provide numerous examples.

4.2 The primary use of information and communication technology to support the validation of non-formal and informal learning practices and arrangements

The primary use of ICT to support the VNFIL practices and arrangements is to provide information on validation opportunities, store qualification registers online, identify and document learning outcomes.

In general, the use of ICT to support the VNFIL practices and arrangements remains limited and is acknowledged in several country reports, e.g. Austria (Luomi-Messerer, 2019a, p. 44), Croatia (Pavkov, 2019a, p. 27), Czech (Stalker, 2019, p. 17), Luxemburg (Dumechin, 2019, p. 22), Malta (Gatt, 2019, p. 29), Montenegro (Dimovska, 2019, p. 18), North-Macedonia (Spasovski, 2019, p. 19), Romania (Balica, 2019, p. 13), Slovakia (Vantuch, 2019, p. 20) and Switzerland (Salini et al., 2019, p. 2). If ICT is used, it is mainly reported for legally not-binding practices and arrangements. More details are outlined below.

Firstly, the use of ICT is often reported for information purposes and aims to increase awareness of validation or existing validation practices and arrangements. These are mostly websites by organisations that inform about existing validation opportunities, such as “valideringsinfo” (Kristensen, 2019, p. 9), “Valida.Suisse” (Salini et al., 2019, p. 20) or “Vox” (Ure, 2019, p. 6). Secondly, the use of ICT is frequently reported to publish qualification registers (i.e. static databases to store qualification standards), the “National Register of Qualifications” in Czechia (Stalker, 2019, p. 1), the “National Reference Point for Occupational Qualifications” in Slovenia (Pavkov, 2019b, p. 8) or the “National Vocational Qualifications Directory” in France (Mathou, 2019a, p. 18), for example. Thirdly, ICT use is reported for skills audits that are not legally binding. For example, public employment services (PES), ministries of education or social partner organisations offer skills audits to integrate people into the labour market and increase the awareness of one’s knowledge, skills and competences, such as “My Career” (de Rick, 2019, p. 5), “Mincompetencemappe” (Husted, 2019, p. 10), “Writeon” (Murphy, 2019, p. 15) or “Acredita” (Vale, 2019, p. 33). Fourthly, ICT is used to create and store portfolios online (called e-portfolios). These are mainly used to give individuals the opportunity to store identified learning outcomes online and if possible, connect them with existing taxonomies of knowledge, skills and competences or qualification standards. These practices and arrangements support the phase of documentation as seen in Estonia (Johnson, 2019, p. 11), Greece (Manoudi, 2019b, p. 16), Ireland (Murphy, 2019, p. 15), Luxemburg (Dumechin, 2019, p. 10), the Netherlands (Duvekot, 2019b, p. 2) and Slovenia (Pavkov, 2019b, p. 26). These practices and arrangements are also frequently used in the third sector, such as initiatives for volunteers or youth workers to collect evidence of experience which can be shown on the labour market or added to applications.

4.3 Limited use of information and communication technology to support the assessment phase

In sum, the use of ICT to support VNFIL practices and arrangements is mainly reported for the purposes and in the sectors described above. However, there are a few exceptions that also deal with assessing learning outcomes (Luomi-Messerer, 2019b, pp. 10–13). Thus, these practices and arrangements are more dynamic and process-oriented and may be used to award a full qualification.

Firstly, “MySkills” (Ball, 2019, p. 10) from Germany is a practice that supports ICT-based assessment and is based on established qualification standards; however, the outcome is not legally binding. “MySkills” is a “large-scale ICT-based assessment project in order to develop and implement digital tests for the assessment of competences gained at work” (Ball, 2019, p. 3) for people that cannot substantiate a formal qualification. Even though it is connected to VET professions, it is conducted by the German PES and targets placement in the labour market. Secondly, “Osaan” (Karttunen, 2019, p. 24) is a practice from Finland that supports an ICT-based assessment and is also based on established qualification standards; however, its results are legally binding. “Osaan” is an e-Tool which “comprises all VET qualifications and the assessment criteria” (Karttunen, 2019, p. 24). Candidates provide evidence on competences in relation to the learning outcomes of the qualification in question. This practice aims at including all “competence-based qualifications” and the results in the award of qualification if assessors deem the candidate as successful (Luomi-Messerer, 2019b, p. 11). Thirdly, an ICT-based assessment currently being developed in France is based on established qualification standards and the results of which are legally binding. It should support the French “validation des acquis de l’expérience (VAE)” (Mathou, 2019a, p. 8) and is described as a “tool to automate the orientation towards the provision of qualifications accessible through VAE most adapted to the professional skills of the candidate.” This practice is supported through the uniform and accepted legal arrangement on which it is based.

ICT supports the expertise of the assessor in various steps of the assessment but may not replace human reasoning in the near future. We observe so little ICT-based assessment because of the partly tacit nature of learning outcomes (Bjørnåvold, 2000a; Polanyi, 1966). In this regard, (implicit) vocational knowledge, skills and competences are mainly manifested in the performance of a specific activity (Engeström et al., 1999; Engeström and Sannino, 2012) or a specific task and are, thus, more challenging to make explicit, to document and assess through the use of ICT alone. Especially within the VET, we have to assume that there is a considerable amount of learning outcomes that cannot be made explicit at all. Consequently, these learning outcomes can only, if at all, be documented and assessed within analogue and face-to-face assessment situations and within work simulations. Another reason why we observe the limited use of ICT in the assessment phase is because some target groups cannot be properly reached through ICT. Additionally, the individualisation of practices and arrangements is hardly possible when standardised ICT tools are used. Thus, especially in VET professions, ICT plays a limited role but may be used to support the expertise of the assessor, for example, by structuring portfolios, providing videos on work samples and so on and so forth.

5. Discussion and conclusion

This paper set out to answer the research question, “How do ICT support practices and arrangements in VFNIL processes?” Based on a qualitative content analysis of 43 country reports describing VNFIL practices and arrangements, we aimed to explore how they are supported through ICT.

5.1 Theoretical contribution

We explore the support of VNFIL through ICT from a socio-technical perspective (Sarker et al., 2019). While most studies on VNFIL deal with its social aspects, we aim to contribute to the body of literature dealing with its technical context (Fahrenbach et al., 2019; Kitto et al., 2020; Volungevičiene˙ et al., 2020). The socio-technical perspective provides a framework that illuminates the interdependent relationship and the mutual co-constitution of the technical (i.e. ICT) and social components (i.e. VNFIL), which are usually not considered in combination. In this regard, the socio-technical perspective (ICT and VNFIL) invites us to rethink our understanding of how we use ICT to support VNFIL practices and arrangements.

Based on our qualitative content analysis of 43 country reports, we observe that current perspectives on ICT-supported VNFIL practices and arrangements focus on the social component and see the technical component (mainly) as a mere means of support. Specifically, we find that the primary use of ICT to support VNFIL practices and arrangements is to provide information and raise awareness on validation opportunities. Furthermore, ICT is used to store qualification registers online. Especially the identification of learning outcomes and the documentation of learning outcomes are supported through ICT-based practices and arrangements. However, there are only a few ICT-supported practices and arrangements that target the assessment phase of the VNFIL and lead to the award of a qualification (see also Luomi-Messerer, 2019b). If ICT is used in this regard, it mainly supports the expertise of the assessor.

The socio-technical perspective allows for a more nuanced understanding. From this perspective, ICT is not only a supporting mechanism in VNFIL practices and arrangements. Rather, the technological and social components are mutually dependent on each other and influence each other in reciprocal interactions (Sarker et al., 2019). This means the availability of new or improved technologies (e.g. blockchain technology and virtual reality devices) may alter our understanding of VNFIL processes. However, it also implies that a better understanding of what VNFIL means allows us to rethink our understanding of technology (i.e. ICT support) in this context. In this regard, we gain a more nuanced understanding of how ICT and VNFIL practices and arrangements are interdependent.

Furthermore, the socio-technical perspective proposes two objectives (i.e. instrumental and humanistic). We acknowledge that compromises may be necessary if these perspectives cannot be brought into line. In other words, sometimes we cannot have both: For example, an ICT system that is cost-efficient and standardised, i.e. one that meets instrumental objectives, may not be easy to use for certain target groups, thus counteracting humanistic objectives to a certain extent. However, the socio-technical perspective provides a framework allowing conflicting objectives to be reflected and possibly balanced or resolved.

5.2 Practical implications

The findings of our research endeavour may inform four different stakeholder groups (Achterkamp and Vos, 2007), namely, practitioners, designers, researchers and policy-makers. Firstly, practitioners who use ICT in VNFIL practices and arrangements in their daily work may use the presented framework to reflect upon their own approach (perceptions, attitudes) towards ICT-supported VNFIL practices and arrangements. This might help to find blind spots or offer development opportunities on both a social and technical dimension. Secondly, designers who develop ICT-supported VNFIL practices and arrangements may use the framework as a picture that captures the “state-of-the-art”. This offers a “design space” in which designers see how different dimensions are described in the literature (e.g. practices and arrangements are tailored towards very diverse target groups). Thirdly, researchers who conduct scientific research on ICT-supported VNFIL practices and arrangements could use the framework as a tool for analysis. The framework introduced above could help conduct a more systematic and theory-based analysis of ICT-supported VNFIL practices and arrangements, which could be categorised according to the framework. Fourthly, policy-makers who develop policy initiatives on VNFIL may find value in our framework. This is stressed in current policy documents that find: “any new policy initiative on validation should have stronger links to flexible learning pathways and should promote the use of innovating solutions (e.g. ICT) for flexible learning” (European Commission, 2020a, p. 138). Furthermore, the European Commission (2020b, p. 5) stresses the importance of “flexible and modular learning opportunities”. Our framework could help policy-makers to meet the European Commission’s needs and requirements in bringing flexible pathways, modular learning opportunities and innovative (ICT-based) solutions into practice.

5.3 Limitations and further research

One limitation of the current study concerns the sample of country reports, which is mainly limited to the European Union. Countries such as South Africa, with a vivid history in VNFIL, were not covered in the sample. Another limitation is that several country reports do not report on practices or arrangements directly, but refer to the earlier country report in which the practices and arrangements were described. These practices and arrangements are missing in the current sample. A further limitation is that the analysis only refers to the aspects described in the country reports. The country reports analysed follow a template, which brings certain aspects to the foreground and leaves others aside.

From a socio-technical perspective, further research should deal with the interdependence of the technical and social components. Specifically, further research could explore how we can alter our understanding of the VNFIL through the use of ICT and how VNFIL`s social component can be better understood through reflecting on its ICT support. On the other side, we see merit in exploring how our understanding of the VNFIL allows us to design better ICT-supported practices and arrangements. As described above, further research on VNFIL practices and arrangements is necessary to evaluate inasmuch as the VNFIL fulfils instrumental objectives and lives up to the promises of humanistic objectives. From a designer’s perspective, further research should aim to develop and evaluate standardised and, in particular, ICT-based practices and arrangements that support all phases of the VNFIL to ensure the quality, legitimacy and acceptance of validation processes.

Figures

Figure 1.

A socio-technical perspective on ICT-support of VNFIL practices and arrangements

Table 1.

Dimensions and frequency of occurrences of codes in the data set

Component	Dimensions	Codes (frequency of occurrences in the data set)
Social component	Area of VNFIL in practice or arrangement	Def: part of the educational system or labour market in which the practice or arrangement is applied− Vocational education and training (40), labour market (25), third sector (20), general education (9), volunteering (8), higher education (7), youth exchanges (1) and non-formal adult education (1)− Skills audits (31)
	Target groups and stakeholders involved in practice or arrangement	Def: any individual or organisational entity that is affected by or is able to affect VNFIL practices and arrangements (Achterkamp and Vos, 2007)− Individuals (40), adults (over 18) (29), job seekers (17), youth (16), individuals with no qualifications (13), natives (13), students (11), refugees (8), immigrants (7), career counsellors (6), youth workers (5), unemployed (3), teachers of functional skills (2), parents (1), teachers (1) and tour guides (1)− Educational authorities (34), professional employment service (23), ministry of labour (11), youth organisations (10), chamber of commerce (8), social partners (6), United Nations Development Programme (2), World Bank (2), academic department (1), army (1), Red Cross (1) and trade unions (1)
	Phases of VNFIL targeted in practice or arrangement	Def: eligibility check, information, identification, documentation, assessment and certification− VNFIL eligibility check (2) and check VNFIL eligibility of candidate (1)− Provision of information and guidance on VNFIL online (6), of information (advice and guidance) (5), of information on qualifications (4), of information on VNFIL (3), of information on validation opportunities (3) of information on VNFIL arrangements and procedures (1)− Identification of learning outcomes (38) and dentification of skills and competences acquired during a stay abroad (3)−Documentation of learning outcomes (108), VNFIL based on documentation presented (22), documentation of formal certificates (diplomas and qualifications) (3), mapping of formal education and training (work and health) (1)− Assessment of learning outcomes (103), competence-based assessment (36), VNFIL based on competence demonstration (17), self-assessment (16), skills assessment (13), assessment and skills profiling (2), testing (summative function) (2) and advisory (formative function) (1)− Certification of learning outcomes (23)
	Learning outcomes targeted in practice or arrangement	Def: knowledge, skills and competences and other characteristics− Targets competences (90), vocational competences (43) and key competences (30)− Targets skills (79), ICT skills (12), soft skills (8) and functional skills (4)− Targets knowledge (52)− Targets work experience (23) and professional and personal experience (21)− Targets working values (1), interests (4), numeracy (1), reading (1) and writing (1)
	Guidance and counselling in practice or arrangement	Def: the responsible person giving advice or information throughout the process−Continuous guidance (11), online guidance (6), e-counselling (3), interactive counselling system (1), employment and skills action plan (1)
	Individualisation of practice or arrangement	Def: individualisation achieved through the presentation of personalised content and by offering the opportunity to account for a person’s individual history− Personal competence development plan (19), individual development plan (8), support use of an individual right scheme through personal training account (4), individual learning path (3), personal training account (3), educational record (1), learning pathway plan (1), individual study plan according to competences and needs (1) and the connection of learning history (retrospective) with learning aspiration (prospective) (1)
Technical component	Outcome of practice or arrangement	Def: tangible outcome after running through a VNFIL practice or arrangement− Competence portfolios (36), certificates of competences (17), formal certificates (diplomas, qualifications, awards) (15), recognised credits (6), badges (3) and record of knowledge in one place (2)
	Form of practice or arrangement	Def: technical or ICT-based method or artefacts to support VNFIL− Artefacts: web (79), website (62), e-tool (20), e-portfolio (15), online tool (15), ICT-tool (13), e-portal (12), ICT-based assessment (9), database (9), digital tool (7), information platform (3), online application form (3), quick digital self-assessment screening tool (3), information portal (2), Moodle (1), open digital badge (1) and web-based learning resource (1)− Structured process (17), service (11), test (10), competence check (6), information material (4), career portfolio development (3), training course (3), assessment toolkit (5), dialogic validation and learning (1), interest test (1), picture questionnaire (1), video evidence (1) and workability assessment (1)− Forms (6), word documents for documentation and application (1) and excel sheets (1)− Methods: create digital portfolios (9) and interviews and debates (6)
	Access of practice or arrangement	Def: a technological implementation used to grant access to practice or arrangement− Login (30), reference code (4), bank identification code (1), certificate card (1), mobile certificate (1) and self-registration system (1)
	Channel of communication used in practice or arrangement	Def: an often technological means of communication used in practice or arrangement− Face-to-face (21), computer (11), telephone (7), attendance at workshops or businesses (1) and peer meetings (intervisions) and supervisions (1)
	Standards	Def: reference standards against which individual learning outcomes are compared− Qualification register (20), occupational standards (5), national qualification requirements (2) and occupational descriptors (1)
Humanistic objectives	Purpose of practice or arrangement	Def: reported reason why the practice or arrangement exists in a context− access to the labour market: validation of occupational competences for the labour market (26), increase employability (4), assessment of foreign qualifications (3), the award of apprenticeship certificate (7), specific assessment in relation to specific qualification standards (2), formal recognition of tour guide certificate (1), improve basic skill level to gain VET qualification (1), gain qualification for the labour market (1) and provide online learning and literacy development (1)− access to education and training courses: admission to a course of study (2), grant access to higher education through VNFIL (2), grant right to the validation of prior learning (1), increase match between possible and required competences (1) and identify all competences mastered by the individual (1)− personal development (i.e. orientation and career guidance): record skills and competences acquired through student work (2) provide career development service (1), provide of non-formal learning opportunities for youth (1) and monitoring changes in soft outcomes (1)
	Outcome of practice or arrangement for participants	Def: positive result of a VNFIL process for the applicant (i.e. added value)− visibility of informal learning (37), supports applications for jobs or education and training opportunities (35), (re)integration into the labour market (2), award of modules or partial qualifications (2), students receive competence points (1) and candidates can obtain a full qualification (1)− increase of motivation (10), taking initiative (4), staying active (2) and staying curious (1)
Technical objectives	Duration of practice or arrangement	Def: the time needed for an individual to run through the practice or arrangement− 6 to 12 months (9), 30 min (4) and 4 h to conclude (3)
	Cost of practice or arrangement	Def: the amount of money an individual has to pay to participate in the practice or arrangement− free of charge (35) and small charge or membership fee (2)

Notes

1. https://europa.eu/europass/de

2. www.profilpass.de/

3. www.cedefop.europa.eu/de/publications-and-resources/country-reports/european-inventory-on-validation