An evaluation of faculty development programme on the design and development of self-learning materials for open distance learning

Abstract

Purpose

This paper seeks to ascertain the effectiveness of a two-week-long faculty development programme (FDP), organized by Indira Gandhi National Open University (IGNOU), for teachers of the open and distance learning (ODL) system, with the aim of upgrading their knowledge and skills for developing print self-learning materials (SLMs). The specific aim of this study is to ascertain whether the objectives of the FDP have been achieved, to examine the effectiveness of the training programme and to suggest measures for improvement in future FDPs regarding the design and development of print SLMs.

Design/methodology/approach

The study was conducted using a descriptive survey research method, through semi-structured questionnaires and adopted purposive sampling. The first survey was conducted immediately after the completion of the programme, in order to collect feedback from the trainees, while the second survey was conducted after six months of the FDP, using the follow-up approach, so that more reliable and authentic results could be obtained.

Findings

The outcomes of the study revealed that the training had been effective in imparting appropriate knowledge and skills to the trainees, with respect to designing print SLMs. However, certain gaps were also identified and have been reported in this paper. Key suggestions have been made to address the shortcomings and improve forthcoming FDPs.

Research limitations/implications

The present research focused on a specific training programme regarding the design and development of SLMs. Therefore, only the teachers and academics who participated in this specialized training activity were considered for the collection of feedback.

Practical implications

The recommendations of this work may be useful for the trainers, FDP coordinators, training institutions and ODL policymakers for planning and designing effective staff development activities for teachers and academics working in ODL institutions. These would be especially useful in the design of FDPs, aimed at orienting teachers who are involved in curriculum planning, design and development of learning resources.

Originality/value

This study is an original research based on the empirical primary data obtained by the researchers. As the largest open university in the world, IGNOU has been playing a key role in staff development for and in ODL in developing countries. This evaluative study of a specialized FDP in the area of SLM design and development is a significant work that may be valuable for planning the staff development strategies and building a training evaluation mechanism.

Citation

Asgar, A. and Satyanarayana, R. (2021), "An evaluation of faculty development programme on the design and development of self-learning materials for open distance learning", Asian Association of Open Universities Journal, Vol. 16 No. 1, pp. 98-115. https://doi.org/10.1108/AAOUJ-11-2020-0094

Introduction

Faculty development and regular upgradation of teaching skills are very crucial aspects of any educational institution, irrespective of its mode of operations, whether it is face-to-face or through distance learning. The quality of teaching–learning processes, learning materials and overall organizational growth depend heavily on the quality of teachers engaged in educational transactions. Similarly, the quality of teachers is linked with continuous professional development (CPD) and different capacity building (CB) measures. However, continuous development of the faculty becomes more important for distance education institutions due to the specific requirements of the open distance learning (ODL) system, which has witnessed a paradigm shift as a result of the change in mediated educational technology and the mode of delivery. Therefore, frequent CB activities are vital not only for the newly recruited teachers but also for existing faculty members to acquaint themselves with rapid structural changes (Asgar and Mythili, 2020) in the ODL system.

Furthermore, a majority of ODL teachers come from conventional educational institutions and have no prior experience of working in a distance learning environment. Therefore, their orientation becomes imperative. According to Mannan (2013), “most of the teachers, tutors and officers recruited at Bangladesh Open University do not have prior experience in ODL. Therefore, it becomes crucial to enhance the capacities of those staff who are placed at various managerial and academic positions for the better practices of ODL.” New entrants are less familiar with the ODL system's open, flexible, innovative and learner-centric approaches, evaluation mechanisms and teaching–learning through SLMs as well as the emerging educational tools. Such a situation demands proper training (Muralimanohar, 1997, as cited in Dimri and Misra, 2008). In addition, there has been a dearth in the supply of trained workforce in ODL institutions. In a recent study, Johry et al. (2019) reported a shortage of skilled workforce and pressed on the need for more skill-based academic programmes and CB initiatives. Highlighting the inadequate supply of professionally trained ODL personnel, Siaciwena (2010) also reported that there is a need for well-qualified and experienced people to plan and manage the implementation of distance education programmes. In view of this scenario and the fact that a systematic training process empowers the faculty to perform defined tasks with expertise and in accordance with the institution's expectations, open universities should utilize their resources to build the capacity of both the academic and non-academic staff through different CB measures. However, training initiatives yield maximum results only if they are planned systematically. Institutions should start with need assessment, followed by proper curriculum planning and implementation. This could then culminate with a good training evaluation plan. Evaluating different CB initiatives such as faculty development programme (FDP), workshop, orientation and refresher programmes is equally important to ascertain the program's effectiveness. It is only through this exercise that the training managers and institutions get to know whether the objectives of the programme have been achieved. Moreover, evaluation of training and educational programmes helps in ensuring quality of CB initiatives and educational programmes.

Statement of the problem

At IGNOU, India, a great deal of time, money, energy and academic input is utilized in conducting CB activities for teaching and non-teaching staff working in the ODL system. A similar situation may also exist in other ODL institutions. Therefore, institutions need to know whether investments in the form of human resources, money and academic inputs are being utilized effectively. As a result of the increasing expenditure in training and development (T&D), a number of organizations are concerned about the returns on these investments. They are reluctant to spend money and seek justification for T&D costs (Topno, 2012). Furthermore, Iyer et al. (2009) opined that although evaluation is a grey area, every organization has had to move towards evaluating its training programme to ascertain return on investment (ROI) and change in staff behaviour in order to justify the investments made in training as well as to improve the quality of the training process.

Background

The Staff Training and Research Institute of Distance Education (STRIDE) at Indira Gandhi National Open University (IGNOU) was upgraded into a training and research institute based on the proposal of the Commonwealth of Learning, the Asian Development Bank and the Ministry of Human Resources Development (MHRD), the Government of India in 1993 (IGNOU, 2021). Since then, this premier institute has been planning and implementing need-based training activities for academics associated with various private/state/central single- (open) and dual-mode institutions. Faculty development in the area of design and development of SLMs has been one of the specialized and important training themes because of the significant role of self-instructional resources in teaching and learning at a distance. SLMs are the heart of the ODL system, but the expectations of pedagogic aspects and instructional delivery need to be addressed further (Gbenoba and Dahunsi, 2014). The regulatory body, the University Grants Commission (UGC, 2020), stressed on greater engagement of learners and the use of technologies in traditional SLMs. Therefore, training programmes/workshops/FDPs on this particular theme are organized frequently by STRIDE.

In this context, the ten-day training programme being evaluated here, entitled Faculty Development Programme on the Design and Development of Self-learning Materials: In Spirit of UGC (ODL) Regulations, 2017, was organized from the 13^th to the 23^rd of November, 2019. The programme was not only aimed at enhancing the capacity of participants in the area of design and development of print learning materials but also intended to help them in their professional growth, career advancement and promotion. According to the Government of India (2018), one-week (5 days) and two-week-long (10 days) FDPs are required for promotion from Level 10 to 11, Level 11 to 12 and Level 12 to 13 under the Career Advancement Scheme (CAS) set by the UGC.

FDP objectives, methods and materials

The programme had specific objectives (Table 4), and strategies were adopted accordingly to fulfil those objectives at the optimum level. Lectures by experts and resource persons, PPT presentations, panel discussions, interactions in workshop mode, sharing of experiences and individual/group-based practical activities were the methods for imparting training. At the end of the FDP, each participant was expected to develop a new module/ unit of print SLMs or transform a draft unit/module into the SLM format in the respective area of their expertise. STRIDE training handbooks on different open distance education (ODE) themes; a copy of the UGC (ODL) Regulations, 2017; the UGC (online courses or programme) Regulation, 2018; various relevant print/e-resources and PPTs by resource persons were used and distributed among the faculty as reference materials. Faculty members of STRIDE and other schools/divisions of IGNOU, subject matter experts from Jamia Millia Islamia, New Delhi, the National Institute of Educational Planning and Administration (NIEPA), New Delhi and officials from the MHRD contributed to the FDP as resource persons.

Significance of the study

The FDP was designed and implemented with well-defined objectives. Its evaluation was done at the end of the programme for improving the quality of future training programmes. The methodology followed by STRIDE to evaluate programmes is based on feedback from participants and their reflections on various aspects of the CB activities. A systematic evaluation of this specialized FDP will empower the STRIDE faculty with better planning skills in accordance with the expectations of regulatory bodies and other stakeholders. This work will also guide other ODL practitioners and institutions in designing, implementing and evaluating CB programmes in a more systematic and professional manner.

Objectives

Specific objectives of the study were to

1. List the characteristics of trainees from different ODL and dual-mode institutions;

2. Elicit trainees' opinions about the FDP vis-à-vis its curriculum, activities and arrangements;

3. Ascertain whether the objectives of the programme were achieved;

4. Assess the effectiveness of the CB programme;

5. Examine participants' post-training aptitude in certain areas related to SLM development and

6. Explore gaps, if any, and propose recommendations for improvement of future FDPs.

Review of literature

Training is a systematic process that starts with training aeeds assessment (TNA), aimed at a particular clientele and followed by the design, development and implementation of training activities. Once the programme is completed, its evaluation helps in assessing its effectiveness, impact and usefulness. The Kirkpatrick ADDIE model of instructional design focuses on evaluation as the key culminating stage, which comes after the phases of analysis, design, development and implementation, but its importance remains intact. Evaluation helps in finding gaps, if any, and guides trainers towards addressing identified shortcomings and improving the training event. It plays a crucial role in ensuring quality in future CB activities. Therefore, Rajeev et al. (2009) laid emphasis on the inclusion of evaluation mechanism at the time of conceptualizing an FDP. According to him, an inbuilt monitoring and evaluation system makes a programme more useful. Prioritizing the assessment aspects, Gade (2020) also advocated the timely evaluation of FDPs to ascertain their impact and effectiveness, so that appropriate steps are taken for making them more relevant and useful. However, training institutions often ignore this vital aspect and do not give as much importance to evaluation as they do to the planning and implementation of staff development programmes. Nevertheless, numerous models and theories are available for conducting evaluation studies. A few popular models are briefly discussed in this paper to present a perspective on the evaluation strategies.

Kirkpatrick model (1959)

This is a very popular model, proposed by Donald Kirkpatrick, which focuses on measuring four kinds of outcomes or levels that are expected from an effective training programme. These four levels are reaction, learning, behaviour and results. Reaction is how well the trainees liked the training program; learning deals with whether the trainees acquired any knowledge, attitude or skills; behaviour evaluates the extent to which the trainees' job behaviour changed as a result of the training; while results try to determine the extent to which the outcomes (i.e. effects on the business or institution) have been impacted by the training programme. This model is the most preferred evaluation framework as it helps in understanding the training evaluation in a very systematic way (Shelton and Alliger, 1993).

D. Phillip's model (1996)

Phillip proposed five levels of evaluation by adding an additional level to Kirkpatrick's four-level evaluation model. His five-level evaluation model incorporates determining the worth of training in monetary terms, i.e. the ROI.

CIPP model (1960)

Daniel L. Stufflebeam developed this model, which refers to the four phases of evaluation: context, input, process and product. This model covers the formative and summative stages of evaluation. It systematically guides the evaluator by posing relevant questions and conducting assessment at the beginning of any educational or training programme during the programme and after its completion.

CIRO model (1970)

The Warr, Bird and Rackson model evaluates four aspects of training – context, input, reaction and outcomes. According to Tennant et al. (2002), this model presses on both pre- and post-measurement of training outcomes. Context deals with TNA and formulation of the programme objectives. Input deals with the design and execution of the programme. At the stage of reaction, effort is made to seek the opinion of trainees about the quality of the training, while the outcome assesses the achievements drawn from a particular training programme.

According to Boulmetis and Dutwin (2014) and Schalock (2001), evaluation should be intended to determine whether and to what degree the objectives or goals of the programme were achieved. If the training programmes successfully achieved certain objectives, one could say that these were effective. Several evaluation studies have been conducted using the models and definitions discussed above. Omar et al. (2009) evaluated a training programme for Iranian health managers using the Kirkpatrick model and developed assessment statements on a five-point Likert scale. Respondents agreed most strongly with the statements that the course gave them a chance to meet colleagues from other parts of Iran and that it made them realize the importance of CPD. The majority of respondents reported that the course engaged credible teachers and was interesting and relevant but had an overemphasis on theory than practical training. Evaluating the effectiveness of a training course at Islamshahr University, Iran, Farjad (2012) reported that the training course was sub-standard in terms of overall effectiveness and needed to be improved in training design, management, financial support and a strong institutional mechanism to motivate employees towards their professional development.

Dimri and Misra (2006) assessed the training programmes for teachers and academics of IGNOU and reported that such programmes were effective as participants derived the expected benefits. However, they recommended that these should be need-based and more frequent. They also suggested organizing longer duration training programmes for the academics. A study conducted by Gowthaman and Awadhiya (2017) in IGNOU indicated that knowledge and skills gained during the training were useful for the non-teaching staff. However, problems related to the lack of infrastructure, self-motivation and time were major barriers in the professional development of non-teaching employees.

Evaluating the effectiveness of a basic teacher training workshop conducted in Nepal at Kirkpatrick level 1 (evaluation of reaction), Piryani et al. (2018) found that the “self-reported perceived confidence level of the medical teachers significantly increased after the workshop.” While evaluating a language teacher training programme in Turkey, Uzun (2016) proposed that training events must meet the societal and individual needs and have a balance of content and practical knowledge. In another study, Rashid (2006) examined the CB programme on the development of SLMs at Bangladesh Open University, and the results revealed that training was useful in providing the required knowledge and technique for creating self-learning modules, radio/TV scripts, audio and video tapes, e-learning resources, teleconferencing, etc. In a study on human resource development at Hanoi Open University, Nguyen (2018) found that the university lacked staff training policies and faced a shortage of workforce and technological infrastructure for online training.

Methodology

The present work was conceptualized and executed in the light of definitions proposed by Boulmetis and Dutwin (2014) as well as Schalock (2001), adopting the evaluation framework of Kirkpatrick levels 1 and 2 (reaction and learning). The review of different models and strategies adopted in different studies also helped researchers in planning evaluation strategies. This exercise was essentially a post-training activity, divided into two stages of evaluation. The first stage of evaluation was executed immediately after the completion of the programme, while the second stage of evaluation was carried out through a follow-up. The follow-up evaluation makes staff development activities more worthwhile for everyone (Pulist, 2017).

The descriptive survey research methodology was adopted to collect data from participants. Qualitative and quantitative data were collected through a semi-structured questionnaire. A mixed-method approach was applied in the present research work, the results of which were based on the feedback received from the participants of the FDP.

Sampling and design of tool

As the present study was limited to the evaluation of a particular programme, purposive sampling method was applied, and all the FDP participants were considered the sample. The faculty members and academics those constituted the sample were from state open universities (SOUs), directorates of distance education (DDEs), dual-mode institutions and private deemed-to-be universities in India.

The research tool was designed as a semi-structured questionnaire and divided into three parts. Part I had four questions based on the five-point Likert scale, enquiring about the relevance and comprehensiveness of the training curriculum, usefulness of the practical activities and the logistics. Part II had 17 items investigating the attainment of the programme objectives and was designed using the four-point Likert scale. Part III carried six open-ended questions related to training materials, expectations, content and general suggestions for further improvements to the programme. Information related to the profile of trainees was collected at the time of registration.

Data collection procedure and analysis

For the purpose of data collection, the survey was conducted in two stages. In the first stage, the survey questionnaire was administered to all participants immediately after the completion of the FDP, with a request to fill the questionnaire and return it to the programme coordinators at the training venue. Respondents of the administered tool represented 18 different institutions, spread over 12 different states of India, as given in Table 1.

A total of 25 questionnaires were distributed and answered. The second phase of the survey was conducted after six months of the completion of the programme. The semi-structured tool was divided into two parts; the first part had five close-ended questions based on the four-point Likert scale and the second part had one open-ended question. The tool (via Google Form) was sent to the participants by email, and 24 participants submitted their feedback online. Quantitative data were analysed using the simple statistical tool, Microsoft Excel, and findings were reported in frequency and percentage. Content analysis method was adopted for analysing qualitative data. Different themes and training areas suggested by the participants were categorized and listed by eliminating repetition.

Findings and discussion

Profile of respondents

One of the objectives of the study was to collect personal information about the trainees such as their gender, designation, experience and discipline of specialization. While analysing their profiles, it was found that 60% of the participants were male and 40% were female (Table 2). Regarding their designation, 76% were Assistant Professors, 8% were Associate Professors, while 12% were Professors. The reason for Assistant Professors being the majority was because the programme was aimed at imparting training and augmenting the skills of new entrants to the ODL system. Institutions were also requested to nominate on priority those faculty members who were new to the ODL system and had not attended any such programmes earlier. In spite of this, it is interesting to report that 12% of the participants were Professors. This fact not only reflects the strong zeal of senior professionals towards acquiring knowledge and skills but also the popularity and need for specialized activity related to the design and development of SLMs.

The analysis also revealed that participants had varied teaching experiences, ranging from one year to 30 years. About 48% had an experience of 1–10 years, 32% had between 11–20 years of experience while 20% of the participants had an experience of 21–30 years. Regarding their specialization, it was found that 28% specialized in commerce and management, 20% in different subjects of social sciences, 16% in humanities and 12% each came from education, computer science, natural sciences and health sciences. Results indicate that a mixed group of professionals were keen on attending the programme and most of them were new entrants to the ODL teaching and learning setup. These findings, pertaining to trainees' profiles, may be helpful in finalizing the target group before organizing any staff development programmes in the future.

Level 1 of Kirkpatrick's evaluation taxonomy refers to the participants' satisfaction and motivation (Phuong et al., 2018). Therefore, efforts were made to get the reactions of trainees on various aspects of the FDP, and results, as reflected in Table 3, show that the majority of the participants (60%) found the programme to be “excellent” in terms of curriculum comprehensiveness and relevance. On the usefulness of practical activities towards designing the SLMs, the majority (60%) found it to be “excellent” and 28% reported that it was “very good”. A majority of the faculty members (52%) opined that overall, the programme was “very good”, while 44% found it to be “excellent”. These findings are in tune with the studies conducted by Piryani et al. (2018) and Omar et al. (2009), who reported that the FDP was interesting and relevant for a majority of the participants and that the confidence level of trainees increased significantly after the programme. Findings also indicated that though the FDP was relevant and useful, more attention was to be paid towards logistical arrangements. The feedback related to the arrangements (i.e. infrastructure, technological arrangements and food) indicated the need to pay more attention to these areas, as these were equally rated as “excellent”, “very good” and “good” (32% each) by the participants. In the previous studies, Asgar and Ratra (2020) had also laid emphasis on the creation of sufficient infrastructure and the use of eresources and online tools by institutions, while Das and Biswas (2017) had pointed to the need for improvement in the quality of food and beverages.

Evaluation should also primarily cover the learning objectives of any training programme (Konopasek et al., 2017). The present evaluation was also aimed at ascertaining whether the programme objectives were attained. Therefore, the 17 objectives framed while designing the programme were listed in the tool and administered to the participants. Results (Table 4) indicate that all objectives were achieved to a “large extent”, except the objective mentioned in S.No.2, which was achieved to “some extent”. If we also include the objective in S. No. 3, where the difference between “large extent” and “some extent” is very narrow, then a conclusion can be drawn that the discussion over the draft National Education Policy (NEP) and UGC (ODL) Regulations, 2017 should have been done more elaborately and effectively. However, results of a relevant study conducted in Iran by Farjad (2012) reported contrary to the results of the present study and revealed that the training course could not meet its objectives, and that its quality needed to be improved through proper training design, implementation and financial support.

Training resources have an indispensable role in teaching and training endeavours. According to Ajoke (2017), learning materials enrich the training activities by making them more appealing. These resources also help participants acquire knowledge and skills in a more convenient way. With STRIDE being a knowledge centre in the ODL system, it has published a plethora of handbooks and manuals on different themes of distance and online education. During the FDP, various print and e-resources related to the design and development of learning materials were made available to the participants. Statistics presented in Figure 1 suggest that the training and learning resources provided to trainees during the FDP were sufficient and fulfilled their immediate requirements, as 88% found the materials to be “adequate”. A previous study conducted by Das and Biswas (2017) on capacity building for IGNOU teachers through the virtual training lounge also reported that workshop content and e-resources were found to be “very useful” by the trainees.

A majority of the participants (96%) were also of the view that the workshop was up to their expectations, and only 4% reported contrary to the above view (Figure 2). An open-ended question sought the participants' suggestions on some themes that they felt should have been part of the FDP curriculum, so that these could be reported in this paper. Training areas suggested by them, as presented in Table 5, make a strong point for the consideration and infusion of emerging media and methods in conventional teaching as well as learning materials and strategies. Further, it is also quite evident from the analysis of suggested themes that the use and integration of digital tools in SLMs were of paramount importance to the trainees. Findings indicate that ODL teachers would be more interested in learning about digital initiatives, MOOCs, OERs, LMS, e-SLM, DEB, curriculum and SLM evaluation. Therefore, distance education institutions will have to gear up for infusing more money in acquiring emerging tools and techniques to empower their faculty through robust CB initiatives. On similar lines, Asgar and Wani (2019) had also suggested that ODE institutions must acknowledge new learners' needs and revamp delivery technologies, especially in the development of print SLMs.

Another round of survey was conducted after six months of the FDP for the purpose of post-training evaluation (follow-up activity). This exercise was meant to assess the levels at which participants had actually benefitted, in terms of building their competencies and the levels at which they were contributing to their workplace after attending the programme. Highlighting the need and importance of follow-up assessment strategies, Pulist (2017) opined that training effectiveness might reach the optimal level if the employees were successfully utilizing the newly learnt knowledge and skills at their workplaces and transferring them to their colleagues. Certain vital aspects dealing with the expected gain of knowledge and skills by trainees were identified for the post-training evaluation phase. Results obtained during this phase of evaluation are discussed below.

SLMs are self-explanatory, self-contained and self-directed, which stimulate independent learning among distance learners (IGNOU, 2016). At the beginning of the FDP, focus remained on trainees' orientation in structuring and customizing SLMs, so that the characteristics of SLMs are maintained. It is evident from Figure 3 that 88% of the trainees felt capable of giving proper structure to SLMs unit/module, to a “large extent”, in their university/institute after attending the specialized FDP. Only 8% of them reported that they were able to structure a unit to “some extent”.

Check your progress questions (CYPQs) or self-assessment questions (SAQs), in-text questions and unit/module-end exercises are a vital and integral part of quality ODL learning materials. These are utilized by learners to assess their learning on their own, with the help of model answers provided at the end of the unit or module. Results showed that in the areas of writing CYPQs/SAQs, unit summary, exercises and model answers, the majority of the faculty (76%) reported that they were able to design/write them to a “large extent”, while 20% of the respondents found themselves capable of performing these tasks to “some extent” (Figure 4). Findings of the study also revealed that although faculty members were capable of designing and writing SAQs/CYPQs, in-text questions and model answers, there was scope for further improvement in these areas as 20% of the participants were able to perform these tasks only to “some extent”. Gowthaman and Awadhiya (2017) have also reported that trainees gained the desired knowledge and skills from the training; however, few gaps needed to be addressed.

According to Asgar and Wani (2019), it is impossible to deny the growing influence and demands of open educational resources (OERs), which have a great potential to be used as print learning materials' content. OERs make education accessible to all by providing free educational content (Saxena and Singh, 2019). Hayman (2018) opined that OERs make education affordable and have the potential to make students and teachers less dependent on commercial books. Realizing the expediency, several ODL institutions have already started utilizing these freely available resources, according to the permissible licences. However, results of the present study indicated that in the domain of searching OERs and developing content for the print SLMs, 56% of the participants expressed their capability for doing so to a “large extent”, 36% could do it to “some extent” and only 4% of trainees reported that they were “poor” in searching and utilizing OERs as content for learning materials (Figure 5). Although results show the popularity of OERs and their promising prospects in digital and online environment, a large number of participants were capable of searching and using OERs only to “some extent”. This gap could be addressed by building the capabilities of the participants further through targeted and focused training as well as practice by the individual faculty.

During the FDPs on the design and development of SLMs conducted by STRIDE, participants are also oriented towards three aspects of editing print SLMs:

1. Content editing.

2. Language editing.

3. Format editing.

Responding to a question on their post-training competency in editing print SLMs, 64% of the trainees opined that they were doing the editing tasks to a “large extent”, while 32% said that they were competent to do it to “some extent”, as reflected in Figure 6. Results indicate that the majority of trainees were capable of performing editing tasks to a “large extent”. However, institutions may plan separate and exclusive training strategies for improving the editing skills, as this area requires a lot of practice for polishing the faculty's language proficiency and editing calibre.

One of the parameters to judge post-training competencies of participants was the transfer of training (TOT), which is considered to be the best practice to retain, implement and strengthen the knowledge and skills gained during training. In the present study, in terms of the transfer of knowledge, training and skills involved in the learning materials preparation, the majority of trainees (68%) expressed that they were capable of providing the training skills they obtained to their fellow colleagues to a “large extent”, 20% noted that they had this competency to “some extent” while 8% opined that they were “poor” in this aspect (Figure 7). These findings are in line with those of Adhikari (2018), who found that most trainees were successful in transferring a number of training skills. Although the number of respondents who were poor in TOT is only 8%, institutions need to focus on these people and utilize the services of the in-house trained faculty to further build the capacity of their own colleagues. This exercise will help institutions saving training costs as well as refresh and polish the faculty's training skills.

Examining the TOT among human resources of Iranian medical science universities, Sayadi et. al. (2017) also suggested setting up an institutional mechanism headed by an experienced faculty to support and motivate trainees to utilize newly learned skills and transfer them to their work.

While responding to a general open-ended question regarding other areas where the training programme was found to be useful, participants expressed the following views:

1. It was helpful in understanding ODE in totality.

2. It provided an overview of SLM preparation, and they learnt key aspects (to design, develop, write and edit the SLMs).

3. They became familiar with blending technology into learning materials.

4. They learnt how to develop print materials not only for university learners but also for school students as well.

5. They realized the importance and vitality of SLMs for self-directed learning.

6. Their concepts on ODL were built and doubts related to distance and online education clarified.

7. They learnt how to prepare learning materials for students in Sanskrit.

The above opinions of the participants suggest that the FDP was also useful towards building concepts and understanding of the ODE system. Trainees became familiar with the characteristics and significance of SLMs as well as the synchronization of technology with the learning materials. Clarifications on different aspects of ODE were also provided during the training.

The findings of both levels of evaluation (reaction and learning) indicate that a majority of the objectives were achieved during the FDP, and that it successfully addressed the participants' expectations. The training curriculum was found to be “comprehensive and relevant”, while the training materials were reported to be “adequate” and the practical activities on SLMs writing were termed as “excellent”. Further, the programme was effective in developing the knowledge and skills required for structuring SLMs, designing SAQs, writing unit summaries, conducting exercises, creating model answers and augmenting the editing skills of the faculty. The majority of trainees reported that they could adequately provide training to their colleagues.

The outcome of this study was in conformity with Kurniawati et al. (2017), who found that the training programme had a medium to large effect on teachers' attitudes and knowledge. However, participants also emphasized the need for in-depth and elaborate discussions on education policy and regulations vis-à-vis the design and development of print resources. They suggested various themes, especially the integration of digital tools in the SLMs, for inclusion in future FDPs. The results further revealed that about 25% of the participants could structure SLMs and write SAQs, summaries, exercises and model answers only to “some extent”; similarly, a significant number of the participants (46%) reported that they could search and use OERs only to “some extent”. Hence, this gap needs to be addressed by their parent institutions by conducting separate training or follow-up activities. Results also indicated that the infrastructure and logistical facilities should be improved, and more time should be allocated for activities. This finding lends support to Omar et al. (2009), who suggested equal emphasis on theory and practical sessions during the training.

Conclusion and recommendations

Faculty development is important for improving teachers' proficiency in specific tasks and their knowledge, skills and behaviour (Steinert, 2014). FDPs help in fulfilling the expectations of individual faculty members by benefitting them professionally, which ultimately benefits students and institutions (Phuong et al., 2018). IGNOU and other Indian SOUs have successfully conducted FDPs on different themes of ODL for capacity building of their teachers and academics. However, very few FDP evaluation studies, particularly on SLM-related trainings or workshops, are available. This may be because evaluation of training is not considered a part of the strategy for implementing a successful programme. Results of this study reveal that on major indicators such as comprehensiveness and relevance of curriculum, attainment of the programme objectives, quality of training materials, fulfilment of trainees' expectations, practical activities, provision of a learning environment and development of the faculty's skills and competencies in SLMs design and development, the FDP was found to be successful and effective. Results are consistent with the prior study done by Dimri and Misra (2006), who reported that teachers derived the expected benefits from CB activities at IGNOU. However, during the investigation, a few shortcomings were also unveiled and recommendations are being made to fulfil those gaps and make forthcoming programmes more useful. These suggestions may also be crucial and significant for other ODL institutions, trainers, curriculum designers and authorities for enhancing the quality of training activities.

1. Infrastructural arrangements required a bit more attention and improvement to avoid technical glitches during the technical/theoretical/practical session or presentation by the participants. Nguyen (2018) reported a lack of training policies, shortage of workforce and technological infrastructure for online training at Hanoi Open University.

2. More extensive and elaborate discussions may be held over policy matters such as the National Education Policy, UGC regulations related to distance and online learning and curriculum design.

3. Considering the exclusive nature of the SLM workshop, where participants are expected to develop a draft module or unit of the print learning materials, there is a need to provide more practical sessions. Uzun (2016) also advocated a balance between the theory and practical aspects in the FDPs.

4. Specialized programmes of this kind should be organized more frequently by STRIDE, IGNOU and other ODL institutions, in view of the importance of SLMs and requirements of such exclusive training. Murthy (2004) also advocated the CPD model for ODL staff development in specialized areas.

5. More exhaustive and practical-based training is required for improving the faculty's capacity in searching/using OERs for print SLMs and improving their editing skills. Johry et al. (2019) highlighted the shortage of a skilled workforce and emphasized skill-based training.

6. Themes such as digital initiatives, production of e-materials/e-SLMs, design of MOOCs, OERs, LMS and the evaluation of learning materials should also be included in the training curriculum. Further, institutions must strive towards the use and integration of digital tools into the SLMs. Gbenoba and Dahunsi (2014) and the UGC (2020) also recommend the infusion of emerging tools and techniques in traditional SLMs. Open universities should take initiatives towards the CPD of teachers in the emerging areas of technology.

7. Trainees may be encouraged to transfer knowledge and skills to colleagues, who develop print SLMs for any academic programme. This will benefit the individual faculty and institution as TOT improves the efficiency and productivity of the whole staff (Sayadi et al., 2017).

8. More evaluation studies may be conducted by researchers targeting short-term and long-term specialized workshops in curriculum planning and development. According to Gade (2020), timely evaluation helps in making training events more relevant and useful.

Figures

Figure 1

Were the training materials adequate?

Figure 2

Did FDP meet your expectations?

Figure 3

Structuring SLM unit/module

Figure 4

Writing unit-end components (CYPQs, summary and model answers)

Figure 5

Searching OERs and developing SLM content

Figure 6

Editing of SLMs

Figure 7

Transfer of knowledge and skills

Table 1

Names of institutions, states and number of participants nominated

S.No	ODL/dual mode institutions/state open universities	States	No. of participants
1	School of Distance Education, the University of Kerala	Kerala	01
2	Department of Education, Magadh University, Gaya	Bihar	01
3	Integral University, Lucknow	Uttar Pradesh	01
4	S.V. Subharti University, Meerut		01
5	Institute of Distance and Open Learning, Gauhati University	Assam	01
6	Directorate of Open and Distance Learning, Dibrugarh University		02
7	Mahatma Gandhi Antarrashtriya Hindi Vishwavidyalaya	Maharashtra	01
8	Datta Meghe Institute of Medical Sciences		01
9	Kavikulaguru Kalidas Sanskrit University		02
10	GITAM deemed to be University	Andhra Pradesh	03
11	Rabindranath Tagore University	Madhya Pradesh	02
12	Sri Ramachandra Institute of Higher Education and Research	Tamil Nadu	02
13	Madurai Kamaraj University		01
14	Saurashtra University	Gujarat	02
15	Uttarakhand Open University	Uttarakhand	01
16	Jagannath University	Rajasthan	01
17	Karnataka State Open University	Karnataka	01
18	JSS Academy of Higher Education and Research		01

Table 2

Characteristics of respondents (n = 25)

Indicator/s	%	Frequency
Gender
Male	60	15
Female	40	10
Designation
Professor	12	03
Associate Professor	08	02
Assistant Professor	76	19
Assistant Registrar (academic)	04	01
Experience (in years)
01–10	48	12
11–20	32	08
21–30	20	05
Discipline (area of specialization)
Education	12	03
Sciences and health sciences	12	03
Commerce and management	28	07
Social sciences	20	05
Computer science	12	03
Humanities	16	04

Table 3

Trainees' opinion (vis-à-vis curriculum, practical and infrastructure, etc.) (in %)

Questions	Excellent	Very good	Good	Satisfactory	Not satisfactory
What about relevancy and comprehensiveness of the FDP curriculum?	60	32	08	0	0
How do you rate usefulness of hands-on-activities/practical on SLM design and development?	60	28	12	0	0
What is your opinion about arrangements (infrastructure, food, refreshments, etc.)?	32	32	32	04	0
What is your overall pinion about the programme?	44	52	04	0	0

Table 4

Participants feedback on achieving programme objectives (in %)

S,No	Question: To what extend this FDP was successful in achieving following objectives?	To large extent	To some extent	Poor	Not at all
1	Define open and distance education and latest developments in ODL	76	24	0	0
2	State major features of the draft NEP	32	68	0	0
3	Discuss UGC (ODL) Regulations, 2017 vis-à-vis design and development of print SLMs	56	44	0	0
4	Explain SLMs	88	08	04	0
5	List out unique characteristics of SLMs	92	08	0	0
6	Illustrate various processes involved in print SLM preparation	72	28	0	0
7	Structure SLM unit/module	92	08	0	0
8	Write introduction	84	16	0	0
9	Frame objectives and learning outcomes	92	08	0	0
10	Differentiate between objectives and learning outcomes	80	20	0	0
11	Describe end of unit components (CYPQ/SAQs, summary/unit-end questions, model answers, etc.)	76	24	0	0
12	Develop SLM content	68	32	0	0
13	State different steps of editing and finalization of content	68	32	0	0
14	Integrate emerging tools/technologies in SLMs	76	24	0	0
15	Identify issues related to plagiarism and copy rights	68	28	0	01
16	Searching/using OERs as print material content	64	36	0	0
17	Provide platform to exchange ideas on practices of different ODL institutions in SLMs development	76	24	0	0

Table 5

Topics suggested for inclusion in future curriculum

Question: What more content/topics could have been added in this programme?
Digital initiative of GOI and redefining role of teacher in 21 century	Open educational resources/searching digital resources/OERs	Different methods of SLM evaluation
Development of content for SWAYAM/MOOC courses	Integration of digital media in SLMs	Production of e-materials and e-SLMs
Emerging tools for creating online content	Technology, OERs and using latest software and technology to include more features in SLMs	Function and the role of Distance Education Bureau (DEB)
Practices in different ODL institution vis-a-vis SLMs and digital integration	Learning management system and its administration	Curriculum evaluation