A study of the self-perceived competencies regarding the knowledge and skills of healthcare providers in the Expanded Program on Immunization in Thailand: A mixed-methods study

Abstract

Purpose

The Expanded Program on Immunization (EPI) has still been found to offer services that do not comply with standards. The purpose of this paper is to ascertain competency in terms of the knowledge and skills of the EPI staff.

Design/methodology/approach

The research design was a mixed-methods approach. The quantitative method employed a questionnaire survey on the perceived competency of 382 EPI staffs from six regions in Thailand. This was paired alongside of the qualitative method, where four staffs were in-depth interviewed, and the performance of the EPI staffs was observed.

Findings

The overall perceived competency in the work of immunization was at a high level. A comparative analysis between the quantitative and qualitative data showed findings in three categories. First, the perception of competency was high, and performance conformed to standards in the preparation of the setting and equipment for providing the service; second, the perception of competency was high, but in the performance of their work the participants did not comply completely with standards for scheduling the immunization appointments or for vaccine storage; and third, the perception of competency was either moderate or low, and the performance of work was inadequate for vaccine estimations, registering reports, and dealing with adverse events following immunization.

Originality/value

The findings showed a gap between perception of knowledge-and-skill competency and actual practice in EPI service. Effective cooperation among involved organizations in order to improve the standard of performance in expanding the quality of EPI service provision in Thailand is suggested.

Citation

Rutchanagul, P. and Sangnimitchaikul, W. (2019), "A study of the self-perceived competencies regarding the knowledge and skills of healthcare providers in the Expanded Program on Immunization in Thailand: A mixed-methods study", Journal of Health Research, Vol. 33 No. 3, pp. 208-218. https://doi.org/10.1108/JHR-08-2018-0082

Introduction

Immunization lies at the core of the control and prevention of major communicable diseases in Thailand. The provision of the Expanded Program on Immunization (EPI) in Thailand has received the praise of the World Health Organization as one of the 13 countries worldwide to have met with success in developing a vaccine-preventable disease-monitoring system for child immunization[1]. The policy has required that from birth, Thai children receive primary immunization against 11 diseases, aimed at the ultimate eradication and elimination of vaccine-preventable communicable disease[2]. In order to achieve its main objective, the successful development of EPI in Thailand has depended on the following four components: the vaccine component, produced in accordance with set standards; the budgetary component, which supports the production and research of new vaccines that will stay abreast of newly occurring diseases; the administrative component that manages production, storage and delivery and management within the service units; and personnel or providers engaged in immunization, which is a very important component inasmuch as it functions as the main mechanism and lies at the very heart of the process that will steer the EPI’s aims toward achieving its objective[3].

It is currently estimated that there are approximately 20,000 EPI staff members in Thailand. However, it was discovered that there are regular personnel transfers in this line of work, causing the services being provided in the area of immunization to frequently fluctuate. Moreover, the nature of this line of work requires the development of continually updated knowledge and skills in rendering services, since the specialized knowledge needed in the administration of vaccines continually undergoes change, and new concepts are constantly arising. For this reason, staff engaged in the work of immunization are crucial to the mechanism that will determine whether the process of immunization in Thailand will achieve the desired work standard. A review of the service standards pertinent to the work of immunization found that carrying out this work in Thailand still had problems of several kinds. Based on information gleaned from the supervision of work performed in the area of immunization by the Department of Disease Control, issues were still found pertaining to services that did not comply with standards, such as the vaccine management system and the cold chain system, which continued to be non-standard compliant[4]. The aforementioned data were either a reflection of human error, incorrect understanding or low skills. Therefore, it is crucial that the managers of the immunization program, especially in the provider group in Thailand, be at least minimally aware of knowledge-and-skill competency levels and how immunization services are provided.

To date, numerous studies have been carried out on parental attitudes toward immunization and healthcare providers[5–7], the vaccination competency of nurse students and nurses[8–10] and knowledge or vaccination competency from the perspective of immunization providers and clients overseas, especially in western countries[11–13]. However, there is no available research on the knowledge and skills of those providing immunization services in Thailand. Consequently, this research describes a systematic study that will provide accurate data on the knowledge-and-skill competency awareness of service providers. These data were gathered in six regions of Thailand to provide an overall picture nationwide. Research findings can serve as empirical evidence for policy setting, and for a plan of work or for guidelines for furthering the capabilities of personnel on a level consistent with the expected competency of a provider. This information can then be applied to the development of work in a continuous and systematic fashion, and contribute to bringing the work of immunization to a greater level of success than before.

Methods

Study design and participants

This study used a mixed-methods approach using a triangulation convergence model. For the convergence model, the researchers collected and analyzed quantitative and qualitative data separately and then the quantitative and qualitative findings were converged by comparing and contrasting findings during the interpretation[14].

In the quantitative method phase, participants were EPI staff consisting of physicians, nurses, pharmacists, public health scholars and public health officials. The sample size was determined using the G*power program 3.1.9.2. software. A sample size of 272 persons was required to achieve a power of 0.95 and an effect size of 0.2 at the α level of 0.05. Because the response rate in the postal mail survey was usually low, the sample number was increased by 50 percent. Therefore, the sample size that was expected to be used contained a total of 408 persons, requiring data to be compiled using the stratified random sampling method. Data were taken from those performing the work in six regions nationwide. One province was then selected from each region: Chon Buri in the East, Phetchaburi in the West, Chiang Mai in the North, Krabi in the South, Khon Kaen in the northeast and Bangkok in the central region.

For the qualitative methods phase of this study, four healthcare providers involved in immunization service from one health service setting in each region were selected through purposive sampling for in-depth interviews and non-participant observation. In summary, six health service settings were included in this phase and 24 healthcare providers from the six regions were interviewed and observed while performing their EPI services.

Research instruments

Two research instruments were employed: the demographic questionnaire and the Perceived Self-Competency for EPI Service Provision Evaluation Form for the quantitative phase of the study. Guidelines for the in-depth interviews and for the observations were used in the qualitative phase.

A demographic questionnaire was used to obtain personal data on the EPI staff, including sex, age, education level, length of time working in the field of immunization and characteristics of work.

The Perceived Self-Competency for EPI Service Provision Evaluation Form was developed by the researchers for measuring the staff member’s perception of his/her competency in performing the work of immunization, including the self-confidence of the person in his/her knowledge and skill in rendering the service and managing the work of immunization. This tool contained a total of 80 questions covering the competency of EPI service based on the standard of service for the EPI training program of the National Vaccine Institute and the quality standards for immunization practices in the Department of Disease Control, Thai Ministry of Public Health. This instrument rated each item by using a five-point Likert scale that ranged from 1 (indicating no confidence) to 5 (indicating high confidence). The score results were subdivided into three levels based on criteria derived from Best’s criteria[15]. These were 1.00–2.33, which meant a low perception of competency; 2.34–3.67, which meant a moderate perception of competency; and 3.68–5.00, which meant a high perception of competency. Content validity was assessed by five experts. The Index of Item Objective Congruence (IOC) was 0.98. The result of the internal consistency using the Cronbach’s α was 0.97.

In-depth interview guidelines for the EPI staff were used pertaining to the perceptions of competency in EPI service. The interview guidelines included five items: How do you usually provide immunization services?; As an EPI staff member, how confident are you in providing immunization services?; Which performance do you have high-perceived competence in and which do you not?; What kind of difficulties have you encountered? Why?; and What are your expectations regarding the development of your knowledge and skills in providing immunization services? Instrument quality was assessed for content validity by five experts and tested by two EPI staff members in order to ascertain their understanding of the questions.

The Evaluation for EPI Service Observation Guidelines was used to assess the work being performed in a well-baby clinic. The guidelines contained a 51-item checklist that needed to be followed and monitored for the EPI service and was developed based on the evaluation forms for the quality standards for immunization practices, Department of Disease Control. Content validity was assessed by five experts and reliability was high (IOC=0.91, inter-rater reliability=0.90). The skills in providing immunization service and all activities encountered by an EPI staff member were also collected through field notes that the researchers took during observation. Field notes were written while the researchers were observing.

Data collection

For the quantitative method phase, the questionnaires, consisting of the demographic questionnaire and the Perceived Self-Competency for EPI Service Provision Evaluation Form, were sent out to approximately 70 EPI staff members in each provinces in the six regions – Chon Buri, Phetchaburi, Chiang Mai, Krabi, Khon Kaen and Bangkok – by postal mail, and 91 percent of them – 382 persons – returned their questionnaires.

For the qualitative method phase, the researchers selected the participants from an EPI service unit in each of the randomly chosen provinces as they did with the quantitative data. Four EPI staff members in the selected service unit were purposively recruited as key informants per unit. The purpose of this phase was to conduct interviews and non-participant observation on the work being performed in the area of immunization by using the in-depth interview guideline for the EPI staff and the observation guideline. During observation, the researchers took field notes to record the activities undertaken that may not have been identified from the audio recording.

Data analysis

Data from the quantitative method were analyzed by using SPSS version 22 with the level of significance at 0.05 in order to describe the frequency and percentage of the demographic data and the perceptions of competency in the administration of immunization of the EPI staff. The data derived from the qualitative method consisted of data based on behavioral observations for which frequencies and percentages were calculated. Additionally, the data from the interviews and field notes from each observation were analyzed using content analysis[16]. The qualitative data analysis was joined with the quantitative data (triangulation) by comparing the two data types in order to arrive at the research results.

Ethical considerations

This research was approved by the Second Ethics Subcommittee on Human Research, Thammasat University, No. 024/2558. The information sheet that explained the data were sent to those that had joined the research project and to everyone in the sample groups in order to explain the details of the project and the protection of rights that applied to the sample groups. All participants who agreed to be a part of this study were required to sign the consent form that was then returned by postal mail together with the questionnaire.

Results

Regarding the quantitative method phase, among the total of 382 participants, 320 were female (83.8 percent) and 62 were male (16.2 percent), with an average age of 40.8±9.04 years. Results showed that 21.5 percent of the participants were between the ages of 36 and 40 years, followed by 18.3 percent that were within 41–45 years of age. Furthermore, 77 percent of the participants had finished their education at bachelor’s degree level. A 68.3 percent majority of the participants were working in the nursing profession, followed by an estimated 19.1 percent of individuals who were public health staff members. The average length of time spent working in the area of immunization was 7.3 years (SD=6.8), with 43.7 percent of the participants having worked from one to five years, followed by 23.0 percent that worked from 5.1 to 10 years. As regards the workplaces, 48.5 percent of the participants worked in primary healthcare centers or in health-promoting hospitals in sub-districts, followed by 26.4 percent that worked in community hospitals. Additionally, 70.4 percent of the participants had been previously trained on an immunization course.

Results for levels of competency regarding knowledge and skills

The results for the quantitative data showed that the participants had a high perception of knowledge-and-skill competency in EPI service. They had an average score of 3.95 (SD=0.61). When compared to the competency score as classified by EPI service, it was found that the participants had the highest perceptions of competency in preparing and administering the vaccines ( x ¯ = 4.12 , SD=0.69), followed by vaccine storage and cold chain system ( x ¯ = 4.05 , SD=0.71), and the lowest perceptions of competency in dealing with adverse events following immunization ( x ¯ = 3.82 ). The details are as shown in Table I.

Results for behavioral observations of performing immunization services

The results for non-participant observations of the performance of EPI services showed that what the participants did was either incomplete or incorrect according to standards for administering the vaccines, vaccine storage, vaccine estimation and vaccine report registration, and dealing with adverse events following immunization. Details are shown in Table II.

The qualitative findings of both observational and interview data revealed that the participants were confident in their skills in administering the vaccine shots. They also felt that they were proficient in their work and in developing work standards in the areas of vaccine storage based on the cold chain system. On the other hand, participants were not confident in their performance of the following: scheduling immunization appointments for a group that was afflicted with health problems or that had come in for vaccinations later than the time appointed; vaccine estimation and vaccine report registration; and caring for adverse events following immunization.

Both quantitative and qualitative findings were used to summarize the knowledge-and-skill competency of the EPI staff members in Thailand. This summary proceeded as indicated below.

Category 1: those involved in the work had high competency perceptions. The work they performed conformed to standards of preparation of the site and equipment for vaccine services and in reporting to parents on giving vaccines. The details are shown in Table III.

Category 2: while those doing the work had high competency perceptions, the work they did was either incomplete or incorrect according to standards for providing services and for vaccine storage. The details are shown in Table IV.

Category 3: although the participants had a high perception of their knowledge-and-skill competency in all aspects of EPI service, they had the lowest of the three levels of perceived competency as follows: administering basic life support, and referring the patient when adverse reactions occurred; identifying the severity of symptoms following immunization and reporting the coverage for routine vaccination services. Their work was either incomplete or incorrect according to standards for making vaccine estimations, registering reports and dealing with adverse events following immunization (Table V).

Discussion

According to the results of the study that examined the perception levels of knowledge-and-skill competency in EPI service, it was found that the participants had high competency perception levels in EPI service provision ( x ¯ = 3.95 , SD=0.61). The reason may be that a 70.4 percent majority of the participants had previously received training in immunization, so they had gained practical knowledge in the administration of vaccinations from experts or from those that were qualified in this field, and then they passed on their own expertise. Nowadays, training programs – three-day programs and one-day programs – are usually completed annually by the Thailand National Vaccine Institute and Department of Disease Control. Many printed materials such as immunization handbooks, guidelines and textbooks are distributed throughout the country both in hard copy format and electronically uploaded files. The monitoring system in each Area Health is randomly carried out by the Division of Vaccine Preventable Disease, Department of Disease Control, on an annual basis. The participants thus acquired knowledge and experience from others and eventually developed a perception of their own competency[17, 18]. This corroborates a theory of Bandura[19], which offers a clear explanation of participant results, namely, that they had already been trained in immunization. The results indicate that they benefited significantly from good role models that had provided them with their knowledge and shared important experiences. Their output consisted of developing the knowledge of service providers and reducing problems and obstacles in work supervision through access to resources for counseling in the performance of their work. This generated clarity in the advice being offered and built up awareness of their job performance[20]. The fact that the participants had a high level of perception of competency in rendering their services in immunization may have resulted from their having worked for long periods of time, an average of seven years, and had an extensive amount of work experience. Furthermore, their experience was directly in the area of immunization, and they were bound to develop self-competency awareness to a high level. It was further revealed that the participants had the highest perceptions of competency in preparing and administering the vaccines, vaccine storage and use of the cold chain system. The above findings concur with a statement of Bandura, who asserted that performing a task successfully strengthens one’s sense of self-efficacy, thus leading to successfully accomplishing a task. In other words, having direct experience of mastering something is a powerful way of increasing one’s self-efficacy, and in this case, influenced the initiation and maintenance of vaccination competence[19].

Meanwhile, the participants had moderate and low perceptions of competency in the areas of administering care whenever adverse reactions following immunization occurred, and in making vaccine estimations and registering reports. The reason may be that the participants consisted mostly of nurses that had no specific duty or direct responsibility in these tasks. The very fact of never having had any experience in carrying out these functions or any specific duty or direct responsibility in carrying them out tends not to generate any perception of competency in work performance, and the work itself may be incomplete or non-standard. Yet, at the same time, there were participants with perceptions of competency that were high, but whose work was either incomplete or noncompliant with standards, especially in administering the vaccine shots and in their storage of the vaccines. There were other factors that affected their personal actions, including, in particular, environmental factors, or the context of the individual[21]. These factors led to job performance in which the results were not correct. For example, the arrangement of the vaccines in the refrigeration units was incorrect because of the limitation of the refrigeration equipment. There was no recording of the refrigeration temperature on the weekends from morning till evening, nor had any instruments been prepared to provide first-aid in the event of undesirable symptoms following immunization. Neither was a suitable space prepared for the observation of post-vaccination symptoms. This situation was the result of the work policies of the immunization units, which were very diverse. They depended on the context of the service sites and the affiliated principal work units. Accordingly, the immunization work systems were implemented in ways that generally differed. They may not have harmonized with the evaluation forms for the standards of the primary vaccine development group, Department of Disease Control, which served as the template in the construction of the research tools for the present study. Furthermore, the policy limitations within each of the work units caused the failure of budgetary support for the work unit itself and led to a shortage of personnel as well. These findings were consistent in that the nurses that were administering immunization shots were knowledgeable in their work, but were subject to certain limitations. Among these limitations was a lack of staff in the workplace, which impeded the services they were providing in administering the vaccines[22].

Conclusion

The data from this research indicated that the individuals engaged in this work have high-level perceptions of their own competency, which serves as a reflection of their knowledge and skill in immunization practices. Nonetheless, some problematic issues were found in the work of the healthcare providers that conflicted with certain stipulated work standards. Accordingly, these findings serve as clear evidence of the need to develop the competency of EPI staff in order to overcome the barriers and to sustain effective services in Thailand.

Recommendations

This study provides insights into national organizations in Thailand including the Thailand National Vaccine Institute or the Division of Vaccine Preventable Disease and the Department of Disease Control. The study can support recommendations for planning staff development policy to enhance adequate and systematic training, monitoring and evaluation throughout the EPI staff’s working life. Moreover, the effectiveness of traditional training might be reviewed and added to new methods that will be able to overcome the constraints between high-perceived competence and unmet standard practices. Additionally, scheduling and rescheduling of immunization, vaccine storage in the cold chain system, vaccine administration and preparation for adverse effects following immunization should be more closely monitored.

Table I

Scores of the person’s perception of immunization competency with regard to those that performed the immunization services

Perception of the person’s immunization competency	Possible score	Actual score	x ¯	SD	Interpretation of results
Making preparations and administering the vaccines	1–5	1–5	4.12	0.69	High
Vaccine storage and cold chain system	1–5	1–5	4.05	0.71	High
Communicable diseases that are preventable by vaccines and EPI scheduling	1–5	1–5	4.00	0.63	High
Basic knowledge pertaining to immunization	1–5	1–5	3.91	0.67	High
Vaccine estimation and vaccine report registration	1–5	1–5	3.87	0.80	High
Dealing with adverse events following immunization (AEFI)	1–5	1–5	3.82	0.68	High
Overview of a person’s perception of competency regarding immunization	1–5	1–5	3.95	0.61	High

Note: n=382 persons

Table II

Frequency and percentage of participants who performed immunization services correctly

	Correct and complete			Correct and complete
Behavioral observations of performing immunization services	n	%	Behavioral observations of performing immunization services	n	%
Making preparations and administering the vaccines			Vaccine-preventable communicable diseases
Health assessment and vaccination screening	20	83.3	Rescheduling vaccination appointments for interrupted or delayed routine vaccinations	4	16.7
Preparation of the site and vaccination equipment	24	100	Vaccine estimation and vaccine report registration
Providing services/vaccine shots	20	83.3	Making vaccine estimations	16	66.7
Reporting to parents after giving vaccines	24	100	Recording vaccine data via an electronic database	16	66.7
Vaccine storage and cold chain system			Registering reports	16	66.7
Vaccine storage and the cold chain system, comprising vaccine preservation	16	66.7	Reporting coverage for vaccination services	16	66.7
Temperature control and regulations	12	50.0	Dealing with adverse events following immunization
Emergency management of the cold chain system	16	66.7	Identifying the severity of symptoms following immunization	20	83.3
			Observation after vaccination	12	50.0
			Administering basic life support and referring the patient	8	33.3

Note: n=24 persons

Table III

Category I: high competency perceptions and high performance in providing services

Quantitative data	Interview data	Observational data
The participants had the highest perceptions of competency in preparedness and in administering vaccines. Their work consisted of setting up the service units, preparing for vaccination and recommendation regarding childhood immunizations for parents	The participants had confidence in their preparedness and in administering the vaccines. They were capable of health-screening assessment for vaccinations “I think that I can do a good job in screening children for vaccinations, administering the vaccines and making appropriate use of the equipment” “I can confidently answer parents’ questions in the well-baby clinic. I then advise my colleagues as we prepare the vaccines for children” “We’re confident in our knowledge of where to administer the vaccines”	Work was performed correctly and completely in preparing the service sites and the equipment that would be used. The workers checked the vaccine types and dosages that would be administered to be sure that they were correct. They also reported to parents on the vaccine regimen and the area where the shots would be given

Table IV

Category II: high competency perceptions but incomplete performance in providing services

Quantitative data	Interview data	Observational data
The participants had the highest perceptions of competency in scheduling the immunization appointments, health assessment and vaccination screening (contraindications and precautions)	The participants had confidence in scheduling appointments and in their skills in administering vaccine shots “Our greatest confidence is in giving the shots and scheduling the immunization appointments in a healthy child. We are 100% capable of this because we generally prepare the vaccines ourselves and we do so rather precisely” However, the participants were not confident in scheduling the immunization appointments in the groups of clients that had health problems, or that had come in for their vaccinations later than scheduled “On scheduling the appointments for children who delayed routine vaccination, we’re still not confident in answering everyone’s questions. We don’t remember everything completely” “We’d like to know more about vaccinations in general practice, vaccine contraindications, for instance, persons with immunocompromising conditions, which vaccines should patients receiving chemotherapy get, and minimum intervals between doses for children whose vaccinations have been delayed”	The majority of the providers could schedule vaccination appointments with normal cases but they were unable to schedule an appointment with clients that had health problems or delayed routine vaccination. Moreover, the work they did was about 83.3% correct and complete in terms of vaccination screening, whereas some clients in the health service setting have not been screened for contraindications prior to administering any vaccine
There was a level-2 high awareness of competency in vaccine storage and of the cold chain system, comprising vaccine preservation, temperature control and regulations and emergency management of the cold chain system	The participants were confident in their work and eventually developed work standards in the area of vaccine storage that was based on the cold chain system “We developed a vaccine storehouse. We developed it according to the curriculum and standards in the Drug Repository. So we tried to make use of the results we got to improve our work” “The part where we were confident was in the cold chain because it involved pharmaceutical work. We were in pharmaceutical work all along and we had a better understanding of registering the Hos XP data reports in the forms. We knew the reasons in doing this work and we were able to link to it so we could apply it to our work” “We were able to manage even when there was a power outage; and we were able to carry on, following the guidelines for maintaining the cold chain system in health-promoting hospitals in sub-districts”	Suitable refrigerators were chosen, and each type of vaccine was made available separately with a label indicating the name of each. Yet, there were some things that were done incorrectly, namely The arrangement of the vaccines in the refrigeration unit was not correct according to principles for proper storage, since, in some places, different types of vaccines were being stored together At times, there was no recording of refrigeration temperatures from morning to evening because the recording came to a stop on the weekends, there were no shift workers on duty and there were no vaccines kept in stock No water bottles or cold packs had been put inside the refrigerator, only other medical supplies within the refrigerator walls The two settings had no emergency management support plan in their cold chain systems

Table V

Category III: low competency perceptions and incomplete performance in providing services

Quantitative data	Interview data	Observational data
The participants had the lowest perceptions of their competency in dealing with adverse events following immunization. This consisted of being observant of symptoms following immunization, administering basic life support and referring the patient when adverse reactions occurred	The participants acknowledged that they still had no substantial knowledge or understanding and that their services were at a low level “We do not quite understand AEFI because of the difficult vaccine terminology” “What happens if I give a shot and make a mistake? What do I do then? Because I’m not a nurse, I don’t understand AEFI and I think it’s not my job” “I’ve never prepared any basic life support equipment, because, if there’s a problem or an adverse reaction, I’ll simply forward that patient to the ER, where there’ll be a physician who’s there regularly, and he can fix the situation right away”	The work they did was about 33.3–83.3% correct and complete. There were certain points, however, where they still did not function properly, as follows Most vaccinated areas were not prepared for observation of symptoms following the vaccinations, and symptoms were not evaluated Most service settings did not have basic life support equipment Half of the service settings did not have a service register that monitored vaccinated patients
The perceptions of competency in making vaccine estimations and registering reports pertaining to vaccination services had nearly the lowest scores	The participants had no confidence in their work in making vaccine estimations and in registering reports on vaccine services “Our no-confidence point was the vaccine disbursement. Saw only the data that they sent us. Didn’t understand much of it” “We had no confidence in this thing about the ‘stock card,’ or in that other thing about the report register, for these things mean nothing to us. We’re just fine”	The work they did was about 66.7–83.3% correct and complete. There were certain points where they still did not function properly, as follows There was no systematic entry of vaccine data into an electronic database; they were still recording what they did in a health register The number of vaccines that had been made available did not correspond to the actual disbursements, and there was no register available for auditing. Data could be viewed only with computers that transmitted the data to the central system