Abstract
Purpose
Out-of-pocket (OOP) payments continue to be a major method of financing healthcare in many low- and middle-income countries including Malaysia. Although macro-level data show that this is a substantial percentage of national health expenditure, at the grassroots level, the amount spent on health by households remains unknown in Malaysia. The purpose of this paper is to assess the validity and reliability of an adapted-for-purpose questionnaire designed to capture urban household health expenditures (HHEs) among Malaysian households.
Design/methodology/approach
This two-part study assessed content validity of the questionnaire using three experts and the reliability of the questionnaire through a test-retest study among 50 OOP-paying patients followed up at one private primary care clinic in Kuala Lumpur. This study was approved by the Malaysian Research Ethics Committee (NMRR-16-172-29311-IIR).
Findings
The validity of the 83-item questionnaire was high, with an item content validity index of 1.00 and a scale content validity index average score of 1.0 agreed to among the evaluating experts. In the test-retest reliability study, the majority of the categorical questionnaire items had perfect agreement values (k=0.81-1.00). Continuous questionnaire items were also found to be highly reliable with no significant differences between the test-retest segments and high correlation coefficient values (intra-class correlation coefficient>0.7).
Originality/value
The HHE questionnaire had excellent content validity and very high test-retest reliability. The results of this study suggest that this questionnaire could be used in Malaysian studies to determine actual urban HHE which is a first step toward developing universal health coverage for all.
Keywords
Citation
Munisamy, M., Thanapalan, T., Piwong, P., Panza, A. and Pongpanich, S. (2018), "Assessment of the validity and reliability of an urban household health expenditure (HHE) questionnaire in Kuala Lumpur, Malaysia", Journal of Health Research, Vol. 32 No. 1, pp. 24-35. https://doi.org/10.1108/JHR-11-2017-003
Publisher
:Emerald Publishing Limited
Copyright © 2018, Murallitharan Munisamy, Tharini Thanapalan, Pattaraporn Piwong, Alessio Panza and Sathirakorn Pongpanich
License
Published in the Journal of Health Reseach. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article ( for both commercial & non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
Introduction
Universal health coverage (UHC) has been made one of the global health priorities, as signified by its inclusion as Target 3.8 on the global Sustainable Development Goals[1]. UHC is based on the Primary Health Care principles from the 1978 Alma-Ata conference, whereby all peoples should be provided with equitable access to quality healthcare based on need, without the user facing financial hardships in the process of seeking and receiving care[2]. This has been a struggle for the low- and middle-income countries where low government expenditure in healthcare alongside low-quality provision of health services has led to citizens largely resorting to using expensive, private services and paying for them via out-of-pocket (OOP) payments[2].
In low- and even lower middle-income households, OOP payments for healthcare cause a great strain on families’ monthly household expenditures[3]. This has been significantly proven to be associated with financial difficulties. When sudden, calamitous health events such as a stroke or a heart attack occur, they may push an individual or family into huge debt and even render them bankrupt, a term called catastrophic health expenditure[4]. Incurring large monthly household health expenditures (HHEs) also often causes households to delay seeking healthcare, leading to further health complications and even poor health outcomes as seen in chronic disease patients[3, 4].
Malaysia is a middle-income country with largely subsidized public healthcare services[5]. Despite the availability of access for Malaysians to healthcare, perceived quality and logistics issues have led to widespread use of private services, especially in private primary care facilities[5]. As Malaysia has been deficient in developing pre-payment mechanisms for health, payments for these private health services are largely via OOP payments[6]. On a macro level, the evidence is clear that private payments in terms of total healthcare expenditure are high and this largely comes from OOP payments[5]. On a micro level however, it is unclear on how much the average Malaysian household spends on healthcare every month and how deeply this healthcare expenditures further impact their overall monthly expenditure. This pilot study aimed to assess the validity and reliability of an adapted-for-purpose questionnaire designed to capture urban HHEs among Malaysian households as part of a larger series of studies to develop a health microinsurance scheme in Malaysia[7, 8].
Methods
This study was conducted in two parts. The first part aimed to establish the content validity of the developed questionnaire via a sample of experts. The second part of the study was conducted to evaluate the reliability of the content-validated questionnaire via a test-retest design. Although the questionnaire had had been used and verified in other settings with a high-validity quotient[9-11], the content validity of the adapted version was nevertheless assessed by three experts: a health economist from Chulalongkorn University, Thailand, and two private primary care physicians who worked in private primary care clinics in Kuala Lumpur.
The setting for the test-retest portion of the study was located among private primary care clinics in Jalan Ipoh, a socio-demographically diverse suburb in Kuala Lumpur, with a population of approximately 268,000 people (2010 census)[12]. One private primary care clinic was purposively selected from among the list of private primary care clinics registered at the Ministry of Health, Malaysia, as the base location for conducting this portion of the study[13]. This study received ethical approval from the Malaysian Research Ethics Committee, Ministry of Health, Malaysia (NMRR-16-172-29311-IIR).
The HHE questionnaire used in this study was adapted-for-use in Malaysia from existing questionnaires being used in Kenya and other settings, previously prepared and based on guidance from WHO manuals[9-11]. The questionnaire consisted of seven sections with 83 questions capturing data from various household members for a range of health- and income-related use and expenditures. Section 1 contained 14 questions on details of the household member answering the questionnaire; and household demographics for each member of the household including gender, age, relationship to the head of household and their education, marital, occupational, relative health condition and chronic disease status, respectively.
The subsequent section containing nine questions captured data on the health-seeking behavior of each household member. Five questions were posed to ascertain in detail whether they had suffered from an episode of illness over the past four weeks, and if so, how and where they sought treatment for it. One question aimed to ascertain whether the household member had sought preventive health services in the past four weeks such as immunization or family planning. The three remaining questions in this section were about household members being hospitalized or admitted as in-patients over the past 12 months. All the subsections also sought answers on whether treatment had been foregone, and the reason for foregoing this treatment if such a situation arose.
Section 3 of the questionnaire elicited details on the utilization of out-patient and other health services by each household member in the past four weeks, if answered in the affirmative in the earlier section. A total of 25 questions were asked covering areas such as: type of illness; type of health services sought; distance of health provider visited from the home and reasons for choosing them; details of medicine or treatment given; costs of treatment; transport used and its costs; and method of payment and source of funds for payment. One additional question was about costs for routine treatment or other health-related expenses such as the purchase of vitamins.
Section 4, on the other hand, elicited details on in-patient admission of the past year for each household member, if this had been answered in the affirmative in Section 2. Similar to Section 3, 22 questions were asked on cause of illness; health provider used and reasons for making that choice; treatment regimens and costs; methods of transport and its costs; and finally, costs incurred from persons accompanying and caring for the sick household member at the hospital.
Section 5 contained six questions on access to health insurance for the members of the household; with questions seeking to determine the type and depth of health insurance coverage, type of premium payment and amounts, and whether payment was covered by an employer. Section 6 contained three questions on housing conditions, amenities and assets of the household, while Section 7 contained three questions on detailed weekly, monthly and annual household consumption expenditure.
The three experts rated the content validity of each item in the questionnaire separately without any discussion between them. The scale was scored as follows: 1= test not being relevant; 2= somewhat relevant; 3= quite relevant and; 4= highly relevant[14]. A rating of either 3 or 4 meant that the item was considered to be of relevance. For each item, the content validity index for items (I-CVI) was computed; dividing the number of experts giving either a 3 or 4 by the total number of experts[14]. The cut-off level for item acceptability incorporating the standard error of the proportion with a panel of fewer than five experts was I-CVI=1.0[15]. Thus, total agreement (the number of items that achieved the I-CVI of 1.00 divided by the total number of items to be validated in the questionnaire) was calculated to represent the proportion of questions that experts deemed quite relevant or highly relevant. The validity of the entire questionnaire was determined via the use of a content validity index for scale average (S-CVI/Ave)[14, 15]. The S-CVI/Ave, defined as the average proportion of total items judged valid by involved assessors, is the average of I-CVIs, obtained by summing them and dividing by the number of items[14, 15]. An acceptable S-CVI/Ave value according to guidelines is a minimum of 0.90[14].
On the completion of content validation, the questionnaire was translated into Malay from its original in English by a qualified member of the study team who was certified proficient in both languages. The translated Malay language instrument was then re-validated by the two Malaysian experts as detailed above who were fluent in both Malay and English. They verified the questionnaire and resolved discrepancies. Two other bilingually fluent language experts from the Department of Languages, University Putra Malaysia, then independently back-translated the questionnaire into English to be compared with the original version. These experts compared the different versions and resolved translation differences via discussion.
Following content validation and translation, the questionnaire was then subjected to a test-retest reliability study. From the selected private primary care clinic, a numbered list of patients who fit the inclusion criteria was compiled. This included: patients living in the study area and patients paying for treatment in the clinic via OOP. A total of 50 patients were then randomly selected from this list using an online random number generator software (http://stattrek.com/statistics/random-number-generator.aspx). Consenting patients were interviewed face to face by a study team member to complete a questionnaire on their monthly HHE. Study team members who conducted the interview underwent a four-hour training session on methods of conducting the interview which also included two “mock interview” sessions with the study’s principal investigator. They were also given a short manual detailing the definitions for the different variables in the questionnaire. A second interview was administered to the same patients in a period beginning 30 days after the earlier interview had been conducted. In order to ensure reliability of the answers obtained, it was ensured that the interview was conducted by a different interviewer compared to the earlier interview.
Descriptive statistics were first used to describe the sample characteristics. Data normality for continuous variables was assessed using the Kolmogorov-Smirnov test. Test-retest reliability was assessed in two different methods for the categorical and continuous items. For categorical items, the percent agreement calculation was calculated for each question; defined as the number of agreement scores divided by the total number of scores between the results of the first and second interview for that question[15, 16]. Test-retest reliability was further evaluated using Cohen’s κ coefficient, a statistic measuring inter-rater agreement for categorical items[15, 16]. Cohen’s κw was used for items with more than two possible responses[15, 16]. The range for κ coefficients was as defined by Landis and Koch: <0 (poor agreement); 0-0.2 (slight agreement); 0.21-0.40 (fair agreement); 0.41-0.60 (moderate agreement); 0.61-0.80 (substantial agreement); 0.81-1.0 (almost perfect agreement)[15, 16]. For continuous items, test-retest reliability was analyzed using a dependent t-test between the results of the first and second interview. An intra-class correlation coefficient (ICC), which measured the agreement between single measures on a linear mixed model, was then calculated along with corresponding 95 percent CI and determined to be high if above 0.7[15]. All statistical analyses were carried out using SPSS version 16.0 with significance levels fixed at p<0.05.
Results
The results of this study are described separately following its validity and reliability subsections. The validity of the questionnaire, as assessed by the three different content experts, is as described in Table I. The I-CVI for individual items’ acceptability was 1.0 for all items, which meant that all three experts were in complete agreement, proving that the individual items were of relevance. The validity of the entire questionnaire was thus also found to be high; with the calculated content validity index score or S-CVI/Ave being 1.0. This high validity meant that at the end of the validity process, no questions were dropped or added into the questionnaire as it was deemed to be valid and fit for purpose.
The sample for assessment of reliability consisted of 50 respondents sampled from the chosen private primary care clinic. The characteristics of the sample are as described in Table II. The sample largely consisted of Malays (40 percent) with 66 percent of them at least having completed secondary school. Within the sample, 84 percent were within the third income quintile with 56 percent of them being employed, monthly wage receiving staff.
Item completion for the test-retest segment of the study was satisfactory as there was an interviewer who guided the respondents through the questionnaire. For analysis, the items were divided by the type of response, i.e. whether being either categorical or continuous in nature, described in Tables III and IV, respectively. Consistency across the demographic question items was high between the two tests, with almost perfect agreement reflected by the k values (0.81-1.00) for many items. A small number of items recorded moderate agreement (k between 0.41 and 0.60) while no item recorded a fair (k=0.21-0.40), slight (k=0.21-0.40) or poor agreement (k=0-0.2).
For the 19 continuous items of the questionnaire, there were no significant differences in the comparison mean values between the test and retest segments. ICC values for 16 of the items were high (ICC>0.7) with three exceptions which were for items relating to the health insurance premium amount, key consumables expenditure for the last one month and key expenditures for the previous year.
Discussion
This study assessed and found that an HHE questionnaire containing 83 items designed for the use in urban Malaysian settings was of high validity and reliability. A valid and reliable detailed and specific questionnaire for the estimation of HHEs is crucial, especially in OOP-dominated health financed countries such as Malaysia. This is because the only other method for estimating these expenditures are via one or two questions which can be found in the National Household Expenditure Surveys; which have been found to vary widely and have proved to be inconclusive in capturing true estimates of a household’s health-related expenditure[17, 18].
As affirmed by the three consulted experts, the questionnaire exhibited excellent content validity, indicating the relevance of the various items in determining actual HHE. This further authenticated this questionnaire which had already been used in other settings and found to be of high validity[10, 19]. Concerns have arisen pertaining to the number of experts required for validating a questionnaire; with some authors suggesting that when there are a small number of evaluators, the content validity index can be inflated merely by chance factors[20, 21]. However, this shortcoming can be overcome by a high representation of agreement between the evaluators in order to encompass all possible variations of rating, i.e. the I-CVI should be 1.00, bearing in mind of course that a minimum of three evaluators are needed[20, 21]. This clause has been satisfied in this study, thus alleviating concerns of whether this validity assessment in itself was valid.
In terms of test-retest reliability, results from assessment of both the categorical and continuous items proved that this questionnaire had adequate reliability to be used in determining HHE in Malaysia. The high consistency seen across test-retest results of the categorical items reflected accuracy of the items as the answers for these items were easily selectable from the options available on the questionnaire such as demographic information, consisted of information which could be easily remembered such as the name of health provider and distance from home to provider and questioned items which incorporated habitual behavior such as preventive care seeking. In comparison, variability in agreement could be seen across items, which concerned attitudes, assessment of quality and awareness which were more dynamic in nature. This pattern of higher agreement values for habitual items and lower values for awareness/attitude items have been seen in other studies as well[22]. Test-retest results of continuous items also showed high agreement across the evaluations; with relatively lower correlation coefficients only seen in items which required distant recall such as exact monetary values for expenditure over the last month or last year. This was undoubtedly due to the presence of a recall bias[22]. However, despite this, the differences were non-significant; a further testament to the “true” reliability of the questionnaire.
Overall, caution should be observed in interpreting the results of this study due to its limitations. The sample for the test-retest reliability part of the study was not a representative sample of the urban Kuala Lumpur population as it was carried out in only one center, and like all such surveys, it is inherently subject to biases such as recall bias as outlined above. Despite these limitations, this study is an important one for the following reasons: first, this study assesses the validity and reliability of the first specific HHE questionnaire for Malaysia; finding it fit for purpose. Second, despite the exposure to certain unavoidable biases, the low variability seen in the results overall indicate that this questionnaire is strongly structured, cohesive and consistent.
Conclusion
This questionnaire will form the backbone of larger scale studies which will provide further information on health expenditure patterns of urban Malaysian households. A happy unintended consequence is that its validity and reliability will be further tested as well. Determining household expenditure patterns is part of the essential building blocks toward developing a sustainable method of health financing which will help improve the health and well-being of all peoples, especially here in Malaysia.
Content validation of household health expenditure questionnaire by content experts
| Expert agreement score | Expert agreement score | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Item no. | Question description | 1 | 2 | 3 | I-CVI | Item no. | Question description | 1 | 2 | 3 | I-CVI |
| Section 1: household demographics | Section 4: in-patient admission – last 1 year | ||||||||||
| Q1 | Names of members | 4 | 4 | 4 | 1.00 | Q49 | Number of in-patient visits | 4 | 4 | 4 | 1.00 |
| Q2 | Relationship to head | 4 | 4 | 4 | 1.00 | Q50 | Duration of admission | 4 | 4 | 4 | 1.00 |
| Q3 | Sex | 4 | 4 | 4 | 1.00 | Q51 | Name of health provider | 4 | 4 | 4 | 1.00 |
| Q4 | Religion | 4 | 4 | 4 | 1.00 | Q52 | Type of health provider | 4 | 4 | 4 | 1.00 |
| Q5 | Age | 4 | 4 | 4 | 1.00 | Q53 | Distance of provider to home | 4 | 4 | 4 | 1.00 |
| Q6 | Attendance to school | 4 | 4 | 4 | 1.00 | Q54 | Owner of health provider | 4 | 4 | 4 | 1.00 |
| Q7 | Highest level of education | 4 | 4 | 4 | 1.00 | Q55 | Reasons for not visiting nearest provider | 3 | 4 | 4 | 1.00 |
| Q8 | Years of education | 4 | 4 | 4 | 1.00 | Q56 | Reasons for visiting nearest provider | 4 | 4 | 4 | 1.00 |
| Q9 | Marital status | 4 | 4 | 4 | 1.00 | Q57 | Reasons for admission | 4 | 4 | 3 | 1.00 |
| Q10 | Main employment activity | 4 | 3 | 4 | 1.00 | Q58 | Cash payment for services | 4 | 4 | 4 | 1.00 |
| Q11 | Occupation | 4 | 4 | 4 | 1.00 | Q59 | Breakdown of payment for services | 4 | 4 | 4 | 1.00 |
| Q12 | Relative health status | 3 | 4 | 4 | 1.00 | Q60 | Mode of payment | 4 | 4 | 4 | 1.00 |
| Q13 | Smoking status | 4 | 4 | 4 | 1.00 | Q61 | Source of funds | 4 | 4 | 4 | 1.00 |
| Q14 | Chronic health condition | 4 | 4 | 4 | 1.00 | Q62 | Source of medicine/drugs | 4 | 4 | 4 | 1.00 |
| Section 2: health-seeking behavior | Q63 | Quality of care satisfaction | 4 | 4 | 4 | 1.00 | |||||
| Q15 | Illness in past 4 weeks | 3 | 4 | 4 | 1.00 | Q65 | Quality of care aspects | 4 | 4 | 4 | 1.00 |
| Q16 | Consult health provider | 4 | 4 | 4 | 1.00 | Q66 | Travel Time to facility | 4 | 4 | 4 | 1.00 |
| Q17 | Compliance to therapy | 4 | 4 | 4 | 1.00 | Q67 | Distance to facility | 4 | 4 | 4 | 1.00 |
| Q18 | Reason for non-compliance | 4 | 4 | 4 | 1.00 | Q68 | Transport costs to provider | 4 | 4 | 4 | 1.00 |
| Q19 | Reason for not seeking care | 4 | 3 | 4 | 1.00 | Q69 | Accompanied by household member | 4 | 4 | 4 | 1.00 |
| Q20 | Preventive care-seeking | 4 | 4 | 4 | 1.00 | Q70 | Identification of member | 4 | 4 | 4 | 1.00 |
| Q21 | Need to admit – past 12 months | 4 | 4 | 4 | 1.00 | Q71 | Days Accompanied | 4 | 4 | 4 | 1.00 |
| Q22 | Admission status | 4 | 4 | 4 | 1.00 | Section 5: access to health insurance | |||||
| Q23 | Reason for non-admission | 4 | 4 | 4 | 1.00 | Q72 | Household members with insurance | 4 | 4 | 4 | 1.00 |
| Section 3: out-patient and health-related services usage over the last 4 weeks | Q73 | Types of coverage | 4 | 4 | 4 | 1.00 | |||||
| Q24 | Number of out-patient visits | 4 | 4 | 4 | 1.00 | Q74 | Payers for coverage | 4 | 4 | 4 | 1.00 |
| Q25 | Type of illness | 4 | 4 | 4 | 1.00 | Q75 | Method of payment | 4 | 4 | 4 | 1.00 |
| Q26 | Type of services | 4 | 4 | 4 | 1.00 | Q76 | Premium amount | 4 | 4 | 4 | 1.00 |
| Q27 | Name of health provider | 4 | 4 | 4 | 1.00 | Q77 | Services covered | 4 | 4 | 4 | 1.00 |
| Q28 | Type of health provider | 4 | 4 | 4 | 1.00 | Section 6: housing conditions, amenities and assets | |||||
| Q29 | Distance of provider to home | 4 | 4 | 4 | 1.00 | Q78 | Type of dwelling | 4 | 4 | 4 | 1.00 |
| Q30 | Owner of health provider | 4 | 4 | 4 | 1.00 | Q79 | Owner of dwelling | 4 | 4 | 4 | 1.00 |
| Q31 | Reasons for not visiting nearest provider | 4 | 4 | 4 | 1.00 | Q80 | Items owned in household | 4 | 4 | 4 | 1.00 |
| Q32 | Reasons for visiting nearest provider | 4 | 3 | 4 | 1.00 | Section 7: household expenditure and consumption | |||||
| Q33 | Purchase of medicine/drugs | 4 | 4 | 4 | 1.00 | Q81 | Key food and beverage expenditure – last 7 days | 4 | 4 | 4 | 1.00 |
| Q34 | Reasons for not purchasing medicine/drugs | 3 | 4 | 4 | 1.00 | Q82 | Key consumables expenditure – last one month | 4 | 4 | 4 | 1.00 |
| Q35 | Reasons for purchasing medicine/drugs | 4 | 4 | 4 | 1.00 | Q83 | Key expenditures – last one year | 4 | 4 | 4 | 1.00 |
| Q36 | Cash payment for services | 4 | 4 | 4 | 1.00 | ||||||
| Q37 | Breakdown of payment for services | 4 | 4 | 4 | 1.00 | ||||||
| Q38 | Mode of payment | 4 | 4 | 4 | 1.00 | ||||||
| Q39 | Source of funds | 4 | 4 | 4 | 1.00 | ||||||
| Q40 | Waiting time pre-consultation | 4 | 4 | 4 | 1.00 | ||||||
| Q41 | Waiting time-post consultation | 4 | 4 | 4 | 1.00 | ||||||
| Q42 | Transport costs to provider | 4 | 4 | 3 | 1.00 | ||||||
| Q43 | Transport method to provider | 4 | 4 | 4 | 1.00 | ||||||
| Q44 | Quality of care satisfaction | 4 | 4 | 4 | 1.00 | ||||||
| Q45 | Quality of care aspects | 4 | 4 | 4 | 1.00 | ||||||
| Q46 | Routine health expenses | 4 | 4 | 4 | 1.00 | ||||||
| Q47 | Household persons with routine health expenses | 4 | 4 | 4 | 1.00 | ||||||
| Q48 | Cost of routine health expenses | 4 | 4 | 4 | 1.00 | ||||||
| Total agreement | 1.00 | ||||||||||
| S-CVI/Ave | 1.00 | ||||||||||
Reliability test-retest component sample characteristics
| Variables (n=50) | Mean (SD) or number (%) | Variables | Mean (SD) or number (%) |
|---|---|---|---|
| Ethnicity | Income quintile | ||
| Malay | 20 (40%) | 1 | 19 (38%) |
| Chinese | 8 (16%) | 2 | 10 (20%) |
| Indian | 10 (20%) | 3 | 13 (26%) |
| Others | 8 (16%) | 4 | 5 (10%) |
| Mixed (more than 1 ethnicity) | 4 (8%) | 5 | 3 (6%) |
| Highest educational level | Occupation of head of household | ||
| Primary | 17 (34%) | Professional/executive | 8 (16%) |
| Secondary | 13 (26%) | Sales/services/business | 10 (20%) |
| Tertiary | 20 (40%) | Waged staff | 28 (56%) |
| Number of working adults | 1.5 (0.3%) | Student | 3 (6%) |
| Number of children in household | 2.1 (0.6%) | Retired | 0 (0%) |
| Number of elderly (>65) in household | 1.9 (0.2%) | Carer | 1 (2%) |
| Number of individuals with chronic diseases | 1.4 (0.2%) | Unemployed | 0 (0%) |
| Number of individuals with disabilities | 0.1 (0.1%) | ||
| Number of individuals with private health insurance | 0.2 (0.4%) | ||
| Number of individuals with employer health coverage | 0.6 (0.3%) | ||
Test-retest reliability results for categorical items in household health expenditure questionnaire
| Item no. | Question description | Percentage agreement | κ (SEk)a | 95% CI | Item no. | Question description | Percentage agreement | κ (SEk)a | 95% CI |
|---|---|---|---|---|---|---|---|---|---|
| Section 1: household demographics | Section 4: in-patient admission – last 1 year | ||||||||
| Q1 | Names of members | 99.2 | 0.86 (0.16) | 0.81-0.99 | Q51 | Name of health provider | 99.1 | 0.93 (0.12) | 0.89-0.96 |
| Q2 | Relationship to head | 99.6 | 0.91 (0.10) | 0.85-0.97 | Q52 | Type of health provider | 98.2 | 0.96 (0.31) | 0.93-0.99 |
| Q3 | Sex | 99.2 | 0.89 (0.11) | 0.84-0.93 | Q54 | Owner of health provider | 97.8 | 0.87 (0.15) | 0.82-0.92 |
| Q4 | Religion | 94.6 | 0.82 (0.11) | 0.77-0.84 | Q55 | Reasons for not visiting nearest provider | 84.3 | 0.57 (0.09) | 0.53-0.60 |
| Q5 | Age | 99.4 | 0.81 (0.30) | 0.77-0.84 | Q56 | Reasons for visiting nearest provider | 81.5 | 0.61 (0.17) | 0.57-0.65 |
| Q6 | Attendance to school | 98.3 | 0.76 (0.10) | 0.72-0.80 | Q57 | Reasons for admission | 90.6 | 0.67 (0.12) | 0.63-0.70 |
| Q7 | Highest level of education | 98.7 | 0.69 (0.26) | 0.65-0.71 | Q58 | Cash payment for services | 93.7 | 0.58 (0.10) | 0.55-0.62 |
| Q8 | Years of education | 93.9 | 0.60 (0.15) | 0.56-0.63 | Q60 | Mode of payment | 94.5 | 0.63 (0.09) | 0.60-0.66 |
| Q9 | Marital status | 99.6 | 0.87 (0.06) | 0.82-0.92 | Q61 | Source of funds | 96.2 | 0.74 (0.31) | 0.71-0.77 |
| Q10 | Main employment activity | 96.2 | 0.78 (0.24) | 0.73-0.82 | Q62 | Source of medicine/drugs | 97.4 | 0.86 (0.15) | 0.83-0.89 |
| Q11 | Occupation | 94.5 | 0.83 (0.12) | 0.80-0.85 | Q63 | Quality of care satisfaction | 89.1 | 0.79 (0.11) | 0.75-0.82 |
| Q12 | Relative health status | 93.6 | 0.85 (0.09) | 0.81-0.89 | Q65 | Quality of care aspects | 85.2 | 0.58 (0.22) | 0.54-0.61 |
| Q13 | Smoking status | 98.9 | 0.92 (0.13) | 0.98-0.98 | Q69 | Accompanied by household member | 97.2 | 0.64 (0.23) | 0.60-0.68 |
| Q14 | Chronic health condition | 97.3 | 0.65 (0.32) | 0.61-0.69 | Q70 | Identification of member | 94.3 | 0.82 (0.17) | 0.78-0.86 |
| Section 2: health-seeking behavior | Section 5: access to health insurance | ||||||||
| Q16 | Consult health provider | 88.6 | 0.78 (0.09) | 0.74-0.83 | Q72 | Household members with insurance | 90.7 | 0.77 (0.18) | 0.73-0.81 |
| Q17 | Compliance to therapy | 77.7 | 0.44 (0.14) | 0.42-0.46 | Q73 | Types of coverage | 91.6 | 0.91 (0.08) | 0.86-0.95 |
| Q18 | Reason for non-compliance | 82.6 | 0.48 (0.13) | 0.43-0.52 | Q74 | Payers for coverage | 94.3 | 0.84 (0.15) | 0.78-0.90 |
| Q19 | Reason for not seeking care | 89.4 | 0.76 (0.27) | 0.73-0.80 | Q75 | Method of payment | 99.8 | 0.92 (0.10) | 0.88-0.96 |
| Q20 | Preventive care-seeking | 92.1 | 0.67 (0.19) | 0.62-0.71 | Q77 | Services covered | 79.4 | 0.56 (0.22) | 0.51-0.60 |
| Q21 | Need to admit – past 12 months | 86.4 | 0.91 (0.16) | 0.85-0.94 | Section 6: housing conditions, amenities and assets | ||||
| Q22 | Admission status | 95.3 | 0.87 (0.08) | 0.82-0.93 | Q78 | Type of dwelling | 99.4 | 0.82 (0.16) | 0.79-0.85 |
| Q23 | Reason for non-admission | 84.2 | 0.71 (0.12) | 0.67-0.73 | Q79 | Owner of dwelling | 99.2 | 0.87 (0.12) | 0.83-0.92 |
| Section 3: out-patient and health-related services usage over the last 4 weeks | Q80 | Items owned in household | 82.9 | 0.73 (0.27) | 0.68-0.77 | ||||
| Q25 | Type of illness | 87.3 | 0.65 (0.19) | 0.62-0.68 | |||||
| Q26 | Type of services | 88.4 | 0.69 (0.24) | 0.66-0.72 | |||||
| Q27 | Name of health provider | 99.3 | 0.86 (0.17) | 0.83-0.89 | |||||
| Q28 | Type of health provider | 95.6 | 0.79 (0.13) | 0.75-0.82 | |||||
| Q30 | Owner of health provider | 92.7 | 0.80 (0.10) | 0.76-0.84 | |||||
| Q31 | Reasons for not visiting nearest provider | 78.4 | 0.53 (0.26) | 0.50-0.56 | |||||
| Q32 | Reasons for visiting nearest provider | 94.1 | 0.85 (0.07) | 0.82-0.88 | |||||
| Q33 | Purchase of medicine/drugs | 75.2 | 0.44 (0.17) | 0.42-0.47 | |||||
| Q34 | Reasons for not purchasing medicine/drugs | 89.1 | 0.87 (0.10) | 0.83-0.91 | |||||
| Q35 | Reasons for purchasing medicine/drugs | 90.5 | 0.76 (0.15) | 0.72-0.79 | |||||
| Q38 | Mode of payment | 97.1 | 0.83 (0.13) | 0.79-0.87 | |||||
| Q39 | Source of funds | 96.4 | 0.91 (0.09) | 0.88-0.94 | |||||
| Q40 | Waiting time pre-consultation | 82.7 | 0.69 (0.26) | 0.66-0.72 | |||||
| Q41 | Waiting time-post consultation | 89.5 | 0.81 (0.29) | 0.78-0.83 | |||||
| Q43 | Transport method to provider | 93.2 | 0.76 (0.35) | 0.72-0.79 | |||||
| Q44 | Quality of care satisfaction | 94.6 | 0.81 (0.13) | 0.77-0.85 | |||||
| Q45 | Quality of care aspects | 91.8 | 0.77 (0.39) | 0.72-0.81 | |||||
| Q46 | Routine health expenses | 92.7 | 0.66 (0.18) | 0.63-0.69 | |||||
| Q47 | Household persons with routine health expenses | 98.9 | 0.89 (0.24) | 0.83-0.95 | |||||
Note: aFor categorical items with more than two outcomes, a weighted κ coefficient was calculated
Test-retest reliability results for continuous items in household health expenditure questionnaire
| Item no. | Question description | Test mean (SD) | Retest mean (SD) | p-value | ICC | 95% CI |
|---|---|---|---|---|---|---|
| Section 2: health-seeking behavior | ||||||
| Q15 | Illness in past 4 weeks | 2.8 | 2.6 | 0.26 | 0.91 | 0.88-0.94 |
| Section 3: out-patient and health-related services usage over the last 4 weeks | ||||||
| Q24 | Number of out-patient visits | 1.6 | 1.5 | 0.54 | 0.87 | 0.81-0.94 |
| Q29 | Distance of provider to home (in km) | 0.14 | 0.13 | 0.27 | 0.77 | 0.71-0.82 |
| Q36 | Cash payment for services (in RM) | 89.90 | 84.10 | 0.10 | 0.86 | 0.82-0.91 |
| Q37 | Breakdown of payment for services (in %) | 44.6 | 45.0 | 0.26 | 0.83 | 0.77-0.88 |
| Q42 | Transport costs to provider (in RM) | 11.10 | 10.90 | 0.10 | 0.91 | 0.86-0.95 |
| Q48 | Cost of routine health expenses (in RM) | 157.00 | 152.60 | 0.09 | 0.74 | 0.68-0.79 |
| Section 4: in-patient admission-last 1 year | ||||||
| Q49 | Number of in-patient visits | 0.6 | 0.5 | 0.33 | 0.92 | 0.90-0.96 |
| Q50 | Duration of admission (in days) | 3.8 | 4.2 | 0.28 | 0.90 | 0.85-0.95 |
| Q53 | Distance of provider to home (in km) | 0.24 | 0.29 | 0.15 | 0.91 | 0.87-0.96 |
| Q59 | Breakdown of payment for services (in %) | 67.2 | 65.8 | 0.17 | 0.85 | 0.81-0.89 |
| Q66 | Travel time to facility (in hours) | 1.3 | 1.0 | 0.32 | 0.96 | 0.91-0.99 |
| Q67 | Distance to facility (in km) | 11.3 | 10.9 | 0.13 | 0.84 | 0.79-0.88 |
| Q68 | Transport costs to provider | 102.10 | 101.40 | 0.11 | 0.83 | 0.81-0.86 |
| Q71 | Days accompanied | 3.1 | 2.9 | 0.46 | 0.74 | 0.69-0.77 |
| Section 5: access to health insurance | ||||||
| Q76 | Premium amount | 1,892.00 | 1,993.50 | 0.08 | 0.63 | 0.55-0.68 |
| Section 7: household expenditure and consumption | ||||||
| Q81 | Key food and beverage expenditure – last 7 days | 559.30 | 587.40 | 0.11 | 0.72 | 0.67-0.76 |
| Q82 | Key consumables expenditure – last one month | 1,810.40 | 1,796.70 | 0.09 | 0.63 | 0.59-0.70 |
| Q83 | Key expenditures – last one year | 16,011.20 | 16,175.00 | 0.07 | 0.61 | 0.58-0.65 |
References
1.Tangcharoensathien V, Mills A, Palu T. Accelerating health equity: the key role of universal health coverage in the sustainable development goals. BMC Med. 2015; 13(101): 1-5.
2.Horton R, Das P. Universal health coverage: not why, what, or when – but how? Lancet. 2015; 385(9974): 1156-57. doi: 10.1016/s0140-6736(14)61742-6
3.World Health Organization [WHO]. The World Health Report 2000: health systems: improving performance. Geneva: World Health Organization; 2009 [cited 2016 Feb 20]. Available from: www.who.int/whr/1999/en/whr99_en.pdf?ua=1
4.World Health Organization [WHO]. The World Health Report 2010 – health systems financing: the path to universal coverage. Geneva: World Health Organization; 2010 [cited 2016 Mar 2]. Available from: www.who.int/whr/2010/whr10_en.pdf?ua=1
5.Safurah JK, Khairiyah AM, Nour HO, Healy J. Malaysia health system review: health systems in transition. Asia Pacific Observatory on Health Systems and Policies. Report No. 3, Geneva: World Health Organization; 2013 [cited 2016 Feb 17]. Available from: www.wpro.who.int/asia_pacific_observatory/hits/series/Malaysia_Health_Systems_Review2013.pdf
6.Carrin G, Evans D, Xu K. Designing health financing policy towards universal coverage. Bull World Health Organ. 2007; 85(9): 652.
7.Murallitharan M. Effectiveness of a health microinsurance scheme for private primary care in Malaysia (HMI); 2016 [cited 2016 Apr 27]. Available from: https://clinicaltrials.gov/ct2/show/NCT02696174
8.Murallitharan M. Effectiveness of health microinsurance scheme in private primary care in Malaysia; 2016 [cited 2016 Apr 27]. Available from: www.isrctn.com/ISRCTN10261528
9.Ministry of Medical Services, Ministry of Public Health and Sanitation. Kenya National Health Accounts 2009/10. Health System 20/20. Nairobi: Ministry of Medical Services; 2011.
10.Chuma J, Maina T. Catastrophic health care spending and impoverishment in Kenya. BMC Health Serv Res. 2012; 12(413): 1-9.
11.Xu HR, Saksena P, Evans D. Validity and comparability of out-of-pocket health expenditure from household surveys: a review of the literature and current survey instruments. Discussion Paper No. 1, Geneva: WHO; 2011 [cited 2016 Mar 22]. Available from: www.who.int/health_financing/documents/dp_e_11_01-oop_errors.pdf
12.Ministry of Health Malaysia. List of registered private clinics up to 31 Dec 2015 (Medical). Private Medical Practice Division. Kuala Lumpur Ministry of Health Malaysia (MOH); 2016 [cited 2016 Feb 2]. Available from: http://medicalprac.moh.gov.my/v2/uploads/04%20Klinik%20PERUBATAN%20SWASTA%2031%20DIS%202015.pdf (in Malay).
13.Department of Statistics Malaysia. Population and housing census of Malaysia 2010. Kuala Lumpur Department of Statistics Malaysia; 2011.
14.Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health. 2006; 29(5): 489-97. doi: 10.1002/nur.20147
15.Chiwaridzo M, Chikasha TN, Naidoo N, Dambi JM, Tadyanemhandu C, Munambah N, et al.. Content validity and test-retest reliability of a low back pain questionnaire in Zimbabwean adolescents. Arch Physiother. 2017; 7(1): 3. doi: 10.1186/s40945-017-0031-y
16.McNeely J, Strauss SM, Wright S, Rotrosen J, Khan R, Lee JD, et al. Test-retest reliability of a self-administered alcohol, smoking and substance involvement screening test (ASSIST) in primary care patients. J Subst Abuse Treat. 2014; 47(1): 93-101. doi: 10.1016/j.jsat.2014.01.007
17.Xu K, Ravndal F, Evans DB, Carrin G. Assessing the reliability of household expenditure data: results of the World Health Survey. Health Policy. 2009; 91(3): 297-305. doi: 10.1016/j.healthpol.2009.01.002
18.Lavado RF, Brooks BP, Hanlon M. Estimating health expenditure shares from household surveys. Bull World Health Organ. 2013; 91(7): 519-24C. doi: 10.2471/BLT.12.115535
19.Buigut S, Ettarh R, Amendah DD. Catastrophic health expenditure and its determinants in Kenya slum communities. Int J Equity Health. 2015; 14(46): 1-12. doi: 10.1186/s12939-015-0168-9
20.Gilbert GE, Prion S. Making sense of methods and measurement: Lawshe’s content validity index. Clin Simul Nur. 2016; 12(12): 530-31.
21.Rutherford-Hemming T. Determining content validity and reporting a content validity index for simulation scenarios. Nurs Educ Perspect. 2015; 36(6): 389-93.
22.Kim SJ, Han JA, Kim YH, Choi BY, Kim SY, Lee HJ, et al. Test-retest reliability of health behavior items in the Community Health Survey in South Korea. Epidemiol Health. 2015; 37: e2015045. doi: 10.4178/epih/e2015045