Assessment of the validity and reliability of an urban household health expenditure (HHE) questionnaire in Kuala Lumpur, Malaysia

Murallitharan Munisamy (College of Public Health Sciences, Chulalongkorn University, Bangkok, Thailand)
Tharini Thanapalan (Klinik Kesihatan Ibu dan Anak Cheras Makmur, Ministry of Health, Kuala Lumpur, Malaysia)
Pattaraporn Piwong (College of Public Health Sciences, Chulalongkorn University, Bangkok, Thailand)
Alessio Panza (College of Public Health Sciences, Chulalongkorn University, Bangkok, Thailand)
Sathirakorn Pongpanich (College of Public Health Sciences, Chulalongkorn University, Bangkok, Thailand)

Journal of Health Research

ISSN: 2586-940X

Article publication date: 20 December 2017

Issue publication date: 18 April 2018

Abstract

Purpose

Out-of-pocket (OOP) payments continue to be a major method of financing healthcare in many low- and middle-income countries including Malaysia. Although macro-level data show that this is a substantial percentage of national health expenditure, at the grassroots level, the amount spent on health by households remains unknown in Malaysia. The purpose of this paper is to assess the validity and reliability of an adapted-for-purpose questionnaire designed to capture urban household health expenditures (HHEs) among Malaysian households.

Design/methodology/approach

This two-part study assessed content validity of the questionnaire using three experts and the reliability of the questionnaire through a test-retest study among 50 OOP-paying patients followed up at one private primary care clinic in Kuala Lumpur. This study was approved by the Malaysian Research Ethics Committee (NMRR-16-172-29311-IIR).

Findings

The validity of the 83-item questionnaire was high, with an item content validity index of 1.00 and a scale content validity index average score of 1.0 agreed to among the evaluating experts. In the test-retest reliability study, the majority of the categorical questionnaire items had perfect agreement values (k=0.81-1.00). Continuous questionnaire items were also found to be highly reliable with no significant differences between the test-retest segments and high correlation coefficient values (intra-class correlation coefficient>0.7).

Originality/value

The HHE questionnaire had excellent content validity and very high test-retest reliability. The results of this study suggest that this questionnaire could be used in Malaysian studies to determine actual urban HHE which is a first step toward developing universal health coverage for all.

Keywords

Citation

Munisamy, M., Thanapalan, T., Piwong, P., Panza, A. and Pongpanich, S. (2018), "Assessment of the validity and reliability of an urban household health expenditure (HHE) questionnaire in Kuala Lumpur, Malaysia", Journal of Health Research, Vol. 32 No. 1, pp. 24-35. https://doi.org/10.1108/JHR-11-2017-003

Publisher

:

Emerald Publishing Limited

Copyright © 2018, Murallitharan Munisamy, Tharini Thanapalan, Pattaraporn Piwong, Alessio Panza and Sathirakorn Pongpanich

License

Published in the Journal of Health Reseach. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article ( for both commercial & non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode


Introduction

Universal health coverage (UHC) has been made one of the global health priorities, as signified by its inclusion as Target 3.8 on the global Sustainable Development Goals[1]. UHC is based on the Primary Health Care principles from the 1978 Alma-Ata conference, whereby all peoples should be provided with equitable access to quality healthcare based on need, without the user facing financial hardships in the process of seeking and receiving care[2]. This has been a struggle for the low- and middle-income countries where low government expenditure in healthcare alongside low-quality provision of health services has led to citizens largely resorting to using expensive, private services and paying for them via out-of-pocket (OOP) payments[2].

In low- and even lower middle-income households, OOP payments for healthcare cause a great strain on families’ monthly household expenditures[3]. This has been significantly proven to be associated with financial difficulties. When sudden, calamitous health events such as a stroke or a heart attack occur, they may push an individual or family into huge debt and even render them bankrupt, a term called catastrophic health expenditure[4]. Incurring large monthly household health expenditures (HHEs) also often causes households to delay seeking healthcare, leading to further health complications and even poor health outcomes as seen in chronic disease patients[3, 4].

Malaysia is a middle-income country with largely subsidized public healthcare services[5]. Despite the availability of access for Malaysians to healthcare, perceived quality and logistics issues have led to widespread use of private services, especially in private primary care facilities[5]. As Malaysia has been deficient in developing pre-payment mechanisms for health, payments for these private health services are largely via OOP payments[6]. On a macro level, the evidence is clear that private payments in terms of total healthcare expenditure are high and this largely comes from OOP payments[5]. On a micro level however, it is unclear on how much the average Malaysian household spends on healthcare every month and how deeply this healthcare expenditures further impact their overall monthly expenditure. This pilot study aimed to assess the validity and reliability of an adapted-for-purpose questionnaire designed to capture urban HHEs among Malaysian households as part of a larger series of studies to develop a health microinsurance scheme in Malaysia[7, 8].

Methods

This study was conducted in two parts. The first part aimed to establish the content validity of the developed questionnaire via a sample of experts. The second part of the study was conducted to evaluate the reliability of the content-validated questionnaire via a test-retest design. Although the questionnaire had had been used and verified in other settings with a high-validity quotient[9-11], the content validity of the adapted version was nevertheless assessed by three experts: a health economist from Chulalongkorn University, Thailand, and two private primary care physicians who worked in private primary care clinics in Kuala Lumpur.

The setting for the test-retest portion of the study was located among private primary care clinics in Jalan Ipoh, a socio-demographically diverse suburb in Kuala Lumpur, with a population of approximately 268,000 people (2010 census)[12]. One private primary care clinic was purposively selected from among the list of private primary care clinics registered at the Ministry of Health, Malaysia, as the base location for conducting this portion of the study[13]. This study received ethical approval from the Malaysian Research Ethics Committee, Ministry of Health, Malaysia (NMRR-16-172-29311-IIR).

The HHE questionnaire used in this study was adapted-for-use in Malaysia from existing questionnaires being used in Kenya and other settings, previously prepared and based on guidance from WHO manuals[9-11]. The questionnaire consisted of seven sections with 83 questions capturing data from various household members for a range of health- and income-related use and expenditures. Section 1 contained 14 questions on details of the household member answering the questionnaire; and household demographics for each member of the household including gender, age, relationship to the head of household and their education, marital, occupational, relative health condition and chronic disease status, respectively.

The subsequent section containing nine questions captured data on the health-seeking behavior of each household member. Five questions were posed to ascertain in detail whether they had suffered from an episode of illness over the past four weeks, and if so, how and where they sought treatment for it. One question aimed to ascertain whether the household member had sought preventive health services in the past four weeks such as immunization or family planning. The three remaining questions in this section were about household members being hospitalized or admitted as in-patients over the past 12 months. All the subsections also sought answers on whether treatment had been foregone, and the reason for foregoing this treatment if such a situation arose.

Section 3 of the questionnaire elicited details on the utilization of out-patient and other health services by each household member in the past four weeks, if answered in the affirmative in the earlier section. A total of 25 questions were asked covering areas such as: type of illness; type of health services sought; distance of health provider visited from the home and reasons for choosing them; details of medicine or treatment given; costs of treatment; transport used and its costs; and method of payment and source of funds for payment. One additional question was about costs for routine treatment or other health-related expenses such as the purchase of vitamins.

Section 4, on the other hand, elicited details on in-patient admission of the past year for each household member, if this had been answered in the affirmative in Section 2. Similar to Section 3, 22 questions were asked on cause of illness; health provider used and reasons for making that choice; treatment regimens and costs; methods of transport and its costs; and finally, costs incurred from persons accompanying and caring for the sick household member at the hospital.

Section 5 contained six questions on access to health insurance for the members of the household; with questions seeking to determine the type and depth of health insurance coverage, type of premium payment and amounts, and whether payment was covered by an employer. Section 6 contained three questions on housing conditions, amenities and assets of the household, while Section 7 contained three questions on detailed weekly, monthly and annual household consumption expenditure.

The three experts rated the content validity of each item in the questionnaire separately without any discussion between them. The scale was scored as follows: 1= test not being relevant; 2= somewhat relevant; 3= quite relevant and; 4= highly relevant[14]. A rating of either 3 or 4 meant that the item was considered to be of relevance. For each item, the content validity index for items (I-CVI) was computed; dividing the number of experts giving either a 3 or 4 by the total number of experts[14]. The cut-off level for item acceptability incorporating the standard error of the proportion with a panel of fewer than five experts was I-CVI=1.0[15]. Thus, total agreement (the number of items that achieved the I-CVI of 1.00 divided by the total number of items to be validated in the questionnaire) was calculated to represent the proportion of questions that experts deemed quite relevant or highly relevant. The validity of the entire questionnaire was determined via the use of a content validity index for scale average (S-CVI/Ave)[14, 15]. The S-CVI/Ave, defined as the average proportion of total items judged valid by involved assessors, is the average of I-CVIs, obtained by summing them and dividing by the number of items[14, 15]. An acceptable S-CVI/Ave value according to guidelines is a minimum of 0.90[14].

On the completion of content validation, the questionnaire was translated into Malay from its original in English by a qualified member of the study team who was certified proficient in both languages. The translated Malay language instrument was then re-validated by the two Malaysian experts as detailed above who were fluent in both Malay and English. They verified the questionnaire and resolved discrepancies. Two other bilingually fluent language experts from the Department of Languages, University Putra Malaysia, then independently back-translated the questionnaire into English to be compared with the original version. These experts compared the different versions and resolved translation differences via discussion.

Following content validation and translation, the questionnaire was then subjected to a test-retest reliability study. From the selected private primary care clinic, a numbered list of patients who fit the inclusion criteria was compiled. This included: patients living in the study area and patients paying for treatment in the clinic via OOP. A total of 50 patients were then randomly selected from this list using an online random number generator software (http://stattrek.com/statistics/random-number-generator.aspx). Consenting patients were interviewed face to face by a study team member to complete a questionnaire on their monthly HHE. Study team members who conducted the interview underwent a four-hour training session on methods of conducting the interview which also included two “mock interview” sessions with the study’s principal investigator. They were also given a short manual detailing the definitions for the different variables in the questionnaire. A second interview was administered to the same patients in a period beginning 30 days after the earlier interview had been conducted. In order to ensure reliability of the answers obtained, it was ensured that the interview was conducted by a different interviewer compared to the earlier interview.

Descriptive statistics were first used to describe the sample characteristics. Data normality for continuous variables was assessed using the Kolmogorov-Smirnov test. Test-retest reliability was assessed in two different methods for the categorical and continuous items. For categorical items, the percent agreement calculation was calculated for each question; defined as the number of agreement scores divided by the total number of scores between the results of the first and second interview for that question[15, 16]. Test-retest reliability was further evaluated using Cohen’s κ coefficient, a statistic measuring inter-rater agreement for categorical items[15, 16]. Cohen’s κw was used for items with more than two possible responses[15, 16]. The range for κ coefficients was as defined by Landis and Koch: <0 (poor agreement); 0-0.2 (slight agreement); 0.21-0.40 (fair agreement); 0.41-0.60 (moderate agreement); 0.61-0.80 (substantial agreement); 0.81-1.0 (almost perfect agreement)[15, 16]. For continuous items, test-retest reliability was analyzed using a dependent t-test between the results of the first and second interview. An intra-class correlation coefficient (ICC), which measured the agreement between single measures on a linear mixed model, was then calculated along with corresponding 95 percent CI and determined to be high if above 0.7[15]. All statistical analyses were carried out using SPSS version 16.0 with significance levels fixed at p<0.05.

Results

The results of this study are described separately following its validity and reliability subsections. The validity of the questionnaire, as assessed by the three different content experts, is as described in Table I. The I-CVI for individual items’ acceptability was 1.0 for all items, which meant that all three experts were in complete agreement, proving that the individual items were of relevance. The validity of the entire questionnaire was thus also found to be high; with the calculated content validity index score or S-CVI/Ave being 1.0. This high validity meant that at the end of the validity process, no questions were dropped or added into the questionnaire as it was deemed to be valid and fit for purpose.

The sample for assessment of reliability consisted of 50 respondents sampled from the chosen private primary care clinic. The characteristics of the sample are as described in Table II. The sample largely consisted of Malays (40 percent) with 66 percent of them at least having completed secondary school. Within the sample, 84 percent were within the third income quintile with 56 percent of them being employed, monthly wage receiving staff.

Item completion for the test-retest segment of the study was satisfactory as there was an interviewer who guided the respondents through the questionnaire. For analysis, the items were divided by the type of response, i.e. whether being either categorical or continuous in nature, described in Tables III and IV, respectively. Consistency across the demographic question items was high between the two tests, with almost perfect agreement reflected by the k values (0.81-1.00) for many items. A small number of items recorded moderate agreement (k between 0.41 and 0.60) while no item recorded a fair (k=0.21-0.40), slight (k=0.21-0.40) or poor agreement (k=0-0.2).

For the 19 continuous items of the questionnaire, there were no significant differences in the comparison mean values between the test and retest segments. ICC values for 16 of the items were high (ICC>0.7) with three exceptions which were for items relating to the health insurance premium amount, key consumables expenditure for the last one month and key expenditures for the previous year.

Discussion

This study assessed and found that an HHE questionnaire containing 83 items designed for the use in urban Malaysian settings was of high validity and reliability. A valid and reliable detailed and specific questionnaire for the estimation of HHEs is crucial, especially in OOP-dominated health financed countries such as Malaysia. This is because the only other method for estimating these expenditures are via one or two questions which can be found in the National Household Expenditure Surveys; which have been found to vary widely and have proved to be inconclusive in capturing true estimates of a household’s health-related expenditure[17, 18].

As affirmed by the three consulted experts, the questionnaire exhibited excellent content validity, indicating the relevance of the various items in determining actual HHE. This further authenticated this questionnaire which had already been used in other settings and found to be of high validity[10, 19]. Concerns have arisen pertaining to the number of experts required for validating a questionnaire; with some authors suggesting that when there are a small number of evaluators, the content validity index can be inflated merely by chance factors[20, 21]. However, this shortcoming can be overcome by a high representation of agreement between the evaluators in order to encompass all possible variations of rating, i.e. the I-CVI should be 1.00, bearing in mind of course that a minimum of three evaluators are needed[20, 21]. This clause has been satisfied in this study, thus alleviating concerns of whether this validity assessment in itself was valid.

In terms of test-retest reliability, results from assessment of both the categorical and continuous items proved that this questionnaire had adequate reliability to be used in determining HHE in Malaysia. The high consistency seen across test-retest results of the categorical items reflected accuracy of the items as the answers for these items were easily selectable from the options available on the questionnaire such as demographic information, consisted of information which could be easily remembered such as the name of health provider and distance from home to provider and questioned items which incorporated habitual behavior such as preventive care seeking. In comparison, variability in agreement could be seen across items, which concerned attitudes, assessment of quality and awareness which were more dynamic in nature. This pattern of higher agreement values for habitual items and lower values for awareness/attitude items have been seen in other studies as well[22]. Test-retest results of continuous items also showed high agreement across the evaluations; with relatively lower correlation coefficients only seen in items which required distant recall such as exact monetary values for expenditure over the last month or last year. This was undoubtedly due to the presence of a recall bias[22]. However, despite this, the differences were non-significant; a further testament to the “true” reliability of the questionnaire.

Overall, caution should be observed in interpreting the results of this study due to its limitations. The sample for the test-retest reliability part of the study was not a representative sample of the urban Kuala Lumpur population as it was carried out in only one center, and like all such surveys, it is inherently subject to biases such as recall bias as outlined above. Despite these limitations, this study is an important one for the following reasons: first, this study assesses the validity and reliability of the first specific HHE questionnaire for Malaysia; finding it fit for purpose. Second, despite the exposure to certain unavoidable biases, the low variability seen in the results overall indicate that this questionnaire is strongly structured, cohesive and consistent.

Conclusion

This questionnaire will form the backbone of larger scale studies which will provide further information on health expenditure patterns of urban Malaysian households. A happy unintended consequence is that its validity and reliability will be further tested as well. Determining household expenditure patterns is part of the essential building blocks toward developing a sustainable method of health financing which will help improve the health and well-being of all peoples, especially here in Malaysia.

Content validation of household health expenditure questionnaire by content experts

Expert agreement score Expert agreement score
Item no. Question description 1 2 3 I-CVI Item no. Question description 1 2 3 I-CVI
Section 1: household demographics Section 4: in-patient admission – last 1 year
Q1 Names of members 4 4 4 1.00 Q49 Number of in-patient visits 4 4 4 1.00
Q2 Relationship to head 4 4 4 1.00 Q50 Duration of admission 4 4 4 1.00
Q3 Sex 4 4 4 1.00 Q51 Name of health provider 4 4 4 1.00
Q4 Religion 4 4 4 1.00 Q52 Type of health provider 4 4 4 1.00
Q5 Age 4 4 4 1.00 Q53 Distance of provider to home 4 4 4 1.00
Q6 Attendance to school 4 4 4 1.00 Q54 Owner of health provider 4 4 4 1.00
Q7 Highest level of education 4 4 4 1.00 Q55 Reasons for not visiting nearest provider 3 4 4 1.00
Q8 Years of education 4 4 4 1.00 Q56 Reasons for visiting nearest provider 4 4 4 1.00
Q9 Marital status 4 4 4 1.00 Q57 Reasons for admission 4 4 3 1.00
Q10 Main employment activity 4 3 4 1.00 Q58 Cash payment for services 4 4 4 1.00
Q11 Occupation 4 4 4 1.00 Q59 Breakdown of payment for services 4 4 4 1.00
Q12 Relative health status 3 4 4 1.00 Q60 Mode of payment 4 4 4 1.00
Q13 Smoking status 4 4 4 1.00 Q61 Source of funds 4 4 4 1.00
Q14 Chronic health condition 4 4 4 1.00 Q62 Source of medicine/drugs 4 4 4 1.00
Section 2: health-seeking behavior Q63 Quality of care satisfaction 4 4 4 1.00
Q15 Illness in past 4 weeks 3 4 4 1.00 Q65 Quality of care aspects 4 4 4 1.00
Q16 Consult health provider 4 4 4 1.00 Q66 Travel Time to facility 4 4 4 1.00
Q17 Compliance to therapy 4 4 4 1.00 Q67 Distance to facility 4 4 4 1.00
Q18 Reason for non-compliance 4 4 4 1.00 Q68 Transport costs to provider 4 4 4 1.00
Q19 Reason for not seeking care 4 3 4 1.00 Q69 Accompanied by household member 4 4 4 1.00
Q20 Preventive care-seeking 4 4 4 1.00 Q70 Identification of member 4 4 4 1.00
Q21 Need to admit – past 12 months 4 4 4 1.00 Q71 Days Accompanied 4 4 4 1.00
Q22 Admission status 4 4 4 1.00 Section 5: access to health insurance
Q23 Reason for non-admission 4 4 4 1.00 Q72 Household members with insurance 4 4 4 1.00
Section 3: out-patient and health-related services usage over the last 4 weeks Q73 Types of coverage 4 4 4 1.00
Q24 Number of out-patient visits 4 4 4 1.00 Q74 Payers for coverage 4 4 4 1.00
Q25 Type of illness 4 4 4 1.00 Q75 Method of payment 4 4 4 1.00
Q26 Type of services 4 4 4 1.00 Q76 Premium amount 4 4 4 1.00
Q27 Name of health provider 4 4 4 1.00 Q77 Services covered 4 4 4 1.00
Q28 Type of health provider 4 4 4 1.00 Section 6: housing conditions, amenities and assets
Q29 Distance of provider to home 4 4 4 1.00 Q78 Type of dwelling 4 4 4 1.00
Q30 Owner of health provider 4 4 4 1.00 Q79 Owner of dwelling 4 4 4 1.00
Q31 Reasons for not visiting nearest provider 4 4 4 1.00 Q80 Items owned in household 4 4 4 1.00
Q32 Reasons for visiting nearest provider 4 3 4 1.00 Section 7: household expenditure and consumption
Q33 Purchase of medicine/drugs 4 4 4 1.00 Q81 Key food and beverage expenditure – last 7 days 4 4 4 1.00
Q34 Reasons for not purchasing medicine/drugs 3 4 4 1.00 Q82 Key consumables expenditure – last one month 4 4 4 1.00
Q35 Reasons for purchasing medicine/drugs 4 4 4 1.00 Q83 Key expenditures – last one year 4 4 4 1.00
Q36 Cash payment for services 4 4 4 1.00
Q37 Breakdown of payment for services 4 4 4 1.00
Q38 Mode of payment 4 4 4 1.00
Q39 Source of funds 4 4 4 1.00
Q40 Waiting time pre-consultation 4 4 4 1.00
Q41 Waiting time-post consultation 4 4 4 1.00
Q42 Transport costs to provider 4 4 3 1.00
Q43 Transport method to provider 4 4 4 1.00
Q44 Quality of care satisfaction 4 4 4 1.00
Q45 Quality of care aspects 4 4 4 1.00
Q46 Routine health expenses 4 4 4 1.00
Q47 Household persons with routine health expenses 4 4 4 1.00
Q48 Cost of routine health expenses 4 4 4 1.00
Total agreement 1.00
S-CVI/Ave 1.00

Reliability test-retest component sample characteristics

Variables (n=50) Mean (SD) or number (%) Variables Mean (SD) or number (%)
Ethnicity Income quintile
Malay 20 (40%) 1 19 (38%)
Chinese 8 (16%) 2 10 (20%)
Indian 10 (20%) 3 13 (26%)
Others 8 (16%) 4 5 (10%)
Mixed (more than 1 ethnicity) 4 (8%) 5 3 (6%)
Highest educational level Occupation of head of household
Primary 17 (34%) Professional/executive 8 (16%)
Secondary 13 (26%) Sales/services/business 10 (20%)
Tertiary 20 (40%) Waged staff 28 (56%)
Number of working adults 1.5 (0.3%) Student 3 (6%)
Number of children in household 2.1 (0.6%) Retired 0 (0%)
Number of elderly (>65) in household 1.9 (0.2%) Carer 1 (2%)
Number of individuals with chronic diseases 1.4 (0.2%) Unemployed 0 (0%)
Number of individuals with disabilities 0.1 (0.1%)
Number of individuals with private health insurance 0.2 (0.4%)
Number of individuals with employer health coverage 0.6 (0.3%)

Test-retest reliability results for categorical items in household health expenditure questionnaire

Item no. Question description Percentage agreement κ (SEk)a 95% CI Item no. Question description Percentage agreement κ (SEk)a 95% CI
Section 1: household demographics Section 4: in-patient admission – last 1 year
Q1 Names of members 99.2 0.86 (0.16) 0.81-0.99 Q51 Name of health provider 99.1 0.93 (0.12) 0.89-0.96
Q2 Relationship to head 99.6 0.91 (0.10) 0.85-0.97 Q52 Type of health provider 98.2 0.96 (0.31) 0.93-0.99
Q3 Sex 99.2 0.89 (0.11) 0.84-0.93 Q54 Owner of health provider 97.8 0.87 (0.15) 0.82-0.92
Q4 Religion 94.6 0.82 (0.11) 0.77-0.84 Q55 Reasons for not visiting nearest provider 84.3 0.57 (0.09) 0.53-0.60
Q5 Age 99.4 0.81 (0.30) 0.77-0.84 Q56 Reasons for visiting nearest provider 81.5 0.61 (0.17) 0.57-0.65
Q6 Attendance to school 98.3 0.76 (0.10) 0.72-0.80 Q57 Reasons for admission 90.6 0.67 (0.12) 0.63-0.70
Q7 Highest level of education 98.7 0.69 (0.26) 0.65-0.71 Q58 Cash payment for services 93.7 0.58 (0.10) 0.55-0.62
Q8 Years of education 93.9 0.60 (0.15) 0.56-0.63 Q60 Mode of payment 94.5 0.63 (0.09) 0.60-0.66
Q9 Marital status 99.6 0.87 (0.06) 0.82-0.92 Q61 Source of funds 96.2 0.74 (0.31) 0.71-0.77
Q10 Main employment activity 96.2 0.78 (0.24) 0.73-0.82 Q62 Source of medicine/drugs 97.4 0.86 (0.15) 0.83-0.89
Q11 Occupation 94.5 0.83 (0.12) 0.80-0.85 Q63 Quality of care satisfaction 89.1 0.79 (0.11) 0.75-0.82
Q12 Relative health status 93.6 0.85 (0.09) 0.81-0.89 Q65 Quality of care aspects 85.2 0.58 (0.22) 0.54-0.61
Q13 Smoking status 98.9 0.92 (0.13) 0.98-0.98 Q69 Accompanied by household member 97.2 0.64 (0.23) 0.60-0.68
Q14 Chronic health condition 97.3 0.65 (0.32) 0.61-0.69 Q70 Identification of member 94.3 0.82 (0.17) 0.78-0.86
Section 2: health-seeking behavior Section 5: access to health insurance
Q16 Consult health provider 88.6 0.78 (0.09) 0.74-0.83 Q72 Household members with insurance 90.7 0.77 (0.18) 0.73-0.81
Q17 Compliance to therapy 77.7 0.44 (0.14) 0.42-0.46 Q73 Types of coverage 91.6 0.91 (0.08) 0.86-0.95
Q18 Reason for non-compliance 82.6 0.48 (0.13) 0.43-0.52 Q74 Payers for coverage 94.3 0.84 (0.15) 0.78-0.90
Q19 Reason for not seeking care 89.4 0.76 (0.27) 0.73-0.80 Q75 Method of payment 99.8 0.92 (0.10) 0.88-0.96
Q20 Preventive care-seeking 92.1 0.67 (0.19) 0.62-0.71 Q77 Services covered 79.4 0.56 (0.22) 0.51-0.60
Q21 Need to admit – past 12 months 86.4 0.91 (0.16) 0.85-0.94 Section 6: housing conditions, amenities and assets
Q22 Admission status 95.3 0.87 (0.08) 0.82-0.93 Q78 Type of dwelling 99.4 0.82 (0.16) 0.79-0.85
Q23 Reason for non-admission 84.2 0.71 (0.12) 0.67-0.73 Q79 Owner of dwelling 99.2 0.87 (0.12) 0.83-0.92
Section 3: out-patient and health-related services usage over the last 4 weeks Q80 Items owned in household 82.9 0.73 (0.27) 0.68-0.77
Q25 Type of illness 87.3 0.65 (0.19) 0.62-0.68
Q26 Type of services 88.4 0.69 (0.24) 0.66-0.72
Q27 Name of health provider 99.3 0.86 (0.17) 0.83-0.89
Q28 Type of health provider 95.6 0.79 (0.13) 0.75-0.82
Q30 Owner of health provider 92.7 0.80 (0.10) 0.76-0.84
Q31 Reasons for not visiting nearest provider 78.4 0.53 (0.26) 0.50-0.56
Q32 Reasons for visiting nearest provider 94.1 0.85 (0.07) 0.82-0.88
Q33 Purchase of medicine/drugs 75.2 0.44 (0.17) 0.42-0.47
Q34 Reasons for not purchasing medicine/drugs 89.1 0.87 (0.10) 0.83-0.91
Q35 Reasons for purchasing medicine/drugs 90.5 0.76 (0.15) 0.72-0.79
Q38 Mode of payment 97.1 0.83 (0.13) 0.79-0.87
Q39 Source of funds 96.4 0.91 (0.09) 0.88-0.94
Q40 Waiting time pre-consultation 82.7 0.69 (0.26) 0.66-0.72
Q41 Waiting time-post consultation 89.5 0.81 (0.29) 0.78-0.83
Q43 Transport method to provider 93.2 0.76 (0.35) 0.72-0.79
Q44 Quality of care satisfaction 94.6 0.81 (0.13) 0.77-0.85
Q45 Quality of care aspects 91.8 0.77 (0.39) 0.72-0.81
Q46 Routine health expenses 92.7 0.66 (0.18) 0.63-0.69
Q47 Household persons with routine health expenses 98.9 0.89 (0.24) 0.83-0.95

Note: aFor categorical items with more than two outcomes, a weighted κ coefficient was calculated

Test-retest reliability results for continuous items in household health expenditure questionnaire

Item no. Question description Test mean (SD) Retest mean (SD) p-value ICC 95% CI
Section 2: health-seeking behavior
Q15 Illness in past 4 weeks 2.8 2.6 0.26 0.91 0.88-0.94
Section 3: out-patient and health-related services usage over the last 4 weeks
Q24 Number of out-patient visits 1.6 1.5 0.54 0.87 0.81-0.94
Q29 Distance of provider to home (in km) 0.14 0.13 0.27 0.77 0.71-0.82
Q36 Cash payment for services (in RM) 89.90 84.10 0.10 0.86 0.82-0.91
Q37 Breakdown of payment for services (in %) 44.6 45.0 0.26 0.83 0.77-0.88
Q42 Transport costs to provider (in RM) 11.10 10.90 0.10 0.91 0.86-0.95
Q48 Cost of routine health expenses (in RM) 157.00 152.60 0.09 0.74 0.68-0.79
Section 4: in-patient admission-last 1 year
Q49 Number of in-patient visits 0.6 0.5 0.33 0.92 0.90-0.96
Q50 Duration of admission (in days) 3.8 4.2 0.28 0.90 0.85-0.95
Q53 Distance of provider to home (in km) 0.24 0.29 0.15 0.91 0.87-0.96
Q59 Breakdown of payment for services (in %) 67.2 65.8 0.17 0.85 0.81-0.89
Q66 Travel time to facility (in hours) 1.3 1.0 0.32 0.96 0.91-0.99
Q67 Distance to facility (in km) 11.3 10.9 0.13 0.84 0.79-0.88
Q68 Transport costs to provider 102.10 101.40 0.11 0.83 0.81-0.86
Q71 Days accompanied 3.1 2.9 0.46 0.74 0.69-0.77
Section 5: access to health insurance
Q76 Premium amount 1,892.00 1,993.50 0.08 0.63 0.55-0.68
Section 7: household expenditure and consumption
Q81 Key food and beverage expenditure – last 7 days 559.30 587.40 0.11 0.72 0.67-0.76
Q82 Key consumables expenditure – last one month 1,810.40 1,796.70 0.09 0.63 0.59-0.70
Q83 Key expenditures – last one year 16,011.20 16,175.00 0.07 0.61 0.58-0.65

References

 1.Tangcharoensathien V, Mills A, Palu T. Accelerating health equity: the key role of universal health coverage in the sustainable development goals. BMC Med. 2015; 13(101): 1-5.

 2.Horton R, Das P. Universal health coverage: not why, what, or when – but how? Lancet. 2015; 385(9974): 1156-57. doi: 10.1016/s0140-6736(14)61742-6

 3.World Health Organization [WHO]. The World Health Report 2000: health systems: improving performance. Geneva: World Health Organization; 2009 [cited 2016 Feb 20]. Available from: www.who.int/whr/1999/en/whr99_en.pdf?ua=1

 4.World Health Organization [WHO]. The World Health Report 2010 – health systems financing: the path to universal coverage. Geneva: World Health Organization; 2010 [cited 2016 Mar 2]. Available from: www.who.int/whr/2010/whr10_en.pdf?ua=1

 5.Safurah JK, Khairiyah AM, Nour HO, Healy J. Malaysia health system review: health systems in transition. Asia Pacific Observatory on Health Systems and Policies. Report No. 3, Geneva: World Health Organization; 2013 [cited 2016 Feb 17]. Available from: www.wpro.who.int/asia_pacific_observatory/hits/series/Malaysia_Health_Systems_Review2013.pdf

 6.Carrin G, Evans D, Xu K. Designing health financing policy towards universal coverage. Bull World Health Organ. 2007; 85(9): 652.

 7.Murallitharan M. Effectiveness of a health microinsurance scheme for private primary care in Malaysia (HMI); 2016 [cited 2016 Apr 27]. Available from: https://clinicaltrials.gov/ct2/show/NCT02696174

 8.Murallitharan M. Effectiveness of health microinsurance scheme in private primary care in Malaysia; 2016 [cited 2016 Apr 27]. Available from: www.isrctn.com/ISRCTN10261528

 9.Ministry of Medical Services, Ministry of Public Health and Sanitation. Kenya National Health Accounts 2009/10. Health System 20/20. Nairobi: Ministry of Medical Services; 2011.

10.Chuma J, Maina T. Catastrophic health care spending and impoverishment in Kenya. BMC Health Serv Res. 2012; 12(413): 1-9.

11.Xu HR, Saksena P, Evans D. Validity and comparability of out-of-pocket health expenditure from household surveys: a review of the literature and current survey instruments. Discussion Paper No. 1, Geneva: WHO; 2011 [cited 2016 Mar 22]. Available from: www.who.int/health_financing/documents/dp_e_11_01-oop_errors.pdf

12.Ministry of Health Malaysia. List of registered private clinics up to 31 Dec 2015 (Medical). Private Medical Practice Division. Kuala Lumpur Ministry of Health Malaysia (MOH); 2016 [cited 2016 Feb 2]. Available from: http://medicalprac.moh.gov.my/v2/uploads/04%20Klinik%20PERUBATAN%20SWASTA%2031%20DIS%202015.pdf (in Malay).

13.Department of Statistics Malaysia. Population and housing census of Malaysia 2010. Kuala Lumpur Department of Statistics Malaysia; 2011.

14.Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health. 2006; 29(5): 489-97. doi: 10.1002/nur.20147

15.Chiwaridzo M, Chikasha TN, Naidoo N, Dambi JM, Tadyanemhandu C, Munambah N, et al.. Content validity and test-retest reliability of a low back pain questionnaire in Zimbabwean adolescents. Arch Physiother. 2017; 7(1): 3. doi: 10.1186/s40945-017-0031-y

16.McNeely J, Strauss SM, Wright S, Rotrosen J, Khan R, Lee JD, et al. Test-retest reliability of a self-administered alcohol, smoking and substance involvement screening test (ASSIST) in primary care patients. J Subst Abuse Treat. 2014; 47(1): 93-101. doi: 10.1016/j.jsat.2014.01.007

17.Xu K, Ravndal F, Evans DB, Carrin G. Assessing the reliability of household expenditure data: results of the World Health Survey. Health Policy. 2009; 91(3): 297-305. doi: 10.1016/j.healthpol.2009.01.002

18.Lavado RF, Brooks BP, Hanlon M. Estimating health expenditure shares from household surveys. Bull World Health Organ. 2013; 91(7): 519-24C. doi: 10.2471/BLT.12.115535

19.Buigut S, Ettarh R, Amendah DD. Catastrophic health expenditure and its determinants in Kenya slum communities. Int J Equity Health. 2015; 14(46): 1-12. doi: 10.1186/s12939-015-0168-9

20.Gilbert GE, Prion S. Making sense of methods and measurement: Lawshe’s content validity index. Clin Simul Nur. 2016; 12(12): 530-31.

21.Rutherford-Hemming T. Determining content validity and reporting a content validity index for simulation scenarios. Nurs Educ Perspect. 2015; 36(6): 389-93.

22.Kim SJ, Han JA, Kim YH, Choi BY, Kim SY, Lee HJ, et al. Test-retest reliability of health behavior items in the Community Health Survey in South Korea. Epidemiol Health. 2015; 37: e2015045. doi: 10.4178/epih/e2015045

Corresponding author

Murallitharan Munisamy can be contacted at: murallimd@gmail.com