- Split View
-
Views
-
Cite
Cite
Yinhuan Hu, Zixia Zhang, Jinzhu Xie, Guanping Wang, The Outpatient Experience Questionnaire of comprehensive public hospital in China: development, validity and reliability, International Journal for Quality in Health Care, Volume 29, Issue 1, February 2017, Pages 40–46, https://doi.org/10.1093/intqhc/mzw133
- Share Icon Share
Abstract
The objective of this study is to describe the development of the Outpatient Experience Questionnaire (OPEQ) and to assess the validity and reliability of the scale.
Literature review, patient interviews, Delphi method and Cross-sectional validation survey.
Six comprehensive public hospitals in China.
The survey was carried out on a sample of 600 outpatients.
Acceptability of the questionnaire was assessed according to the overall response rate, item non-response rate and the average completion time. Correlation coefficients and confirmatory factor analysis were used to test construct validity. Delphi method was used to assess the content validity of the questionnaire. Cronbach's coefficient alpha and split-half reliability coefficient were used to estimate the internal reliability of the questionnaire.
The overall response rate was 97.2% and the item non-response rate ranged from 0% to 0.3%. The mean completion time was 6 min. The Spearman correlations of item-total score ranged from 0.466 to 0.765. The results of confirmatory factor analysis showed that all items had factor loadings above 0.40 and the dimension intercorrelation ranged from 0.449 to 0.773, the goodness of fit of the questionnaire was reasonable. The overall authority grade of expert consultation was 0.80 and Kendall's coefficient of concordance W was 0.186. The Cronbach's coefficients alpha of six dimensions ranged from 0.708 to 0.895, the split-half reliability coefficient (Spearman–Brown coefficient) was 0.969.
The OPEQ is a promising instrument covering the most important aspects which influence outpatient experiences of comprehensive public hospital in China. It has good evidence for acceptability, validity and reliability.
Patient experience is an important indicator of healthcare quality from the patient's view [1]. The measurement of patient perceptions is increasingly recognized as a critical component in the evaluation of healthcare interventions and health service quality [2]. For example, the results of patient experience survey have been the basis for the payment to hospital, for the quality management of hospital and for the patient's hospital choice in American and European countries [3]. Patient experience scale is an effective way to obtain information about patient experience [4]. So far, a lot of researches about the patient experience measurement has been done, such as the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) developed for patient experience and satisfaction monitoring in United States [5], Picker Scale (PPE-15) widely used in Europe [6], French inpatient experience questionnaire [7], core, common patient experience questions (CCQs) used to survey day procedure and overnight-admitted patients of hospitals in Australia [8], the Outpatient Experiences Questionnaire (OPEQ) in Norway [9]. These scales cannot be simply applied to China because of different health system structures, hospital environments and local cultures [10]. For example,improving parking is an important priority of quality improvement for Australian outpatients [11], however, outpatients in Chinese hospitals do not pay much attention to it because of their comparatively low proportion of car use in outpatient visits. Meanwhile, medical charges influence chiefly the satisfaction of Chinese patients owing to the low reimbursement rate of health insurance [12], but it is not be much concerned by patients in the countries with well-developed health insurance. The researches on patient experience in China are started later than the Occidental world, the current measurement tools developed for patient experience in China are only applicable to hospitalized patients, such as the Hong Kong Inpatient Experience Questionnaire (HKIEQ) [10], Chinese Hospital Patient Experience and Satisfaction Monitoring Scale (CHPESMS) [13] and the inpatient experience questionnaire (IPEQ) [14].
Outpatient department is the window of Chinese public hospital, it is the first point of entry for the vast majority of patients arriving at hospitals. Studer Group and HCAHPS both have confirmed that patient satisfaction of the outpatient departments has a major impact on hospital inpatient satisfaction and overall satisfaction [15]. If a patient is not satisfied with the outpatient services, it is difficult to change his/her impression on the hospital. The number of outpatients accounts for a large proportion of total patients amount in Chinese public hospitals. Therefore, giving patients a good ‘first impression’ is very important for patient experience improvement and good hospital brand culture development.
In recent years, doctor–patient conflicts have become very serious in China [16, 17]. According to the Chinese Hospital Association, the incidence of reported medical disputes in China has increased by 22.9% on average per year since 2002. Doctor–patient conflicts arise in the interaction process between patients and healthcare providers. Thus, analysing the reasons of the doctor–patient conflicts from the micro-level of patient experience and identifying problems of healthcare provision can help to improve the quality of medical services and to build a harmonious doctor–patient relationship in China.
The goal of this study is to develop a outpatient experience measurement tool for Chinese comprehensive public hospital with high degree of reliability and validity, which can help us to identify problems of outpatient services from the patient's view. It will be useful for quality improvement of health services and harmonious doctor–patient relationship rebuilding. Furthermore, it also provides the basis for performance evaluation of hospitals and for health insurance payments to hospitals.
Methods
Study design
The survey was conducted by trained interviewers through face-to-face interviews in the outpatient department based on the OPEQ. In order to exclude confounding factors and eliminate the ideological concerns of patients, the investigators could not contact with the medical staffs of hospitals and must promise to the participants that the findings will not be feed back to the hospital they visited.
Participants
The survey was carried out on a sample of 600 outpatients from six comprehensive public hospitals (three tertiary hospitals and three secondary hospitals, 100 outpatients per hospital) in Hubei province, China [18]. First, the hospitals were chosen by representative sampling method. We chose a university hospital (tertiary hospital), a province-affiliated tertiary hospital, a city-affiliated tertiary hospital, a secondary hospital in big city and two county hospitals (secondary hospital) in two different counties as sample hospitals. These hospitals represent typically diffrent types of comprehensive public hospitals in China. Second, in each sampled hospital, the face-to-face interviews were conducted in outpatient department when outpatients ended their visits. The outpatients were sampled randomly by the interviewer, that is, the interviewer selected the outpatients who met the inclusion criteria of the survey, asked them questions according to the questionnaire and noted their responses. When an invited outpatient refused to participate in the survey, the interviewer turned to another until 100 interviews were conducted in each sampled hospital. The inclusion criteria of participants were as follows: (i) visit procedures were completed, (ii) aged ≧18 years old, (iii) informed consent, willing to join the study and (iv) able to describe their own experiences independently and accurately.
Questionnaire development
There were three phases in the development of the OPEQ: (i) formed a pre-scale through literature review, outpatient interviews and two-rounds expert consultation; (ii) conducted the pre-survey and identified scale items and dimensions according to the results of statistical analysis based on the pre-survey, and formed the OPEQ; (iii) tested the validity and reliability of the scale OPEQ according to the analysis results of the survey.
Form a pre-scale
The initial item pool of the scale was developed in three steps. First, we conducted a thorough literature review to identify instruments that used to measure patient experience. As a second step, we conducted interviews with 15 outpatients to obtain information of their visit experiences and the way they explain or interpret them. Thirdly, based on literature review and outpatient interviews, we conducted a two-rounds expert consultation. We modified some items, eliminated items that were not relevant and added items that were considered important. The final pre-scale including 5-dimensions, 37-items with 5-point Likert scale was developed.
Form the OPEQ
The pre-survey in a sample of 222 adult outpatients was conducted with the pre-scale during June and July 2015. The response rate, the critical ratio method, the variation coefficient method, the correlation analysis and the exploratory factor analysis were used to select items of the scale. Response rates of each item were calculated and items with response rate over 80% were included. The critical ratio (CR) method was to calculate total score of each respondent and sort the scores from high to low, the total scores in the top 27% were divided as high-score group and the total scores in the bottom 27% were divided as low-score group, t-test was used to test the difference in means of each item for the two groups, if P < 0.01, which means that the difference was statistically significant, the item was included. The coefficient of variation (CV) is the ratio of standard deviation and mean, greater coefficient of variation indicates higher sensitivity of the item, if CV of the item > 15%, the item was included. The correlation analysis was to calculate item-dimention correlation and item-total score correlation for each item, if the correlation > 0.4, then the item was included. Kaiser–Meyer–Olkin (KMO) measure and the Bartlett test of sphericity were used to determine appropriateness of factor analysis. The results showed a very good appropriateness (KMO: 0.910, Bartlett significance: 0.000), thus principal component analysis was conducted and oblique rotation (promax) was used. Finally, 28 items were included and categorized into six different factors explaining 63.5% of the total variance according to Kaiser criterion (Eigenvalue > 1). If a item was excluded by one of the above methods, then it will be excluded. According to the results of above statistical analysis, the 6-dimension, 28-item scale OPEQ was developed, which was sensitive, independent, representative and stability.
Test the validity and reliability of the scale
The SPSS 19.0 and AMOS 17.0 were used to analyse Cronbach's coefficients alpha, split-half reliability and the construct validity of the questionnaire, which indicate whether the OPEQ has good validity and reliability.
Statistical analysis
Data entry and management were performed using EpiData, version 3.1. Double-entry data input ensured accuracy. AMOS 17.0 was used to conduct the confirmatory factor analysis. Descriptive statistics of the sampled patients demographics and descriptive analysis for the items of OPEQ were presented using frequencies and percentages. Ceiling effects were evaluated by calculating the percentage of outpatients selecting the highest response category. The common limit of ceiling effects was 85% [19]. The scoring of the scale was computed by summing the item scores. Psychometric properties of 28 evaluative items were tested for acceptability, content validity, construct validity and reliability.
Acceptability: Acceptability of the questionnaire was judged according to the overall response rate, the item non-response rate and the average completion time [20, 21]. The overall response rate was the number of respondents accounts for the initial sample size [22]. The item non-response rate was calculated as the proportion of responders who refused to provide responses to each item. These non-responses were counted as missing values when calculating scores. Mean imputation was conducted for the missing item scores. The average completion time was calculated to test the appropriateness of the survey instrument length, which is an important index for the acceptability of the questionnaire.
Validity: The validity of the scale includes construct validity and content validity. Correlation coefficients and confirmatory factor analysis (CFA) were used to test construct validity. The total score was the summated score for all items of the scale. Dimension intercorrelation was used to evaluate the dimension structure of the scale. If the Spearman correlations of item-total score were higher than 0.4, and dimension intercorrelation was less than 0.80 [23], the Spearman correlations of dimension-total score were higher than dimension intercorrelation, then the questionnaire has good relevance and discrimination. Factor loadings were examined with the criterion of greater than 0.4 to assess the appropriateness of CFA resulting structure [24], Goodness of fit of the questionnaire was tested using goodness of fit index, such as x2, GFI, CFI, TLI, RMSEA and RMR [25], if CFI > 0.90,TLI > 0.90,RMSEA < 0.08, it indicates that the goodness of fit is reasonable and acceptable [26, 27]. Delphi method was used to test the content validity of the questionnaire. In this study, 32 experts from different fields including scientific researchers, hospital administrators, statisticians, doctors and nurses were invited to attend the two-rounds expert consultation about the dimensions and the items of the questionnaire and their language expressions. The overall authority grade of the experts consultation and the response rates of the two-rounds consultation were used to indicate the validity of the consultation. Kendall's coefficient of concordance W was computed to examine the consistency of experts’ ratings for the items. Literature review shows that it is reasonable if the overall authority grade ≧ 0.70 [28, 29]. Combining with the experts’ suggestions and healthcare requirements, we revised dimensions and items of the questionnaire.
Reliability: Reliability of the questionnaire was estimated using Cronbach's coefficient alpha and split-half reliability coefficient, which checked if items of questionnaire measured the same concept. Cronbach's coefficient alpha of each dimension was computed to evaluate the extent of respondents answering the items consistently. The Cronbach's coefficient alpha ranges from 0 to 1. Literature review shows that if Cronbach's coefficient alpha ≧ 0.70, then the reliability of the scale is considered acceptable [30]. The split-half reliability coefficient is an alternative estimation of internal consistency reliability [31, 32]. With this method, items were divided into two half-scales so that they are as equivalent as possible. According to the correlation of the two half-scales, the reliability of the total scale can be estimated. We computed the correlation between the two half-scale scores and then applied the Spearman–Brown prophecy formula to estimate the reliability of the total scale.
Ethical consideration
Ethics approval was obtained from the Ethics Committee of Tongji Medical College, Huazhong University of Science and Technology. Informed verbal consent was conducted prior to the survey interview. Participants were allowed to withdraw from the survey at any time. All interviews of the survey were kept confidential and anonymous.
Results
A total of 600 outpatients were sampled, 17 questionnaires were not valid and were excluded. The overall response rate was 97.2% (583/600). In total, 583 questionnaires were analysed. Respondents characteristics are shown in Table 1.
Characteristics . | n . | % . |
---|---|---|
Gender | ||
Male | 240 | 41.2 |
Female | 343 | 58.8 |
Age (years) | ||
18–44 | 348 | 59.7 |
45–59 | 129 | 22.1 |
≧60 | 106 | 18.2 |
Type of payment | ||
Out-of-pocket payment totally | 323 | 55.4 |
Third-party reimbursed totally | 21 | 3.6 |
Out-of-pocket payment and third-party reimbursed mixed | 239 | 41.0 |
Education level | ||
Master or above | 11 | 1.9 |
Bachelor /college degree | 200 | 34.3 |
Technical secondary school/senior high school | 187 | 32.1 |
Junior middle school | 127 | 21.8 |
Primary or below | 58 | 9.9 |
Average monthly personal income (in CNY ¥) | ||
<500 | 23 | 3.9 |
500–999 | 45 | 7.7 |
1000–1999 | 102 | 17.5 |
2000–2999 | 146 | 25.0 |
3000–3999 | 121 | 20.8 |
4000–4999 | 83 | 14.2 |
≧5000 | 63 | 10.8 |
First visit to the hospital | ||
Yes | 151 | 25.9 |
No | 432 | 74.1 |
Self-rated health status | ||
Very good | 48 | 8.2 |
Good | 185 | 31.7 |
Fair | 269 | 46.1 |
Poor | 72 | 12.3 |
Very poor | 9 | 1.5 |
Marital status | ||
Single | 95 | 16.3 |
Married | 469 | 80.4 |
Divorced/widow | 19 | 3.3 |
Characteristics . | n . | % . |
---|---|---|
Gender | ||
Male | 240 | 41.2 |
Female | 343 | 58.8 |
Age (years) | ||
18–44 | 348 | 59.7 |
45–59 | 129 | 22.1 |
≧60 | 106 | 18.2 |
Type of payment | ||
Out-of-pocket payment totally | 323 | 55.4 |
Third-party reimbursed totally | 21 | 3.6 |
Out-of-pocket payment and third-party reimbursed mixed | 239 | 41.0 |
Education level | ||
Master or above | 11 | 1.9 |
Bachelor /college degree | 200 | 34.3 |
Technical secondary school/senior high school | 187 | 32.1 |
Junior middle school | 127 | 21.8 |
Primary or below | 58 | 9.9 |
Average monthly personal income (in CNY ¥) | ||
<500 | 23 | 3.9 |
500–999 | 45 | 7.7 |
1000–1999 | 102 | 17.5 |
2000–2999 | 146 | 25.0 |
3000–3999 | 121 | 20.8 |
4000–4999 | 83 | 14.2 |
≧5000 | 63 | 10.8 |
First visit to the hospital | ||
Yes | 151 | 25.9 |
No | 432 | 74.1 |
Self-rated health status | ||
Very good | 48 | 8.2 |
Good | 185 | 31.7 |
Fair | 269 | 46.1 |
Poor | 72 | 12.3 |
Very poor | 9 | 1.5 |
Marital status | ||
Single | 95 | 16.3 |
Married | 469 | 80.4 |
Divorced/widow | 19 | 3.3 |
Characteristics . | n . | % . |
---|---|---|
Gender | ||
Male | 240 | 41.2 |
Female | 343 | 58.8 |
Age (years) | ||
18–44 | 348 | 59.7 |
45–59 | 129 | 22.1 |
≧60 | 106 | 18.2 |
Type of payment | ||
Out-of-pocket payment totally | 323 | 55.4 |
Third-party reimbursed totally | 21 | 3.6 |
Out-of-pocket payment and third-party reimbursed mixed | 239 | 41.0 |
Education level | ||
Master or above | 11 | 1.9 |
Bachelor /college degree | 200 | 34.3 |
Technical secondary school/senior high school | 187 | 32.1 |
Junior middle school | 127 | 21.8 |
Primary or below | 58 | 9.9 |
Average monthly personal income (in CNY ¥) | ||
<500 | 23 | 3.9 |
500–999 | 45 | 7.7 |
1000–1999 | 102 | 17.5 |
2000–2999 | 146 | 25.0 |
3000–3999 | 121 | 20.8 |
4000–4999 | 83 | 14.2 |
≧5000 | 63 | 10.8 |
First visit to the hospital | ||
Yes | 151 | 25.9 |
No | 432 | 74.1 |
Self-rated health status | ||
Very good | 48 | 8.2 |
Good | 185 | 31.7 |
Fair | 269 | 46.1 |
Poor | 72 | 12.3 |
Very poor | 9 | 1.5 |
Marital status | ||
Single | 95 | 16.3 |
Married | 469 | 80.4 |
Divorced/widow | 19 | 3.3 |
Characteristics . | n . | % . |
---|---|---|
Gender | ||
Male | 240 | 41.2 |
Female | 343 | 58.8 |
Age (years) | ||
18–44 | 348 | 59.7 |
45–59 | 129 | 22.1 |
≧60 | 106 | 18.2 |
Type of payment | ||
Out-of-pocket payment totally | 323 | 55.4 |
Third-party reimbursed totally | 21 | 3.6 |
Out-of-pocket payment and third-party reimbursed mixed | 239 | 41.0 |
Education level | ||
Master or above | 11 | 1.9 |
Bachelor /college degree | 200 | 34.3 |
Technical secondary school/senior high school | 187 | 32.1 |
Junior middle school | 127 | 21.8 |
Primary or below | 58 | 9.9 |
Average monthly personal income (in CNY ¥) | ||
<500 | 23 | 3.9 |
500–999 | 45 | 7.7 |
1000–1999 | 102 | 17.5 |
2000–2999 | 146 | 25.0 |
3000–3999 | 121 | 20.8 |
4000–4999 | 83 | 14.2 |
≧5000 | 63 | 10.8 |
First visit to the hospital | ||
Yes | 151 | 25.9 |
No | 432 | 74.1 |
Self-rated health status | ||
Very good | 48 | 8.2 |
Good | 185 | 31.7 |
Fair | 269 | 46.1 |
Poor | 72 | 12.3 |
Very poor | 9 | 1.5 |
Marital status | ||
Single | 95 | 16.3 |
Married | 469 | 80.4 |
Divorced/widow | 19 | 3.3 |
Acceptability
Most of the respondents completed the questionnaire within 10 min, the mean completion time was 6 min. The item non-response rate ranged from 0 to 0.3%. Twelve of the 28 items had 100% response. Ceiling effects ranged from 6.7% to 37.0% (Table 2). Considering the completion time and high overall response rate and low item non-response rate, the questionnaire was well accepted by outpatients.
Dimensions and items . | Item non-response, n (%) . | Ceiling effect, n (%) . |
---|---|---|
Physical environment and convenience | ||
1. Waiting time was short | 0 | 39 (6.7) |
2. Registration procure was easy | 0 | 162 (27.8) |
3. Dispensary/payment was convenient | 2 (0.3) | 122 (20.9) |
4. Visit instructions and indication signs were clear | 1 (0.2) | 153 (26.2) |
5. Outpatient department was clean | 2 (0.3) | 131 (22.5) |
6. Outpatient department was quiet | 1 (0.2) | 63 (10.8) |
Doctor–patient communication | ||
7. Health professionals explained things clearly and understandably | 1 (0.2) | 166 (28.5) |
8. Health professionals listened to you carefully | 0 | 147 (25.2) |
9. You had enough time to communicate with the doctor | 1 (0.2) | 99 (17.0) |
10. Health professionals treated you with courtesy and respect | 2 (0.3) | 142 (24.4) |
11. Health professionals cared about your anxieties or fears | 0 | 83 (14.2) |
12. You were involved in decision-making about treatment | 1 (0.2) | 69 (11.8) |
13. Your opinions/thoughts were respected by health professionals | 0 | 96 (16.5) |
14. Health professionals would like to protect personal privacy | 0 | 131 (22.5) |
Health information | ||
15. You were given explanations concerning your illness | 0 | 166 (28.5) |
16. You were informed the signals of dangerous conditions related to your illness when you are at home | 2 (0.3) | 177 (30.4) |
17. You were informed of health knowledge related to your illness | 0 | 177 (30.4) |
18. You were given explanation about the following examination or treatment | 1 (0.2) | 134 (23.0) |
19. You were given explanation about the results of examination/test | 1 (0.2) | 172 (29.5) |
20. Health professionals explained the drug effects in a way you could understand | 1 (0.2) | 132 (22.6) |
21. You were informed of medication precautions (directions and dosage, side effects, contraindications, etc.) | 1 (0.2) | 168 (28.8) |
Medical expenses | ||
22. Charges of the visit were reasonable | 0 | 56 (9.6) |
23. Charges of the visit were transparent | 0 | 83 (14.2) |
24. Expenses of the visit were too expensive to afford | 0 | 92 (15.8) |
Short-time outcome | ||
25. This visit could help you reduce or prevent your health problems | 1 (0.2) | 115 (19.7) |
26. You know how to handle such health problems after this visit | 0 | 89 (15.3) |
General satisfaction | ||
27. You were satisfied with this visit in general | 1 (0.2) | 147 (25.2) |
28. You would choose this hospital again if you have demands | 2 (0.3) | 216 (37.0) |
Dimensions and items . | Item non-response, n (%) . | Ceiling effect, n (%) . |
---|---|---|
Physical environment and convenience | ||
1. Waiting time was short | 0 | 39 (6.7) |
2. Registration procure was easy | 0 | 162 (27.8) |
3. Dispensary/payment was convenient | 2 (0.3) | 122 (20.9) |
4. Visit instructions and indication signs were clear | 1 (0.2) | 153 (26.2) |
5. Outpatient department was clean | 2 (0.3) | 131 (22.5) |
6. Outpatient department was quiet | 1 (0.2) | 63 (10.8) |
Doctor–patient communication | ||
7. Health professionals explained things clearly and understandably | 1 (0.2) | 166 (28.5) |
8. Health professionals listened to you carefully | 0 | 147 (25.2) |
9. You had enough time to communicate with the doctor | 1 (0.2) | 99 (17.0) |
10. Health professionals treated you with courtesy and respect | 2 (0.3) | 142 (24.4) |
11. Health professionals cared about your anxieties or fears | 0 | 83 (14.2) |
12. You were involved in decision-making about treatment | 1 (0.2) | 69 (11.8) |
13. Your opinions/thoughts were respected by health professionals | 0 | 96 (16.5) |
14. Health professionals would like to protect personal privacy | 0 | 131 (22.5) |
Health information | ||
15. You were given explanations concerning your illness | 0 | 166 (28.5) |
16. You were informed the signals of dangerous conditions related to your illness when you are at home | 2 (0.3) | 177 (30.4) |
17. You were informed of health knowledge related to your illness | 0 | 177 (30.4) |
18. You were given explanation about the following examination or treatment | 1 (0.2) | 134 (23.0) |
19. You were given explanation about the results of examination/test | 1 (0.2) | 172 (29.5) |
20. Health professionals explained the drug effects in a way you could understand | 1 (0.2) | 132 (22.6) |
21. You were informed of medication precautions (directions and dosage, side effects, contraindications, etc.) | 1 (0.2) | 168 (28.8) |
Medical expenses | ||
22. Charges of the visit were reasonable | 0 | 56 (9.6) |
23. Charges of the visit were transparent | 0 | 83 (14.2) |
24. Expenses of the visit were too expensive to afford | 0 | 92 (15.8) |
Short-time outcome | ||
25. This visit could help you reduce or prevent your health problems | 1 (0.2) | 115 (19.7) |
26. You know how to handle such health problems after this visit | 0 | 89 (15.3) |
General satisfaction | ||
27. You were satisfied with this visit in general | 1 (0.2) | 147 (25.2) |
28. You would choose this hospital again if you have demands | 2 (0.3) | 216 (37.0) |
Dimensions and items . | Item non-response, n (%) . | Ceiling effect, n (%) . |
---|---|---|
Physical environment and convenience | ||
1. Waiting time was short | 0 | 39 (6.7) |
2. Registration procure was easy | 0 | 162 (27.8) |
3. Dispensary/payment was convenient | 2 (0.3) | 122 (20.9) |
4. Visit instructions and indication signs were clear | 1 (0.2) | 153 (26.2) |
5. Outpatient department was clean | 2 (0.3) | 131 (22.5) |
6. Outpatient department was quiet | 1 (0.2) | 63 (10.8) |
Doctor–patient communication | ||
7. Health professionals explained things clearly and understandably | 1 (0.2) | 166 (28.5) |
8. Health professionals listened to you carefully | 0 | 147 (25.2) |
9. You had enough time to communicate with the doctor | 1 (0.2) | 99 (17.0) |
10. Health professionals treated you with courtesy and respect | 2 (0.3) | 142 (24.4) |
11. Health professionals cared about your anxieties or fears | 0 | 83 (14.2) |
12. You were involved in decision-making about treatment | 1 (0.2) | 69 (11.8) |
13. Your opinions/thoughts were respected by health professionals | 0 | 96 (16.5) |
14. Health professionals would like to protect personal privacy | 0 | 131 (22.5) |
Health information | ||
15. You were given explanations concerning your illness | 0 | 166 (28.5) |
16. You were informed the signals of dangerous conditions related to your illness when you are at home | 2 (0.3) | 177 (30.4) |
17. You were informed of health knowledge related to your illness | 0 | 177 (30.4) |
18. You were given explanation about the following examination or treatment | 1 (0.2) | 134 (23.0) |
19. You were given explanation about the results of examination/test | 1 (0.2) | 172 (29.5) |
20. Health professionals explained the drug effects in a way you could understand | 1 (0.2) | 132 (22.6) |
21. You were informed of medication precautions (directions and dosage, side effects, contraindications, etc.) | 1 (0.2) | 168 (28.8) |
Medical expenses | ||
22. Charges of the visit were reasonable | 0 | 56 (9.6) |
23. Charges of the visit were transparent | 0 | 83 (14.2) |
24. Expenses of the visit were too expensive to afford | 0 | 92 (15.8) |
Short-time outcome | ||
25. This visit could help you reduce or prevent your health problems | 1 (0.2) | 115 (19.7) |
26. You know how to handle such health problems after this visit | 0 | 89 (15.3) |
General satisfaction | ||
27. You were satisfied with this visit in general | 1 (0.2) | 147 (25.2) |
28. You would choose this hospital again if you have demands | 2 (0.3) | 216 (37.0) |
Dimensions and items . | Item non-response, n (%) . | Ceiling effect, n (%) . |
---|---|---|
Physical environment and convenience | ||
1. Waiting time was short | 0 | 39 (6.7) |
2. Registration procure was easy | 0 | 162 (27.8) |
3. Dispensary/payment was convenient | 2 (0.3) | 122 (20.9) |
4. Visit instructions and indication signs were clear | 1 (0.2) | 153 (26.2) |
5. Outpatient department was clean | 2 (0.3) | 131 (22.5) |
6. Outpatient department was quiet | 1 (0.2) | 63 (10.8) |
Doctor–patient communication | ||
7. Health professionals explained things clearly and understandably | 1 (0.2) | 166 (28.5) |
8. Health professionals listened to you carefully | 0 | 147 (25.2) |
9. You had enough time to communicate with the doctor | 1 (0.2) | 99 (17.0) |
10. Health professionals treated you with courtesy and respect | 2 (0.3) | 142 (24.4) |
11. Health professionals cared about your anxieties or fears | 0 | 83 (14.2) |
12. You were involved in decision-making about treatment | 1 (0.2) | 69 (11.8) |
13. Your opinions/thoughts were respected by health professionals | 0 | 96 (16.5) |
14. Health professionals would like to protect personal privacy | 0 | 131 (22.5) |
Health information | ||
15. You were given explanations concerning your illness | 0 | 166 (28.5) |
16. You were informed the signals of dangerous conditions related to your illness when you are at home | 2 (0.3) | 177 (30.4) |
17. You were informed of health knowledge related to your illness | 0 | 177 (30.4) |
18. You were given explanation about the following examination or treatment | 1 (0.2) | 134 (23.0) |
19. You were given explanation about the results of examination/test | 1 (0.2) | 172 (29.5) |
20. Health professionals explained the drug effects in a way you could understand | 1 (0.2) | 132 (22.6) |
21. You were informed of medication precautions (directions and dosage, side effects, contraindications, etc.) | 1 (0.2) | 168 (28.8) |
Medical expenses | ||
22. Charges of the visit were reasonable | 0 | 56 (9.6) |
23. Charges of the visit were transparent | 0 | 83 (14.2) |
24. Expenses of the visit were too expensive to afford | 0 | 92 (15.8) |
Short-time outcome | ||
25. This visit could help you reduce or prevent your health problems | 1 (0.2) | 115 (19.7) |
26. You know how to handle such health problems after this visit | 0 | 89 (15.3) |
General satisfaction | ||
27. You were satisfied with this visit in general | 1 (0.2) | 147 (25.2) |
28. You would choose this hospital again if you have demands | 2 (0.3) | 216 (37.0) |
Validity
The Spearman correlations of item-total score ranged from 0.466 to 0.765, all were greater than 0.4 criterion. It was lower than 0.50 for only two items: item 2 and item 3. All the P values were statistically significant (P < 0.01; Table 3). Dimension intercorrelation from confirmatory factor analysis ranged from 0.449 to 0.773, all were less than 0.80 criterion, which indicated that each dimension was unique to be considered separate measure. The Spearman correlations of dimension-total score were higher than dimension intercorrelation (Table 4). The results of confirmatory factor analysis also showed that the item factor loadings ranged from 0.502 to 0.957, all items had factor loadings above the 0.40 criterion (Table 3), the model fit indices were x2/df = 2.775, GFI = 0.893, CFI = 0.930, TLI = 0.921, RMSEA = 0.055, RMR = 0.038, the goodness of fit of the questionnaire was reasonable.
Dimensions and items . | Spearman correlations of item-total score . | Factor loadings . |
---|---|---|
Physical environment and convenience | ||
1. Waiting time was short | 0.586 | 0.689 |
2. Registration procure was easy | 0.499 | 0.603 |
3. Dispensary/payment was convenient | 0.466 | 0.584 |
4. Visit instructions and indication signs were clear | 0.537 | 0.568 |
5. Outpatient department was clean | 0.505 | 0.557 |
6. Outpatient department was quiet | 0.552 | 0.649 |
Doctor–patient communication | ||
7. Health professionals explained things clearly and understandably | 0.508 | 0.502 |
8. Health professionals listened to you carefully | 0.712 | 0.746 |
9. You had enough time to communicate with the doctor | 0.726 | 0.746 |
10. Health professionals treated you with courtesy and respect | 0.732 | 0.755 |
11. Health professionals cared about your anxieties or fears | 0.692 | 0.753 |
12. You were involved in decision-making about treatment | 0.657 | 0.694 |
13. Your opinions/thoughts were respected by health professionals | 0.689 | 0.733 |
14. Health professionals would like to protect personal privacy | 0.590 | 0.581 |
Health information | ||
15. You were given explanations concerning your illness | 0.678 | 0.706 |
16. You were informed the signals of dangerous conditions related to your illness when you are at home | 0.672 | 0.752 |
17. You were informed of health knowledge related to your illness | 0.748 | 0.808 |
18. You were given explanation about the following examination or treatment | 0.697 | 0.741 |
19. You were given explanation about the results of examination/test | 0.703 | 0.732 |
20. Health professionals explained the drug effects in a way you could understand | 0.755 | 0.774 |
21. You were informed of medication precautions (directions and dosage, side effects, contraindications, etc.) | 0.692 | 0.684 |
Medical expenses | ||
22. Charges of the visit were reasonable | 0.692 | 0.866 |
23. Charges of the visit were transparent | 0.654 | 0.795 |
24. Expenses of the visit were too expensive to afford | 0.591 | 0.698 |
Short-time outcome | ||
25. This visit could help you reduce or prevent your health problems | 0.618 | 0.718 |
26. You know how to handle such health problems after this visit | 0.655 | 0.771 |
General satisfaction | ||
27. You were satisfied with this visit in general | 0.765 | 0.957 |
28. You would choose this hospital again if you have demands | 0.634 | 0.759 |
Dimensions and items . | Spearman correlations of item-total score . | Factor loadings . |
---|---|---|
Physical environment and convenience | ||
1. Waiting time was short | 0.586 | 0.689 |
2. Registration procure was easy | 0.499 | 0.603 |
3. Dispensary/payment was convenient | 0.466 | 0.584 |
4. Visit instructions and indication signs were clear | 0.537 | 0.568 |
5. Outpatient department was clean | 0.505 | 0.557 |
6. Outpatient department was quiet | 0.552 | 0.649 |
Doctor–patient communication | ||
7. Health professionals explained things clearly and understandably | 0.508 | 0.502 |
8. Health professionals listened to you carefully | 0.712 | 0.746 |
9. You had enough time to communicate with the doctor | 0.726 | 0.746 |
10. Health professionals treated you with courtesy and respect | 0.732 | 0.755 |
11. Health professionals cared about your anxieties or fears | 0.692 | 0.753 |
12. You were involved in decision-making about treatment | 0.657 | 0.694 |
13. Your opinions/thoughts were respected by health professionals | 0.689 | 0.733 |
14. Health professionals would like to protect personal privacy | 0.590 | 0.581 |
Health information | ||
15. You were given explanations concerning your illness | 0.678 | 0.706 |
16. You were informed the signals of dangerous conditions related to your illness when you are at home | 0.672 | 0.752 |
17. You were informed of health knowledge related to your illness | 0.748 | 0.808 |
18. You were given explanation about the following examination or treatment | 0.697 | 0.741 |
19. You were given explanation about the results of examination/test | 0.703 | 0.732 |
20. Health professionals explained the drug effects in a way you could understand | 0.755 | 0.774 |
21. You were informed of medication precautions (directions and dosage, side effects, contraindications, etc.) | 0.692 | 0.684 |
Medical expenses | ||
22. Charges of the visit were reasonable | 0.692 | 0.866 |
23. Charges of the visit were transparent | 0.654 | 0.795 |
24. Expenses of the visit were too expensive to afford | 0.591 | 0.698 |
Short-time outcome | ||
25. This visit could help you reduce or prevent your health problems | 0.618 | 0.718 |
26. You know how to handle such health problems after this visit | 0.655 | 0.771 |
General satisfaction | ||
27. You were satisfied with this visit in general | 0.765 | 0.957 |
28. You would choose this hospital again if you have demands | 0.634 | 0.759 |
Dimensions and items . | Spearman correlations of item-total score . | Factor loadings . |
---|---|---|
Physical environment and convenience | ||
1. Waiting time was short | 0.586 | 0.689 |
2. Registration procure was easy | 0.499 | 0.603 |
3. Dispensary/payment was convenient | 0.466 | 0.584 |
4. Visit instructions and indication signs were clear | 0.537 | 0.568 |
5. Outpatient department was clean | 0.505 | 0.557 |
6. Outpatient department was quiet | 0.552 | 0.649 |
Doctor–patient communication | ||
7. Health professionals explained things clearly and understandably | 0.508 | 0.502 |
8. Health professionals listened to you carefully | 0.712 | 0.746 |
9. You had enough time to communicate with the doctor | 0.726 | 0.746 |
10. Health professionals treated you with courtesy and respect | 0.732 | 0.755 |
11. Health professionals cared about your anxieties or fears | 0.692 | 0.753 |
12. You were involved in decision-making about treatment | 0.657 | 0.694 |
13. Your opinions/thoughts were respected by health professionals | 0.689 | 0.733 |
14. Health professionals would like to protect personal privacy | 0.590 | 0.581 |
Health information | ||
15. You were given explanations concerning your illness | 0.678 | 0.706 |
16. You were informed the signals of dangerous conditions related to your illness when you are at home | 0.672 | 0.752 |
17. You were informed of health knowledge related to your illness | 0.748 | 0.808 |
18. You were given explanation about the following examination or treatment | 0.697 | 0.741 |
19. You were given explanation about the results of examination/test | 0.703 | 0.732 |
20. Health professionals explained the drug effects in a way you could understand | 0.755 | 0.774 |
21. You were informed of medication precautions (directions and dosage, side effects, contraindications, etc.) | 0.692 | 0.684 |
Medical expenses | ||
22. Charges of the visit were reasonable | 0.692 | 0.866 |
23. Charges of the visit were transparent | 0.654 | 0.795 |
24. Expenses of the visit were too expensive to afford | 0.591 | 0.698 |
Short-time outcome | ||
25. This visit could help you reduce or prevent your health problems | 0.618 | 0.718 |
26. You know how to handle such health problems after this visit | 0.655 | 0.771 |
General satisfaction | ||
27. You were satisfied with this visit in general | 0.765 | 0.957 |
28. You would choose this hospital again if you have demands | 0.634 | 0.759 |
Dimensions and items . | Spearman correlations of item-total score . | Factor loadings . |
---|---|---|
Physical environment and convenience | ||
1. Waiting time was short | 0.586 | 0.689 |
2. Registration procure was easy | 0.499 | 0.603 |
3. Dispensary/payment was convenient | 0.466 | 0.584 |
4. Visit instructions and indication signs were clear | 0.537 | 0.568 |
5. Outpatient department was clean | 0.505 | 0.557 |
6. Outpatient department was quiet | 0.552 | 0.649 |
Doctor–patient communication | ||
7. Health professionals explained things clearly and understandably | 0.508 | 0.502 |
8. Health professionals listened to you carefully | 0.712 | 0.746 |
9. You had enough time to communicate with the doctor | 0.726 | 0.746 |
10. Health professionals treated you with courtesy and respect | 0.732 | 0.755 |
11. Health professionals cared about your anxieties or fears | 0.692 | 0.753 |
12. You were involved in decision-making about treatment | 0.657 | 0.694 |
13. Your opinions/thoughts were respected by health professionals | 0.689 | 0.733 |
14. Health professionals would like to protect personal privacy | 0.590 | 0.581 |
Health information | ||
15. You were given explanations concerning your illness | 0.678 | 0.706 |
16. You were informed the signals of dangerous conditions related to your illness when you are at home | 0.672 | 0.752 |
17. You were informed of health knowledge related to your illness | 0.748 | 0.808 |
18. You were given explanation about the following examination or treatment | 0.697 | 0.741 |
19. You were given explanation about the results of examination/test | 0.703 | 0.732 |
20. Health professionals explained the drug effects in a way you could understand | 0.755 | 0.774 |
21. You were informed of medication precautions (directions and dosage, side effects, contraindications, etc.) | 0.692 | 0.684 |
Medical expenses | ||
22. Charges of the visit were reasonable | 0.692 | 0.866 |
23. Charges of the visit were transparent | 0.654 | 0.795 |
24. Expenses of the visit were too expensive to afford | 0.591 | 0.698 |
Short-time outcome | ||
25. This visit could help you reduce or prevent your health problems | 0.618 | 0.718 |
26. You know how to handle such health problems after this visit | 0.655 | 0.771 |
General satisfaction | ||
27. You were satisfied with this visit in general | 0.765 | 0.957 |
28. You would choose this hospital again if you have demands | 0.634 | 0.759 |
Dimensions . | Items included in each dimension . | Dimensions . | |||||
---|---|---|---|---|---|---|---|
1 . | 2 . | 3 . | 4 . | 5 . | 6 . | ||
1. Physical environment and convenience | 6 | (0.779)a | |||||
2. Doctor–patient communication | 8 | 0.640 | (0.876)a | ||||
3. Health information | 7 | 0.547 | 0.773 | (0.895)a | |||
4. Medical expenses | 3 | 0.518 | 0.614 | 0.588 | (0.824)a | ||
5. Short-time outcome | 2 | 0.449 | 0.595 | 0.672 | 0.612 | (0.708)a | |
6. General satisfaction | 2 | 0.484 | 0.601 | 0.579 | 0.517 | 0.492 | (0.841)a |
Total score | 28 | 0.758 | 0.905 | 0.894 | 0.752 | 0.731 | 0.706 |
Dimensions . | Items included in each dimension . | Dimensions . | |||||
---|---|---|---|---|---|---|---|
1 . | 2 . | 3 . | 4 . | 5 . | 6 . | ||
1. Physical environment and convenience | 6 | (0.779)a | |||||
2. Doctor–patient communication | 8 | 0.640 | (0.876)a | ||||
3. Health information | 7 | 0.547 | 0.773 | (0.895)a | |||
4. Medical expenses | 3 | 0.518 | 0.614 | 0.588 | (0.824)a | ||
5. Short-time outcome | 2 | 0.449 | 0.595 | 0.672 | 0.612 | (0.708)a | |
6. General satisfaction | 2 | 0.484 | 0.601 | 0.579 | 0.517 | 0.492 | (0.841)a |
Total score | 28 | 0.758 | 0.905 | 0.894 | 0.752 | 0.731 | 0.706 |
aCronbach's coefficients alpha of each dimension.
Dimensions . | Items included in each dimension . | Dimensions . | |||||
---|---|---|---|---|---|---|---|
1 . | 2 . | 3 . | 4 . | 5 . | 6 . | ||
1. Physical environment and convenience | 6 | (0.779)a | |||||
2. Doctor–patient communication | 8 | 0.640 | (0.876)a | ||||
3. Health information | 7 | 0.547 | 0.773 | (0.895)a | |||
4. Medical expenses | 3 | 0.518 | 0.614 | 0.588 | (0.824)a | ||
5. Short-time outcome | 2 | 0.449 | 0.595 | 0.672 | 0.612 | (0.708)a | |
6. General satisfaction | 2 | 0.484 | 0.601 | 0.579 | 0.517 | 0.492 | (0.841)a |
Total score | 28 | 0.758 | 0.905 | 0.894 | 0.752 | 0.731 | 0.706 |
Dimensions . | Items included in each dimension . | Dimensions . | |||||
---|---|---|---|---|---|---|---|
1 . | 2 . | 3 . | 4 . | 5 . | 6 . | ||
1. Physical environment and convenience | 6 | (0.779)a | |||||
2. Doctor–patient communication | 8 | 0.640 | (0.876)a | ||||
3. Health information | 7 | 0.547 | 0.773 | (0.895)a | |||
4. Medical expenses | 3 | 0.518 | 0.614 | 0.588 | (0.824)a | ||
5. Short-time outcome | 2 | 0.449 | 0.595 | 0.672 | 0.612 | (0.708)a | |
6. General satisfaction | 2 | 0.484 | 0.601 | 0.579 | 0.517 | 0.492 | (0.841)a |
Total score | 28 | 0.758 | 0.905 | 0.894 | 0.752 | 0.731 | 0.706 |
aCronbach's coefficients alpha of each dimension.
The content validity of OPEQ was judged according to the results of the Delphi method. The overall authority grade of the experts consultation was 0.80 and the respond rates of two-rounds consultation were 91.43% and 87.5%. Kendall's coefficient of concordance W was 0.186 (X2 = 203.633, P = 0.000, P value was statistically significant). It indicated that the questionnaire had good content validity.
Reliability
Cronbach's coefficients alpha of six dimensions ranged from 0.708 to 0.895, all were greater than 0.70 (Table 4), which indicated good internal consistency reliability of the questionnaire.
According to results of the odd–even split-half reliability analysis, split-half reliability coefficient (Spearman–Brown coefficient) was 0.969 with Cronbach's coefficient alpha 0.891 and 0.883 of the odd-group items and even-group, respectively, which also reflected good internal reliability of the OPEQ.
Discussion
According to the results of all the tests, OPEQ performed well on the acceptability, validity and reliability. Therefore, OPEQ can be considered as an effective instrument for assessing outpatient experiences of Chinese comprehensive public hospital.
The final OPEQ version consisted of 28 items covering six dimensions: physical environment and convenience, doctor–patient communication, health information, medical expenses, short-time outcome and general satisfaction, which is also confirmed by the results of other research [33]. In Chinese hospitals, especially in tertiary hospitals, cumbersome visiting process with overcrowding and long waiting queues in outpatient departments is prevalent and is a prominent experience for outpatients and their families [34]. Therefore, the physical environment and convenience is an important dimension of the OPEQ. Medical expenses of outpatients in China usually cannot be or can only be partially reimbursed by health insurance, which often results in a huge burden for many patients and their families, thus this is also an important dimension in the measurement scale of outpatient experiences in China.
According to Sitzia's research, 134 of 195 articles about patient satisfaction measurement had adopted 5-point Likert scoring method [35]. Garratt et al. [36] concluded that data collected with patient experience questionnaires adopted 5-point Likert scoring method had higher quality and produced better graphics distribution when conducted statistical description. Therefore, OPEQ also adopted 5-point Likert scoring method. Individual responses were scored on a scale of 0 to 100, for 5-point responses, 0 will be assigned for ‘disagree completely’, 25 for ‘disagree’, 50 for ‘fair’, 75 for ‘agree’, 100 for ‘agree completely’. Every participant selected a score based on their visit experiences. The scores represented not only the patients’ evaluation for health services but also the extent to which patient experience could be improved.
Different from other patient experience surveys, such as Hong Kong's HKIEQ, which used the mixed method of telephone and face-to-face interview [10], and HCAHPS in USA, which used mixed method of speech-enabled interactive voice recognition, telephone and a mail-invited web-based mode [37], we adopted only face-to-face interview. The main reason is that outpatients in China always stay in outpatient department of hospital for a short time with no contact information left. Therefore, we have to conduct the survey with the way of face-to-face interview when participants end their visit process. Face-to-face interview often has higher response rate than postal questionnaires or other self-report questionnaires because of the personal contact [38], it can also increase the accuracy of information about patient experiences for those with low literacy. Using this data collection method, we can increase response rate and accuracy of the information, howerver, this OPEQ instrument could be used in self-administered surveys as well,such as hand-out methods that supply a prepaid means of returning a written questionnaire.
We have made great efforts to ensure the authenticity and credibility of the investigation, however, outpatient department of Chinese comprehensive public hospital is the scene of the investigation, so there are some limitations that cannot be avoided. Due to the comparatively short visiting time of outpatients and their frequent movement in outpatient department of Chinese public hospitals, it is difficult to guarantee complete random samples of outpatients and to test Test-retest reliability. However, we have tried to balance the demographic profile of the sample to improve the representativeness with face-to-face interview during the survey. Another limitation is that with the face-to-face interview, the interviewer may intent to select the patients who were more likely to participate the survey and may also potentially influence the response of patients, which will increase the risk of response bias. Despite the assurances given that results would remain confidential, it is possible that some bias was introduced by this method of administration whereby patients perceive that their care may somehow be affected by a negative evaluation. The use of the single mode of administration makes it impossible to determine whether such bias exists. We sampled 600 outpatients who willing to take part in the survey and did not take account of the number of patient who refused to participate, which may increase the response rate.This is also a limitation.
Despite these limitations, OPEQ is still a valid and credible instrument to measure outpatient experiences in Chinese hospitals. OPEQ has good prospects of application in hospital management and health insurance reimbursement. Potential benefits of OPEQ include increased transparency, improved consumer decision-making, and increased incentives for the delivery of high-quality healthcare services.
Conclusion
The OPEQ is the first validated outpatient experience questionnaire in China, which exams outpatient experience of comprehensive public hospital with six dimensions and 28 items. The questionnaire is proven to be reliable and reasonable, it provides an instrument for collecting outpatient experience information in Chinese hospitals. OPEQ also guides us to pay more attention to outpatient experience, to find new ways to ease the doctor–patient relationship in China and to improve the quality of health services from the perspective of patient experience.
Acknowledgements
We thank all interviewers for their data collecting, and all participating hospitals and patients for their supports and valuable opinions.
Funding
This work was supported by a grant from ‘the Fundamental Research Funds for the Central Universities’(2015AC023). The funding body does not play roles in study design, interpretation of data and the manuscript drafting.