- Split View
-
Views
-
Cite
Cite
Joseph R Egger, Kayla Stankevitz, Robert Korom, Philip Angwenyi, Brittney Sullivan, Jun Wang, Sonia Hatfield, Emma Smith, Karishma Popli, Jessica Gross, Evaluating the effects of organizational and educational interventions on adherence to clinical practice guidelines in a low-resource primary-care setting in Kenya, Health Policy and Planning, Volume 32, Issue 6, July 2017, Pages 761–768, https://doi.org/10.1093/heapol/czx004
- Share Icon Share
Abstract
Background Mid-level care providers serve as the backbone of primary care in many parts of sub-Saharan Africa. Despite this, research suggests that the quality and consistency of this care is uneven. This study assessed the degree to which a set of four simple, low-cost interventions could improve adherence to a set of clinical quality measures (CQMs) associated with four common health conditions seen in a resource-constrained primary care setting.
Methods A quasi-experimental, longitudinal study was carried out in three primary care clinics in Nairobi, Kenya from August 2014 to January, 2015. Mid-level clinical officers (COs) at each clinic participated in four interventions aimed at improving CQM adherence. A group of temporary COs acted as a control group. Clinical encounter data were abstracted from eligible medical charts and assessed for CQM adherence. Mixed-effects logistic regression models were then fitted to these data to determine whether adherence to CQMs improved over time, and if this adherence differed by provider type and other characteristics.
Results Adherence to CQMs increased from 41.4% to 77.1% for COs that took part in the intervention, and dropped slightly from 26.5% to 21.8% for temporary COs over the 6-month study period. This difference was statistically different between treatment groups and suggests that environmental interventions alone cannot change behaviour. Adherence also varied significantly by health condition, but did not vary by provider gender, age or clinic site.
Conclusions This study demonstrates the potential for low-tech, low-cost interventions to improve the quality of care delivered by mid-level care providers in resource-constrained settings. Given the widespread utilization of mid-level care providers across sub-Saharan Africa, multicomponent interventions such as this one, that consist of simple educational modules and clinic-based feedback sessions, could lead to substantial improvements in the quality of primary care in these settings.
The consistency of quality primary health care is low in many areas of sub-Saharan Africa, including Kenya.
A simple, low-cost set of interventions aimed at mid-level care providers was shown to be highly effective in rapidly improving this clinical quality.
These findings have important implications, and could be used to significantly improve clinical quality, and thus health outcomes, for conditions commonly seen in primary care settings across sub-Saharan Africa.
Introduction
Access to quality primary health care in Kenya is not universal. Kenyans employed in the formal economy have access to health insurance through the National Hospital Insurance Fund (NHIF), which reimburses both public and private health care providers. However, members of the informal economy, which comprises 80% of the workforce, are not required by law to have health insurance (Xinhuanet 2015). As a result, only about 20% of Kenyans have access to medical coverage (World Bank 2014). Even among covered beneficiaries, service levels frequently fail to meet patient needs (Open Capital Advisors 2012). This inadequate health insurance coverage for the majority of Kenyans has led to the emergence of a health sector with a mixture of public, private and faith-based health care facilities.
Nearly 60% of health services in Kenya are provided through the private sector (Wamai 2009). Those who can afford the out-of-pocket expense often seek care from private medical centres, while many of those who cannot, seek care from low-fee, small, private clinics. While private facilities are perceived to deliver higher quality health services than their public counterparts, likely a result of their improved infrastructure and facilities (Chen et al. 2014), published studies do not support this view. One study that evaluated public and private facilities in Kenya and Ghana found that over 90% of facilities were performing at < 50% when rated against World Health Organization (WHO) clinical practice guidelines for outpatient care (Spreng et al. 2014). But generally, for both private medical centres and small clinics, the quality and consistency of health care is largely unknown in sub-Saharan Africa because it is not being measured. And while much more research is needed, some studies have noted the promise private sector facilities hold to improve access to quality care for low- and middle-income individuals (Patouillard et al. 2007).
In both public and private outpatient clinics in Kenya, services are often delivered by clinical officers (COs), non-physician health care providers who complete three years of clinical training and a 1-year internship before receiving their license. COs emerged to address the shortage of medical officers in the 1960s and are trained today as primary care providers with opportunities for specialization in several areas, including paediatrics, reproductive health, public health and dental health, among other areas (Mbindyo et al. 2013). Many countries in sub-Saharan Africa use non-physician clinicians (Mullan and Frehywot 2007), and several studies have shown that patient satisfaction, trust and perception of quality is similar for non-physician clinicians and medical doctors (Rao et al. 2013). A systematic review on the quality of care delivered by mid-level providers in Africa also found that there was little difference in the care delivered by mid-level care providers and medical doctors in tertiary care settings (Lassi et al. 2013).
Ensuring a high standard of quality for outpatient clinical care in low- and middle-income countries (LMICs) can be costly and complex. In fact, the measurement of clinical quality itself is lacking in many ambulatory care settings in LMICs and well-resourced health systems alike. The use of clinical quality guidelines, such as those endorsed by the WHO or National Quality Forum (NQF), has been advocated to increase quality and consistency of health care (Chinnock et al. 2005; Kredo et al. 2012; Mark 2013). However, there is currently no consensus on the types of interventions that are most effective or lead to a long-term improvement in the uptake and adherence of clinical quality measures (CQMs). Recent research indicates that adherence to international clinical quality guidelines among clinical staff remains low in inpatient settings in sub-Saharan Africa (Mwaniki et al. 2014), and there is scant literature on adherence to best practices for outpatient encounters. Behavior change among clinical providers is multi-dimensional, thus, consistent provider adherence to clinical quality guidelines can be difficult to ensure. Lack of consistent standards leads to great variation in patient care (e.g. diagnosis and treatment) for the same condition between providers and facilities, which affects the quality of outpatient care for low- and middle-income individuals.
To improve the overall quality of care, organizations have invested in numerous approaches. While one study in Malawi found training mid-level providers on integrating primary care guidelines to be useful, they faced challenges with staff turnover and training allowances (Sodhi et al. 2014). Another study found that continued medical education alone did not improve quality of care (Spreng et al. 2014), and a recent study from Tanzania found low adherence to Integrative Management of Childhood Illnesses (IMCI) guidelines among first-line providers (Walter et al. 2009). A systematic review of pay for performance schemes in low- and middle-income countries found that they had little effect in improving the delivery of health interventions (Witter et al. 2012). One study highlighted the importance of creating an enabling work environment in order to improve provider motivation, job satisfaction and work performance of mid-level care providers in low- and middle-income countries (McAuliffe et al. 2009). Other studies support the use of clinical decision support systems to improve provider performance; however, these tools are often electronic and require internet connectivity to operate, which can pose additional implementation challenges in already constrained settings (Main et al. 2010; Ali et al. 2011). Integrated approaches seem to hold more potential for improvement than single interventions. One study advocated establishing competencies for mid-level providers linked to new provider orientation, job descriptions, performance reviews and career promotion to drive a higher consistent standard of quality care (Melnyk et al. 2014). One quality improvement framework that garnered success focused on integrated implementation techniques that included a self-assessment tool, one-on-one mentorship, systems improvements and a resource centre (Li et al. 2015). However, despite this recent literature, there remains a poor understanding of the challenges in implementing these interventions, as well as their effects on clinical quality in low-resource settings, such as Kenya.
This study aimed to measure adherence to clinical practice guidelines by mid-level care providers in a private, ambulatory care setting in Nairobi, Kenya, before and after implementation of a clinical quality assurance intervention. We further sought to assess the perception of this intervention by clinical providers and the key factors responsible for its success. A better understanding of how to educate these providers on the consistent and continual use of clinical quality guidelines to improve patient care could substantially benefit Kenya and other LMICs using mid-level providers.
Methods
A quasi-experimental, longitudinal study was performed from 1 August 2014 to 15 January 2015 (i.e. the study period) to measure the effectiveness of an integrated approach to improve provider performance in the primary care outpatient setting in Nairobi, Kenya. The study was carried out in three Penda Health primary care centres; one in the Kitengela suburb of Nairobi, one in Umoja estate in Nairobi, and one in a small clinic on the campus of Management University of Africa (MUA), also in Nairobi. Penda is a private social enterprise based in Nairobi that owns and operates primary care health centres catering to low- and middle-income clients, and provides comprehensive primary care services leveraging mid-level care providers (i.e. COs). As a condition of employment at Penda Health, all medical providers agree to have Penda monitor their adherence to clinical quality guidelines through the assessment of their performance on CQMs. CQMs are widely used to measure and quantify healthcare processes, outcomes, patient perceptions, and organizational structure and systems that are associated with the ability to provide high-quality health care (National Quality Forum 2016).
Intervention
Between 15 September and 15 November 2014, Penda staff implemented four interventions aimed at improving the adherence to clinical quality guidelines related to four commonly presented health conditions at Penda clinics: urinary tract infection; vaginal discharge; tonsillitis and childhood diarrhoea. These interventions included:
Online educational modules—assigned to each provider to assist in the explanation of the clinical guidelines related to each CQM. Clinical guidelines were uploaded into Shift Planning software (Shift Planning, San Francisco, CA), and each clinician was given a login profile to access and review the content. Emails and text messages were sent reminding COs to review the guidelines.
Continuing education sessions—a 2-h educational training session, per health condition, that each Penda provider was required to facilitate and participate in to supplement the online educational content.
Monthly feedback meetings—Clinicians were emailed monthly with results of their performance on various metrics. The e-mail served as a precursor to one-on-one discussion between the Penda Medical Director and each CO.
Systemic environmental changes—including the provision of materials, such as posters, signs and changes in patient documentation forms intended to remind providers to follow clinical quality guidelines.
The four interventions were implemented concurrently; however, the rollout was staggered by health condition. Specifically, CQMs related to urinary tract infection and tonsillitis were implemented beginning 15 September, vaginal discharge beginning 15 October and childhood diarrhoea beginning 15 November.
Evaluation and data
For this study, adherence was defined as successfully performing all CQMs related to each of the four medical conditions (Appendix 1). Effectiveness of the four interventions to improve provider adherence was measured through the development and use of 17 CQMs (Appendix 1). The quasi-experimental, longitudinal study utilized a group of temporary COs (i.e. locum COs), employed by Penda on an ad hoc basis, to act as the study’s control group. These locum COs substitute for Penda staff COs when they are off, sick, on holiday or otherwise unable to work. The locum COs acted as a reasonable control group because they are similar to Penda COs in many characteristics, including demographic characteristics and medical training, but were not directly exposed to three of the four study interventions. Locum COs were exposed to systemic changes at each clinic, an environmental factor that had the potential to influence all COs. The primary study hypothesis, therefore, was that the probability of overall CQM adherence would increase over the study period, but that this increase would be significantly greater among Penda COs, as compared to locums.
The CQMs utilized for this study were developed using clinical practice guidelines from the WHO and the Ministry of Health, Kenya, as a basic framework. Comparisons were made to the latest practice guidelines in the USA (www.guidelines.gov) and by the use of other resources, such as UpToDate™, guidelines from the Infectious Disease Society of America (IDSA) and American Academy of Pediatrics (AAP). Clinical practices applicable to the Kenyan setting were then incorporated to create clinical guidelines relevant to Penda clinics with input from expert clinicians with experience in these settings.
Clinical encounter data were abstracted monthly during the study period (1 August 2014–15 January 2015) from relevant patient medical charts by senior Penda staff using an abstraction template in MS Excel. All encounters, from each of the three Penda clinics, that met patient eligibility criteria, were included in this study. Patients were eligible for the study if they presented to a Penda clinic during this time period with a chief complaint related to one of the four study conditions (urinary tract infection; vaginal discharge; tonsillitis and childhood diarrhoea). Clinical encounter data were abstracted beginning 1 month prior to the intervention (i.e. August 2014) to establish baseline CQM values. These data included patient age and sex, diagnosis, medical centre site, and a determination of whether the clinician adhered to each relevant CQM, coded as a 0 (no) or 1 (yes). Response coding for all 17 CQMs can be found in Appendix 1. Data were collected on all Penda and locum CO providers working at a Penda clinic during the study period, including their age, gender, training location, internship sites, years since graduation and employment history. Additionally, information from routine unstructured in-person interviews with Penda COs related to their perceptions of the interventions was also collected at the time of each interview. Finally, unstructured in-person interviews with members of the Penda leadership were conducted to assess the implementation and effects of the interventions on Penda’s organizational processes and provider behaviour. Ethical approval for this study was obtained from institutional review boards at Duke University and AMREF Health Africa.
Analysis
The primary unit of analysis was CQM measure, denoted as a 0 (non-adherence) or 1 (adherence). The number of individual CQMs ranged from 3 to 5 depending on the condition. All patient encounters related to one of the four health conditions were eligible for initial study inclusion. However, encounters were deleted from the final analysis if the encounter did not occur during the study period, or if the provider was not a CO (e.g. if they were a MD). Clinical encounter inclusion criteria, by medical condition, can be found in Appendix 2.
To estimate the effect of the intervention, a mixed-effects logistic regression model was fitted to the binary dependent variable CQM adherence. Fixed effects for time (linear, days since intervention start date), CO type (Penda or locum), health condition (categorical) and medical centre site (categorical) were included as independent variables, adjusting for the possible CO-level confounders, including age (linear) and gender (categorical). Since, depending on health outcome, there were up to five CQMs related to each clinical encounter, a random intercept for clinical encounter was included to account for correlation in the response. A second random intercept for provider identity was also included to account for correlated responses within providers over time. This resulted in a 3-level model with measured adherence nested within clinical encounter, nested within specific CO provider. We further hypothesized that the probability of CQM adherence may be a function of years of CO experience. While we did not have this variable measured for all COs, we included this variable as a possible confounder in pre-specified models for the COs with this measured variable and compared these results to results unadjusted for years of CO experience. Finally, to formally test whether CQM adherence varied over time by CO type, a parameter for the multiplicative interaction between CO type and time was fitted in the same model. This parameter also adjusted for any difference in CQM adherence between Penda and locum providers at baseline.
To better understand the pattern in the change in adherence over time, a non-parametric Lowess curve was fitted to a daily roll-up of the proportion of eligible CQMs to which the CO adhered to recommended guidelines. Various bandwidths, in days, indicating the local weights were fitted in separate models. All analyses were performed using Stata version 13 software (Stata Corp., College Station, TX). Qualitative data collected from interviews of both Penda leadership and clinical staff were reviewed for common themes; however, no formal coding or analysis of unstructured data was performed.
Results
The final dataset included 2564 records (i.e. unique CQM values), and 704 clinical encounters representing 645 unique patients seen at Penda medical centres during the study period. The distribution of age, gender and diagnosis of patients was similar between locums and Penda COs (Table 1). The total number of COs during the study period was 20; seven were Penda employees and 13 were locums. The mean age of the providers was 30.2 years and 28.5 years for locum and Penda COs, respectively. The proportion of COs that were female was also similar across provider type (locum: 47.2%; Penda: 48.6%). Data for one provider were excluded because s/he was an MD. The minimum number of encounters overseen by a CO was one, the maximum was 134 (mean: 37.0; SD: 44.0).
CO provider characteristic . | Locum CO . | Penda CO . |
---|---|---|
(n=13) . | (n=7) . | |
Age, mean (SD) | 22.7 (12.6) | 21.2 (14.0) |
Years since degree conferred, mean (SD) | 5.5a (0.9) | 5.1 (1.3) |
Gender, (% female) | 79.4% | 78.0% |
Diagnoses, N (%) | ||
UTI, | 352 (40%) | 697 (41%) |
Childhood diarrhoea | 96 (11%) | 267 (16%) |
Vaginal discharge | 229 (26%) | 265 (16%) |
Tonsillitis | 203 (23%) | 455 (27%) |
Total encounters (N) | 880 | 1684 |
CO provider characteristic . | Locum CO . | Penda CO . |
---|---|---|
(n=13) . | (n=7) . | |
Age, mean (SD) | 22.7 (12.6) | 21.2 (14.0) |
Years since degree conferred, mean (SD) | 5.5a (0.9) | 5.1 (1.3) |
Gender, (% female) | 79.4% | 78.0% |
Diagnoses, N (%) | ||
UTI, | 352 (40%) | 697 (41%) |
Childhood diarrhoea | 96 (11%) | 267 (16%) |
Vaginal discharge | 229 (26%) | 265 (16%) |
Tonsillitis | 203 (23%) | 455 (27%) |
Total encounters (N) | 880 | 1684 |
One missing value, n = 12.
CO provider characteristic . | Locum CO . | Penda CO . |
---|---|---|
(n=13) . | (n=7) . | |
Age, mean (SD) | 22.7 (12.6) | 21.2 (14.0) |
Years since degree conferred, mean (SD) | 5.5a (0.9) | 5.1 (1.3) |
Gender, (% female) | 79.4% | 78.0% |
Diagnoses, N (%) | ||
UTI, | 352 (40%) | 697 (41%) |
Childhood diarrhoea | 96 (11%) | 267 (16%) |
Vaginal discharge | 229 (26%) | 265 (16%) |
Tonsillitis | 203 (23%) | 455 (27%) |
Total encounters (N) | 880 | 1684 |
CO provider characteristic . | Locum CO . | Penda CO . |
---|---|---|
(n=13) . | (n=7) . | |
Age, mean (SD) | 22.7 (12.6) | 21.2 (14.0) |
Years since degree conferred, mean (SD) | 5.5a (0.9) | 5.1 (1.3) |
Gender, (% female) | 79.4% | 78.0% |
Diagnoses, N (%) | ||
UTI, | 352 (40%) | 697 (41%) |
Childhood diarrhoea | 96 (11%) | 267 (16%) |
Vaginal discharge | 229 (26%) | 265 (16%) |
Tonsillitis | 203 (23%) | 455 (27%) |
Total encounters (N) | 880 | 1684 |
One missing value, n = 12.
Adherence to CQMs at the beginning of the study was 41.4% for Penda COs and 26.5% for locum COs (Table 2). Over the 6-month study, adherence for Penda COs increased to 77.1% ending in January, 2015, but decreased slightly for locums (21.8%). Adherence also varied significantly by health condition. Furthermore, because the rollout of CQM guidelines was staggered, we did not expect to see an improvement in adherence to each health condition until after initiation of the rollout. Results were largely consistent with this. For example, significant improvement in adherence to vaginal discharge and childhood diarrhoea CQMs did not occur until October and November, the months in which these CQM interventions were rolled out to Penda COs, respectively. Finally, we did not see significant variation in CQM adherence by either patient gender or medical centre site (Table 3).
Month . | Total (N) . | CQM adherence, % (N) . | |
---|---|---|---|
. | . | Locum CO . | Penda CO . |
(n=13) . | (n=7) . | ||
August | 356 | 26.5 (204) | 41.4 (152) |
September | 667 | 33.5 (242) | 53.9 (425) |
October | 540 | 25.3 (170) | 66.5 (370) |
November | 535 | 47.2 (89) | 73.1 (446) |
December | 315 | 26.7 (120) | 73.9 (195) |
January | 151 | 21.8 (55) | 77.1 (96) |
Total | 2564 | 30.0 | 64.3 |
Month . | Total (N) . | CQM adherence, % (N) . | |
---|---|---|---|
. | . | Locum CO . | Penda CO . |
(n=13) . | (n=7) . | ||
August | 356 | 26.5 (204) | 41.4 (152) |
September | 667 | 33.5 (242) | 53.9 (425) |
October | 540 | 25.3 (170) | 66.5 (370) |
November | 535 | 47.2 (89) | 73.1 (446) |
December | 315 | 26.7 (120) | 73.9 (195) |
January | 151 | 21.8 (55) | 77.1 (96) |
Total | 2564 | 30.0 | 64.3 |
Note: Data represent a total of 2564 CQM measures during 702 clinical encounters, by 20 clinical officers.
Month . | Total (N) . | CQM adherence, % (N) . | |
---|---|---|---|
. | . | Locum CO . | Penda CO . |
(n=13) . | (n=7) . | ||
August | 356 | 26.5 (204) | 41.4 (152) |
September | 667 | 33.5 (242) | 53.9 (425) |
October | 540 | 25.3 (170) | 66.5 (370) |
November | 535 | 47.2 (89) | 73.1 (446) |
December | 315 | 26.7 (120) | 73.9 (195) |
January | 151 | 21.8 (55) | 77.1 (96) |
Total | 2564 | 30.0 | 64.3 |
Month . | Total (N) . | CQM adherence, % (N) . | |
---|---|---|---|
. | . | Locum CO . | Penda CO . |
(n=13) . | (n=7) . | ||
August | 356 | 26.5 (204) | 41.4 (152) |
September | 667 | 33.5 (242) | 53.9 (425) |
October | 540 | 25.3 (170) | 66.5 (370) |
November | 535 | 47.2 (89) | 73.1 (446) |
December | 315 | 26.7 (120) | 73.9 (195) |
January | 151 | 21.8 (55) | 77.1 (96) |
Total | 2564 | 30.0 | 64.3 |
Note: Data represent a total of 2564 CQM measures during 702 clinical encounters, by 20 clinical officers.
Variable . | CQM non-adherence . | CQM adherence . | Total (N) . | P valuea . |
---|---|---|---|---|
Total | 1218 (47.5) | 1346 (52.5) | 2564 | |
Patient gender | 0.071 | |||
Male | 281 (50.9) | 271 (49.1) | 552 | |
Female | 937 (46.6) | 1075 (53.4) | 2012 | |
Penda centre | 0.157 | |||
Kitengela | 772 (48.9) | 806 (51.1) | 1,578 | |
Umoja | 440 (45.1) | 535 (54.9) | 975 | |
MUA | 6 (54.6) | 5 (45.5) | 11 | |
Health condition | < 0.001 | |||
UTI | 449 (42.8) | 600 (57.2) | 1049 | |
Childhood diarrhoea | 218 (60.1) | 145 (39.9) | 363 | |
Vaginal discharge | 268 (54.3) | 226 (45.8) | 494 | |
Tonsillitis | 283 (43.0) | 375 (57.0) | 658 | |
Provider type | < 0.001 | |||
Penda CO | 602 (35.7) | 1082 (64.3) | 1684 | |
Locum CO | 616 (70.0) | 264 (30.0) | 880 |
Variable . | CQM non-adherence . | CQM adherence . | Total (N) . | P valuea . |
---|---|---|---|---|
Total | 1218 (47.5) | 1346 (52.5) | 2564 | |
Patient gender | 0.071 | |||
Male | 281 (50.9) | 271 (49.1) | 552 | |
Female | 937 (46.6) | 1075 (53.4) | 2012 | |
Penda centre | 0.157 | |||
Kitengela | 772 (48.9) | 806 (51.1) | 1,578 | |
Umoja | 440 (45.1) | 535 (54.9) | 975 | |
MUA | 6 (54.6) | 5 (45.5) | 11 | |
Health condition | < 0.001 | |||
UTI | 449 (42.8) | 600 (57.2) | 1049 | |
Childhood diarrhoea | 218 (60.1) | 145 (39.9) | 363 | |
Vaginal discharge | 268 (54.3) | 226 (45.8) | 494 | |
Tonsillitis | 283 (43.0) | 375 (57.0) | 658 | |
Provider type | < 0.001 | |||
Penda CO | 602 (35.7) | 1082 (64.3) | 1684 | |
Locum CO | 616 (70.0) | 264 (30.0) | 880 |
All P values are based on a Chi-squared test for trend. P values in bold are statistically significant at the 0.05 level.
Variable . | CQM non-adherence . | CQM adherence . | Total (N) . | P valuea . |
---|---|---|---|---|
Total | 1218 (47.5) | 1346 (52.5) | 2564 | |
Patient gender | 0.071 | |||
Male | 281 (50.9) | 271 (49.1) | 552 | |
Female | 937 (46.6) | 1075 (53.4) | 2012 | |
Penda centre | 0.157 | |||
Kitengela | 772 (48.9) | 806 (51.1) | 1,578 | |
Umoja | 440 (45.1) | 535 (54.9) | 975 | |
MUA | 6 (54.6) | 5 (45.5) | 11 | |
Health condition | < 0.001 | |||
UTI | 449 (42.8) | 600 (57.2) | 1049 | |
Childhood diarrhoea | 218 (60.1) | 145 (39.9) | 363 | |
Vaginal discharge | 268 (54.3) | 226 (45.8) | 494 | |
Tonsillitis | 283 (43.0) | 375 (57.0) | 658 | |
Provider type | < 0.001 | |||
Penda CO | 602 (35.7) | 1082 (64.3) | 1684 | |
Locum CO | 616 (70.0) | 264 (30.0) | 880 |
Variable . | CQM non-adherence . | CQM adherence . | Total (N) . | P valuea . |
---|---|---|---|---|
Total | 1218 (47.5) | 1346 (52.5) | 2564 | |
Patient gender | 0.071 | |||
Male | 281 (50.9) | 271 (49.1) | 552 | |
Female | 937 (46.6) | 1075 (53.4) | 2012 | |
Penda centre | 0.157 | |||
Kitengela | 772 (48.9) | 806 (51.1) | 1,578 | |
Umoja | 440 (45.1) | 535 (54.9) | 975 | |
MUA | 6 (54.6) | 5 (45.5) | 11 | |
Health condition | < 0.001 | |||
UTI | 449 (42.8) | 600 (57.2) | 1049 | |
Childhood diarrhoea | 218 (60.1) | 145 (39.9) | 363 | |
Vaginal discharge | 268 (54.3) | 226 (45.8) | 494 | |
Tonsillitis | 283 (43.0) | 375 (57.0) | 658 | |
Provider type | < 0.001 | |||
Penda CO | 602 (35.7) | 1082 (64.3) | 1684 | |
Locum CO | 616 (70.0) | 264 (30.0) | 880 |
All P values are based on a Chi-squared test for trend. P values in bold are statistically significant at the 0.05 level.
Results of multivariable mixed effects regression indicate that after adjustment for health condition, and accounting for correlation in the response by provider and encounter, the odds of adherence to an individual CQM significantly increased over the 6-month study period for Penda COs (OR, for a one day change over the study period: 1.013; 95% CI: 1.008, 1.018), but did not change for locum COs (OR: 0.999; 95% CI: 0.996, 1.004). These results were very similar to a model adjusting for the effect of years since CO degree conferred (data not shown).
Results of fitting a non-parametric Lowess curve through the daily roll-up of the data, stratified by CO type, indicates that CQM adherence increased steadily for roughly the first 2 months of the study period before levelling off (Figure 1). The Lowess curve also indicates a modest increase in adherence for locum providers over the first few months, but this increase was not sustained and appears to decrease slightly during the final month of the study.
Adherence rates also varied significantly by health condition and by CQM within a health condition (Table 3 and Figure 2), with urinary tract infections having the highest average CQM adherence rates over the 6-month period (44.6%) and childhood diarrhoea averaged the lowest rates at 10.8% (P< 0.001). This result remained in a multivariable regression model after adjustment for the linear effect of time to account for the staggered rollout of CQM guidelines by health condition. In separate regression models, neither CO age, gender nor years of clinical experience were found to be significantly associated with the odds of CQM adherence after adjusting for time (data not shown).
Penda leadership reported the intervention was well received by COs across their facilities. Reports indicated that COs appreciated the opportunity to advance their clinical knowledge, and aspired to provide evidence-based patient care. Providers noted the increased access to clinical mentorship, guidelines and standards promoted by this intervention. Providers were receptive to monthly feedback sessions, where they fielded clinical questions and learned how their performance compared to their fellow Penda COs, as well as Penda’s locum providers. Providers noted the performance feedback was useful to create awareness of areas for improvement and review specific clinical cases. Providers utilized the one-on-one sessions to review the educational materials and prepare for monthly topical continuing education sessions they facilitated for other non-Penda COs from the community. Providers noted their pride in leading continuing education sessions and being leaders in their community, and their gratitude for being able to obtain continuing professional development (CPD) credits as a part of their job, which they could apply towards the renewal of their clinical licenses.
Discussion
The results of our study suggest that relatively simple interventions can dramatically improve clinical quality for low- and middle-income patients in outpatient primary care settings. Results also indicate that the effect of the study’s four interventions was similar across Penda CO provider age and gender, as well as by clinical site. The Hawthorne Effect, which states that individual performance improves simply through an awareness that it is being measured, may also play a significant role in the improvements seen in Penda staff, suggesting that measurement of quality metrics itself may be a key component in a multi-faceted approach to improving performance on CQMs. Furthermore, if interventions can be performed by existing and experienced clinical staff, as is the case with Penda, then they should remain very low cost and highly scalable in their implementation, as these clinics will not need to hire additional staff for this role.
Our results are consistent with recently published reviews of health care worker performance in resource-constrained settings. A 2015 systematic review by the Centers for Disease Control and Prevention, found that clinical interventions, including both supervision, as well as high-intensity training, were effective in both low- and middle-income countries (Rowe 2015). Furthermore, the study found that interventions focused solely on printed materials targeted to health care workers were ineffective in improving performance. A 2013 WHO review of 900 publications on adherence to guidelines related to medicine use came to a similar conclusion; specifically, that multicomponent interventions were more effective than single component ones in improving adherence to clinical guidelines (Holloway et al. 2013).
The short time for improvement in clinical quality adherence to occur across these select health conditions also suggests that uptake of these interventions is rapid and was sustained at least in the short term. Furthermore, there was little evidence that locum CO performance varied significantly at any time during the study period. The slight decrease in CQM adherence by locums at the end of the study may be real, but may also simply be due to random variability in adherence over time. The fact that we did not see an improvement in locum CO performance suggests that environmental interventions, to which the locums were exposed, were not enough to influence CQM adherence and that more targeted provider engagement is necessary.
The observed increase in odds of CQM adherence among Penda providers is a pooled average effect across the four interventions. While this study was not able to separately estimate the effects of each of the interventions, interviews with Penda clinical staff, present in the clinics during the study period, suggest that, of the four interventions, the one-on-one feedback sessions with COs to review their performance, and benchmark against average CO performance, may have had the greatest impact in driving behaviour change and improving CQM adherence. According to Penda staff, these feedback sessions were not only a way to review and reflect on clinical performance, it was also a way to engender Penda’s cultural values, create social cohesion and improve CO self-esteem and pride in their work. With high CO turnover across primary care settings in Kenya, developing this loyalty is not only important for financial viability for a small private healthcare enterprise, but it may also have the added benefit of improving quality by retaining and training staff. Notably, this finding is consistent with previous studies showing that managerial approaches, such as supervision and feedback, are effective interventions to improve clinical performance through improving health workers' job satisfaction and increasing motivation (Rowe et al. 2005). With additional data, future analyses may focus on modelling the acceleration in CQM adherence by provider to more precisely estimate how quickly COs uptake these interventions to improve adherence, as well as to study provider- and facility-level factors that may influence this acceleration.
Adherence to CQM varied by health condition, especially for the conditions childhood diarrhoea and vaginal discharge, even after accounting for the staggered rollout in CQM guidelines. This finding is not surprising and can likely be explained by the total number of CQMs (i.e. the higher the number of CQMs means a higher burden to adherence) and by the number of criteria within each CQM. For example, in the vital sign CQM for childhood diarrhoea, providers were required to record six vital measurements in order to score as compliant. Providers often documented several vital measurements, but never all six, pulling down their performance scores for childhood diarrhoea. Similarly, CQM 6 for vaginal discharge required providers to document four items from a patient’s history, including the number of sexual partners, previous sexually transmitted infections, condom usage and pregnancy status in order to score as compliant.
Our study was not without limitations. CQMs were scored based on what was documented by the provider. Since Penda providers may have had more experience with Penda medical charts than locums, this documentation could have been differential by provider status, thus leading to information bias. However, the medical chart information used to measure CQMs was abstracted by trained senior clinical staff at Penda and was relatively simple to interpret. Therefore, we do not think that this would have led to significant differential misclassification of the outcome. Furthermore, the sample of COs in our study was relatively small (n = 20). While this may have affected the precision of our measured intervention effect, it should not affect the internal validity of our findings. It does, however, make our findings harder to generalize outside of this specific clinical context. Since this was an observational study, clinical officers were not randomized to the treatment group, increasing our risk of biases induced by selection and confounding. The use of locum COs as a control group to estimate the intervention effect was reasonable; however, in terms of adherence rates, locums did not end up being similar to Penda staff at baseline. To account for this potential bias, we adjusted estimates for baseline CQMs rates, as well as other measured covariates. Furthermore, the inference we can make to the specific interventions that were associated with changes in adherence is limited by the observational nature of the study and the fact that all COs were exposed to systemic changes at each clinic. We envision future work will include a cluster-randomized factorial experiment to better isolate the intervention effects and minimize potential spillover effects. Correlation of CQM performance with patient-centred clinical outcomes will be important to establish the link between process measures and outcomes. Future work will also include explicitly studying the implementation of the interventions and how this implementation may vary by clinical setting, health outcome, private/public status, staffing mix and other organizational characteristics.
Conclusion
This study demonstrates the potential for low-tech, low-cost interventions to improve the quality of care delivered by mid-level care providers in resource-constrained settings. Utilization of simple educational modules and clinic-based feedback sessions allowed providers to remain on-site to see patients, facilitating continuing education and performance feedback that was integrated into their work day. Monthly continuing education sessions, taught by Penda COs, cemented their newly acquired medical knowledge and facilitated the dissemination of best practices to other mid-level care providers practicing in the community. Given the widespread utilization of mid-level care providers in Kenya and across sub-Saharan Africa, interventions, such as this one, that increase and evaluate adherence to clinical quality guidelines, could lead to substantial improvements in the quality of primary care.
Acknowledgements
The authors would like to thank the staff at Penda Health for their generosity and cooperation in carrying out this study. This research was supported by Duke University’s Bass Connections program and the Social Entrepreneurship Accelerator at Duke (SEAD, USAID cooperative agreement AID-OAA-A-13-0004).
Funding
This research was supported with funding from the Duke University Bass Connections program, as well as the United States Agency for International Development under cooperative agreement number AID-OAA-A-13-00004.
Conflict of interest statement. None declared.