New tools are needed for malaria control, and recent improvements in malaria surveillance have opened the possibility of transforming surveillance into a core intervention. Implementing this strategy can be challenging in moderate to high transmission settings. However, there is a wealth of practical experience among national malaria control programs and partners working to improve and use malaria surveillance data to guide programming. Granular and timely data are critical to understanding geographic heterogeneity, appropriately defining and targeting interventions packages, and enabling timely decision-making at the operational level. Resources to be targeted based on surveillance data include vector control, case management commodities, outbreak responses, quality improvement interventions, and human resources, including community health workers, as they contribute to a more refined granularity of the surveillance system. Effectively transforming malaria surveillance into a core intervention will require strong global and national leadership, empowerment of subnational and local leaders, collaboration among development partners, and global coordination. Ensuring that national health systems include community health work can contribute to a successful transformation. It will require a strong supply chain to ensure that all suspected cases can be diagnosed and data reporting tools including appropriate electronic devices to provide timely data. Regular data quality audits, decentralized implementation, supportive supervision, data-informed decision-making processes, and harnessing technology for data analysis and visualization are needed to improve the capacity for data-driven decision-making at all levels. Finally, resources must be available to respond programmatically to these decisions.

BACKGROUND: Although supervision is a ubiquitous approach to support health programs and improve health care provider (HCP) performance in low- and middle-income countries (LMICs), quantitative evidence of its effects is unclear. The objectives of this study are to describe the effect of supervision strategies on HCP practices in LMICs and to identify attributes associated with greater effectiveness of routine supervision. METHODS: We performed a secondary analysis of data on HCP practice outcomes (e.g., percentage of patients correctly treated) from a systematic review on improving HCP performance. The review included controlled trials and interrupted time series studies. We described distributions of effect sizes (defined as percentage-point [%-point] changes) for each supervision strategy. To identify attributes associated with supervision effectiveness, we performed random-effects linear regression modeling and examined studies that directly compared different approaches of routine supervision. RESULTS: We analyzed data from 81 studies from 36 countries. For professional HCPs, such as nurses and physicians, primarily working at health facilities, routine supervision (median improvement when compared to controls: 10.7%-points; IQR: 9.9, 27.9) had similar effects on HCP practices as audit with feedback (median improvement: 10.1%-points; IQR: 6.2, 23.7). Two attributes were associated with greater mean effectiveness of routine supervision (p < 0.10): supervisors received supervision (by 8.8-11.5%-points), and supervisors participated in problem-solving with HCPs (by 14.2-20.8%-points). Training for supervisors and use of a checklist during supervision visits were not associated with effectiveness. The effects of supervision frequency (i.e., number of visits per year) and dose (i.e., the number of supervision visits during a study) were unclear. For lay HCPs, the effect of routine supervision was difficult to characterize because few studies existed, and effectiveness in those studies varied considerably. Evidence quality for all findings was low primarily because many studies had a high risk of bias. CONCLUSIONS: Although evidence is limited, to promote more effective supervision, our study supports supervising supervisors and having supervisors engage in problem-solving with HCPs. Supervision's integral role in health systems in LMICs justifies a more deliberate research agenda to identify how to deliver supervision to optimize its effect on HCP practices.

BACKGROUND: Demand for high-quality surveillance data for malaria, and other diseases, is greater than ever before. In Uganda, the primary source of malaria surveillance data is the Health Management Information System (HMIS). However, HMIS data may be incomplete, inaccurate or delayed. Collaborative improvement (CI) is a quality improvement intervention developed in high-income countries, which has been advocated for low-resource settings. In Kayunga, Uganda, a pilot study of CI was conducted in five public health centres, documenting a positive effect on the quality of HMIS and malaria surveillance data. A qualitative evaluation was conducted concurrently to investigate the mechanisms of effect and unintended consequences of the intervention, aiming to inform future implementation of CI. METHODS: The study intervention targeted health workers, including brief in-service training, plus CI with 'plan-do-study-act' (PDSA) cycles emphasizing self-reflection and group action, periodic learning sessions, and coaching from a CI mentor. Health workers collected data on standard HMIS out-patient registers. The qualitative evaluation (July 2015 to September 2016) included ethnographic observations at each health centre (over 12-14 weeks), in-depth interviews with health workers and stakeholders (n = 20), and focus group discussions with health workers (n = 6). RESULTS: The results suggest that the intervention did facilitate improvement in data quality, but through unexpected mechanisms. The CI intervention was implemented as planned, but the PDSA cycles were driven largely by the CI mentor, not the health workers. In this context, characterized by a rigid hierarchy within the health system of limited culture of self-reflection and inadequate training and supervision, CI became an effective form of high-quality training with frequent supervisory visits. Health workers appeared motivated to improve data collection habits by their loyalty to the CI mentor and the potential for economic benefits, rather than a desire for self-improvement. CONCLUSIONS: CI is a promising method of quality improvement and could have a positive impact on malaria surveillance data. However, successful scale-up of CI in similar settings may require deployment of highly skilled mentors. Further research, focusing on the effectiveness of 'real world' mentors using robust study designs, will be required to determine whether CI can be translated effectively and sustainably to low-resource settings.

BACKGROUND: Surveillance data are essential for malaria control, but quality is often poor. The aim of the study was to evaluate the effectiveness of the novel combination of training plus an innovative quality improvement method-collaborative improvement (CI)-on the quality of malaria surveillance data in Uganda. METHODS: The intervention (training plus CI, or TCI), including brief in-service training and CI, was delivered in 5 health facilities (HFs) in Kayunga District from November 2015 to August 2016. HF teams monitored data quality, conducted plan-do-study-act cycles to test changes, attended periodic learning sessions, and received CI coaching. An independent evaluation was conducted to assess data completeness, accuracy, and timeliness. Using an interrupted time series design without a separate control group, data were abstracted from 156,707 outpatient department (OPD) records, laboratory registers, and aggregated monthly reports (MR) for 4 time periods: baseline-12 months, TCI scale-up-5 months; CI implementation-9 months; post-intervention-4 months. Monthly OPD register completeness was measured as the proportion of patient records with a malaria diagnosis with: (1) all data fields completed, and (2) all clinically-relevant fields completed. Accuracy was the relative difference between: (1) number of monthly malaria patients reported in OPD register versus MR, and (2) proportion of positive malaria tests reported in the laboratory register versus MR. Data were analysed with segmented linear regression modelling. RESULTS: Data completeness increased substantially following TCI. Compared to baseline, all-field completeness increased by 60.1%-points (95% confidence interval [CI]: 46.9-73.2%) at mid-point, and clinically-relevant completeness increased by 61.6%-points (95% CI: 56.6-66.7%). A relative - 57.4%-point (95% confidence interval: - 105.5, - 9.3%) change, indicating an improvement in accuracy of malaria test positivity reporting, but no effect on data accuracy for monthly malaria patients, were observed. Cost per additional malaria patient, for whom complete clinically-relevant data were recorded in the OPD register, was $3.53 (95% confidence interval: $3.03, $4.15). CONCLUSIONS: TCI improved malaria surveillance completeness considerably, with limited impact on accuracy. Although these results are promising, the intervention's effectiveness should be evaluated in more HFs, with longer follow-up, ideally in a randomized trial, before recommending CI for wide-scale use.

BACKGROUND: A recent systematic review evaluated the effectiveness of strategies to improve healthcare provider (HCP) performance in low-income and middle-income countries. The review identified strategies with varying effects, including in-service training, supervision and group problem-solving. However, whether their effectiveness changed over time remained unclear. In particular, understanding whether effects decay over time is crucial to improve sustainability. METHODS: We conducted a secondary analysis of data from the aforementioned review to explore associations between time and effectiveness. We calculated effect sizes (defined as percentage-point (%-point) changes) for HCP practice outcomes (eg, percentage of patients correctly treated) at each follow-up time point after the strategy was implemented. We estimated the association between time and effectiveness using random-intercept linear regression models with time-specific effect sizes clustered within studies and adjusted for baseline performance. RESULTS: The primary analysis included 37 studies, and a sensitivity analysis included 77 additional studies. For training, every additional month of follow-up was associated with a 0.19 %-point decrease in effectiveness (95% CI: -0.36 to -0.03). For training combined with supervision, every additional month was associated with a 0.40 %-point decrease in effectiveness (95% CI: -0.68 to -0.12). Time trend results for supervision were inconclusive. For group problem-solving alone, time was positively associated with effectiveness, with a 0.50 %-point increase in effect per month (95% CI: 0.37 to 0.64). Group problem-solving combined with training was associated with large improvements, and its effect was not associated with time. CONCLUSIONS: Time trends in the effectiveness of different strategies to improve HCP practices vary among strategies. Programmes relying solely on in-service training might need periodical refresher training or, better still, consider combining training with group problem-solving. Although more high-quality research is needed, these results, which are important for decision-makers as they choose which strategies to use, underscore the utility of studies with multiple post-implementation measurements so sustainability of the impact on HCP practices can be assessed.

INTRODUCTION: In low/middle-income countries (LMICs), training is often used to improve healthcare provider (HCP) performance. However, important questions remain about how well training works and the best ways to design training strategies. The objective of this study is to characterise the effectiveness of training strategies to improve HCP practices in LMICs and identify attributes associated with training effectiveness. METHODS: We performed a secondary analysis of data from a systematic review on improving HCP performance. The review included controlled trials and interrupted time series, and outcomes measuring HCP practices (eg, percentage of patients correctly treated). Distributions of effect sizes (defined as percentage-point (%-point) changes) were described for each training strategy. To identify effective training attributes, we examined studies that directly compared training approaches and performed random-effects linear regression modelling. RESULTS: We analysed data from 199 studies from 51 countries. For outcomes expressed as percentages, educational outreach visits (median effect size when compared with controls: 9.9 %-points; IQR: 4.3-20.6) tended to be somewhat more effective than in-service training (median: 7.3 %-points; IQR: 3.6-17.4), which seemed more effective than peer-to-peer training (4.0 %-points) and self-study (by 2.0-9.3 %-points). Mean effectiveness was greater (by 6.0-10.4 %-points) for training that incorporated clinical practice and training at HCPs' work site. Attributes with little or no effect were: training with computers, interactive methods or over multiple sessions; training duration; number of educational methods; distance training; trainers with pedagogical training and topic complexity. For lay HCPs, in-service training had no measurable effect. Evidence quality for all findings was low. CONCLUSIONS: Although additional research is needed, by characterising the effectiveness of training strategies and identifying attributes of effective training, decision-makers in LMICs can improve how these strategies are selected and implemented.

BACKGROUND: Quality improvement collaboratives (QICs) have been used to improve health care for decades. Evidence on QIC effectiveness has been reported, but systematic reviews to date have little information from low- and middle-income countries (LMICs). OBJECTIVE: To assess the effectiveness of QICs in LMICs. METHODS: We conducted a systematic review following Cochrane methods, the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach for quality of evidence grading, and the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement for reporting. We searched published and unpublished studies between 1969 and March 2019 from LMICs. We included papers that compared usual practice with QICs alone or combined with other interventions. Pairs of reviewers independently selected and assessed the risk of bias and extracted data of included studies. To estimate strategy effectiveness from a single study comparison, we used the median effect size (MES) in the comparison for outcomes in the same outcome group. The primary analysis evaluated each strategy group with a weighted median and interquartile range (IQR) of MES values. In secondary analyses, standard random-effects meta-analysis was used to estimate the weighted mean MES and 95% confidence interval (CI) of the mean MES of each strategy group. This review is registered with PROSPERO (International Prospective Register of Systematic Reviews): CRD42017078108. RESULTS: Twenty-nine studies were included; most (21/29, 72.4%) were interrupted time series studies. Evidence quality was generally low to very low. Among studies involving health facility-based health care providers (HCPs), for "QIC only", effectiveness varied widely across outcome groups and tended to have little effect for patient health outcomes (median MES less than 2 percentage points for percentage and continuous outcomes). For "QIC plus training", effectiveness might be very high for patient health outcomes (for continuous outcomes, median MES 111.6 percentage points, range: 96.0 to 127.1) and HCP practice outcomes (median MES 52.4 to 63.4 percentage points for continuous and percentage outcomes, respectively). The only study of lay HCPs, which used "QIC plus training", showed no effect on patient care-seeking behaviors (MES -0.9 percentage points), moderate effects on non-care-seeking patient behaviors (MES 18.7 percentage points), and very large effects on HCP practice outcomes (MES 50.4 percentage points). CONCLUSIONS: The effectiveness of QICs varied considerably in LMICs. QICs combined with other invention components, such as training, tended to be more effective than QICs alone. The low evidence quality and large effect sizes for QIC plus training justify additional high-quality studies assessing this approach in LMICs.

BACKGROUND: Health care provider (HCP) performance in low- and middle-income countries (LMICs) is often inadequate. The Health Care Provider Performance Review (HCPPR) is a comprehensive systematic review of the effectiveness and cost of strategies to improve HCP performance in LMICs. We present the HCPPR's methods, describe methodological and contextual attributes of included studies, and examine time trends of study attributes. METHODS: The HCPPR includes studies from LMICs that quantitatively evaluated any strategy to improve HCP performance for any health condition, with no language restrictions. Eligible study designs were controlled trials and interrupted time series. In 2006, we searched 15 databases for published studies; in 2008 and 2010, we completed searches of 30 document inventories for unpublished studies. Data from eligible reports were double-abstracted and entered into a database, which is publicly available. The primary outcome measure was the strategy's effect size. We assessed time trends with logistic, Poisson, and negative binomial regression modeling. We were unable to register with PROSPERO (International Prospective Register of Systematic Reviews) because the protocol was developed prior to the PROSPERO launch. RESULTS: We screened 105,299 citations and included 824 reports from 499 studies of 161 intervention strategies. Most strategies had multiple components and were tested by only one study each. Studies were from 79 countries and had diverse methodologies, geographic settings, HCP types, work environments, and health conditions. Training, supervision, and patient and community supports were the most commonly evaluated strategy components. Only 33.6% of studies had a low or moderate risk of bias. From 1958-2003, the number of studies per year and study quality increased significantly over time, as did the proportion of studies from low-income countries. Only 36.3% of studies reported information on strategy cost or cost-effectiveness. CONCLUSIONS: Studies have reported on the efficacy of many strategies to improve HCP performance in LMICs. However, most studies have important methodological limitations. The HCPPR is a publicly accessible resource for decision-makers, researchers, and others interested in improving HCP performance.

The use of mobile technologies in medicine, or mHealth, holds promise to improve health worker (HW) performance, but evidence is mixed. We conducted a cluster-randomized controlled trial to evaluate the effect of text message reminders to HWs in outpatient health facilities (HFs) on quality of care for malaria, pneumonia, and diarrhea in Malawi. After a baseline HF survey (2,360 patients) in January 2015, 105 HFs were randomized to three arms: 1) text messages to HWs on malaria case management; 2) text messages to HWs on malaria, pneumonia, and diarrhea case management (latter two for children < 5 years); and 3) control arm (no messages). Messages were sent beginning April 2015 twice daily for 6 months, followed by an endline HF survey (2,536 patients) in November 2015. An intention-to-treat analysis with difference-in-differences binomial regression modeling was performed. The proportion of patients with uncomplicated malaria managed correctly increased from 42.8% to 59.6% in the control arm, from 43.7% to 55.8% in arm 1 (effect size -4.7%-points, 95% confidence interval (CI): -18.2, 8.9, P = 0.50) and from 30.2% to 50.9% in arm 2 (effect size 3.9%-points, 95% CI: -14.1, 22.0, P = 0.67). Prescription of first-line antibiotics to children < 5 years with clinically defined pneumonia increased in all arms, but decreased in arm 2 (effect size -4.1%-points, 95% CI: -42.0, 33.8, P = 0.83). Prescription of oral rehydration solution to children with diarrhea declined slightly in all arms. We found no significant improvements in malaria, pneumonia, or diarrhea treatment after HW reminders, illustrating the importance of rigorously testing new interventions before adoption.

BACKGROUND: Mobile health (mHealth), which uses technology such as mobile phones to improve patient health and health care delivery, is increasingly being tested as an intervention to promote health worker (HW) performance. This study assessed the effect of short messaging services (SMS) reminders in a study setting. Following a trial of text-message reminders to HWs to improve case management of malaria and other childhood diseases in southern Malawi that showed little effect, qualitative data was collected to explore the reasons why the intervention was ineffective and describe lessons learned. METHODS: Qualitative data collection was undertaken to lend insight into quantitative results from a trial in which 105 health facilities were randomized to three arms: (1) twice-daily text-message reminders to HWs, including clinicians and drug dispensers, on case management of malaria; (2) twice-daily text-message reminders to HWs on case management of malaria, pneumonia and diarrhoea; and, (3) a control arm. In-depth interviews were conducted with 50 HWs in the intervention arms across seven districts. HWs were asked about acceptability and feasibility of the text-messaging intervention and its perceived impact on recommended case management. The interviews were recorded, transcribed and translated into English for a thematic and framework analysis. Nvivo 11 software was used for data management and analysis. RESULTS: A total of 50 HWs were interviewed at 22 facilities. HWs expressed high acceptance of text-message reminders and appreciated messages as job aids and practical reference material for their day-to-day work. However, HWs said that health systems barriers, including very high outpatient workload, commodity stock-outs, and lack of supportive supervision and financial incentives demotivated them, limited their ability to act on messages and therefore adherence to case management guidelines. Drug dispensers were more likely than clinicians to report usage of text-message reminders. Despite these challenges, nearly all HWs expressed a desire for a longer duration of the SMS intervention. CONCLUSIONS: Text-message reminders to HWs can provide a platform to improve understanding of treatment guidelines and case management decision-making skills, but might not improve actual adherence to guidelines. More interaction, for example through targeted supervision or two-way technology communication, might be an essential intervention component to help address structural barriers and facilitate improved clinical practice.

BACKGROUND: Inadequate health-care provider performance is a major challenge to the delivery of high-quality health care in low-income and middle-income countries (LMICs). The Health Care Provider Performance Review (HCPPR) is a comprehensive systematic review of strategies to improve health-care provider performance in LMICs. METHODS: For this systematic review we searched 52 electronic databases for published studies and 58 document inventories for unpublished studies from the 1960s to 2016. Eligible study designs were controlled trials and interrupted time series. We only included strategy-versus-control group comparisons. We present results of improving health-care provider practice outcomes expressed as percentages (eg, percentage of patients treated correctly) or as continuous measures (eg, number of medicines prescribed per patient). Effect sizes were calculated as absolute percentage-point changes. The summary measure for each comparison was the median effect size (MES) for all primary outcomes. Strategy effectiveness was described with weighted medians of MES. This study is registered with PROSPERO, number CRD42016046154. FINDINGS: We screened 216 477 citations and selected 670 reports from 337 studies of 118 strategies. Most strategies had multiple intervention components. For professional health-care providers (generally, facility-based health workers), the effects were near zero for only implementing a technology-based strategy (median MES 1.0 percentage points, IQR -2.8 to 9.9) or only providing printed information for health-care providers (1.4 percentage points, -4.8 to 6.2). For percentage outcomes, training or supervision alone typically had moderate effects (10.3-15.9 percentage points), whereas combining training and supervision had somewhat larger effects than use of either strategy alone (18.0-18.8 percentage points). Group problem solving alone showed large improvements in percentage outcomes (28.0-37.5 percentage points), but, when the strategy definition was broadened to include group problem solving alone or other strategy components, moderate effects were more typical (12.1 percentage points). Several multifaceted strategies had large effects, but multifaceted strategies were not always more effective than simpler ones. For lay health-care providers (generally, community health workers), the effect of training alone was small (2.4 percentage points). Strategies with larger effect sizes included community support plus health-care provider training (8.2-125.0 percentage points). Contextual and methodological heterogeneity made comparisons difficult, and most strategies had low quality evidence. INTERPRETATION: The impact of strategies to improve health-care provider practices varied substantially, although some approaches were more consistently effective than others. The breadth of the HCPPR makes its results valuable to decision makers for informing the selection of strategies to improve health-care provider practices in LMICs. These results also emphasise the need for researchers to use better methods to study the effectiveness of interventions. FUNDING: Bill & Melinda Gates Foundation, CDC Foundation.

Although health outcomes have improved in low-income and middle-income countries (LMICs) in the past several decades, a new reality is at hand. Changing health needs, growing public expectations, and ambitious new health goals are raising the bar for health systems to produce better health outcomes and greater social value. But staying on current trajectory will not suffice to meet these demands. What is needed are high-quality health systems that optimise health care in each given context by consistently delivering care that improves or maintains health, by being valued and trusted by all people, and by responding to changing population needs. Quality should not be the purview of the elite or an aspiration for some distant future; it should be the DNA of all health systems. Furthermore, the human right to health is meaningless without good quality care because health systems cannot improve health without it.

Improving health worker performance is crucial to accelerating progress in helping children to survive and thrive, say Alexander K Rowe and colleagues

A lack of clear guidance for funders, evaluators and improvers on what to include in evaluation proposals can lead to evaluation designs that do not answer the questions stakeholders want to know. These evaluation designs may not match the iterative nature of improvement and may be imposed onto an initiative in a way that is impractical from the perspective of improvers and the communities with whom they work. Consequently, the results of evaluations are often controversial, and attribution remains poorly understood. Improvement initiatives are iterative, adaptive and context-specific. Evaluation approaches and designs must align with these features, specifically in their ability to consider complexity, to evolve as the initiative adapts over time and to understand the interaction with local context. Improvement initiatives often identify broadly defined change concepts and provide tools for care teams to tailor these in more detail to local conditions. Correspondingly, recommendations for evaluation are best provided as broad guidance, to be tailored to the specifics of the initiative. In this paper, we provide practical guidance and recommendations that funders and evaluators can use when developing an evaluation plan for improvement initiatives that seeks to: identify the questions stakeholders want to address; develop the initial program theory of the initiative; identify high-priority areas to measure progress over time; describe the context the initiative will be applied within; and identify experimental or observational designs that will address attribution.

BACKGROUND: During the 2014-2016 Ebola epidemic in West Africa, a key epidemiological feature was disease transmission within healthcare facilities, indicating a need for infection prevention and control (IPC) training and support. METHODS: IPC training was provided to frontline healthcare workers (HCW) in healthcare facilities that were not Ebola treatment units, as well as to IPC trainers and IPC supervisors placed in healthcare facilities. Trainings included both didactic and hands-on components, and were assessed using pre-tests, post-tests and practical evaluations. We calculated median percent increase in knowledge. RESULTS: From October-December 2014, 20 IPC courses trained 1,625 Guineans: 1,521 HCW, 55 IPC trainers, and 49 IPC supervisors. Median test scores increased 40% (interquartile range [IQR]: 19-86%) among HCW, 15% (IQR: 8-33%) among IPC trainers, and 21% (IQR: 15-30%) among IPC supervisors (all P<0.0001) to post-test scores of 83%, 93%, and 93%, respectively. CONCLUSIONS: IPC training resulted in clear improvements in knowledge and was feasible in a public health emergency setting. This method of IPC training addressed a high demand among HCW. Valuable lessons were learned to facilitate expansion of IPC training to other prefectures; this model may be considered when responding to other large outbreaks.

Malaria remains a major cause of preventable death. The World Health Organization estimated that malaria killed 429,000 people (uncertainty range: 235,000–639,000) in 2015, with most deaths occurring in sub-Saharan Africa and among children under 5 years old.1 Nearly two decades ago, the Roll Back Malaria (RBM) partnership was created to help combat this plague. The initial targets were to halve malaria mortality from 2000 to 2010 and again from 2010 to 2015.2 In addition, the Millennium Development Goals (MDGs) included a target of reducing malaria incidence by 2015. With global incidence falling by an estimated 37% compared with 2000, this target has been achieved.3 Current targets, which correspond to the era of the Sustainable Development Goals (SDGs), are to reduce malaria incidence and mortality by 90% from 2015 to 2030.4 More than US$ 20 billion has been dedicated to the fight over the past decade, with an estimated US$ 2.9 billion spent on malaria control and elimination in 2015.1 | Ambitious RBM and MDGs, along with the substantial funding that these goals have attracted, made it important to conduct impact evaluations to show how much malaria control efforts had actually reduced malaria mortality. However, impact evaluations are challenging because measuring malaria mortality directly (i.e., counting all malaria deaths) is not feasible in most of sub-Saharan Africa. The earliest impact evaluations in Africa were sub-national malaria control and research projects, such as those conducted from the 1950s to 1970s in Kenya, Tanzania, and Nigeria.5–7 The relatively small scale of these projects allowed for intensive, high-quality, community-based monitoring of child mortality and entomological measures of malaria transmission, which are generally not feasible for national-level programs.

BACKGROUND: Quality improvement is a recommended strategy to improve implementation levels for evidence-based essential interventions, but experience of and evidence for its effects in low-resource settings are limited. We hypothesised that a systemic and collaborative quality improvement approach covering district, facility and community levels, supported by report cards generated through continuous household and health facility surveys, could improve the implementation levels and have a measurable population-level impact on coverage and quality of essential services. METHODS: Collaborative quality improvement teams tested self-identified strategies (change ideas) to support the implementation of essential maternal and newborn interventions recommended by the World Health Organization. In Tanzania and Uganda, we used a plausibility design to compare the changes over time in one intervention district with those in a comparison district in each country. Evaluation included indicators of process, coverage and implementation practice analysed with a difference-of-differences and a time-series approach, using data from independent continuous household and health facility surveys from 2011 to 2014. Primary outcomes for both countries were birth in health facilities, breastfeeding within 1 h after birth, oxytocin administration after birth and knowledge of danger signs for mothers and babies. Interpretation of the results considered contextual factors. RESULTS: The intervention was associated with improvements on one of four primary outcomes. We observed a 26-percentage-point increase (95% CI 25-28%) in the proportion of live births where mothers received uterotonics within 1 min after birth in the intervention compared to the comparison district in Tanzania and an 8-percentage-point increase (95% CI 6-9%) in Uganda. The other primary indicators showed no evidence of improvement. In Tanzania, we saw positive changes for two other outcomes reflecting locally identified improvement topics. The intervention was associated with an increase in preparation of clean birth kits for home deliveries (31 percentage points, 95% CI 2-60%) and an increase in health facility supervision by district staff (14 percentage points, 95% CI 0-28%). CONCLUSIONS: The systemic quality improvement approach was associated with improvements of only one of four primary outcomes, as well as two Tanzania-specific secondary outcomes. Reasons for the lack of effects included limited implementation strength as well a relatively short follow-up period in combination with a 1-year recall period for population-based estimates and a limited power of the study to detect changes smaller than 10 percentage points. TRIAL REGISTRATION: Pan African Clinical Trials Registry: PACTR201311000681314.

Pneumonia and diarrhea are leading causes of child deaths in Malawi. Guidelines to manage childhood illnesses in resource-poor settings exist, but studies have reported low health-care worker (HCW) adherence to guidelines. We conducted a health facility survey from January to March 2015 to assess HCW management of pneumonia and diarrhea in children < 5 years of age in southern Malawi, and to determine factors associated with case management quality. Descriptive statistics and multivariable logistic regression models examined patient, HCW, and health facility factors associated with recommended pneumonia and diarrhea management, using Malawi's national guidelines as the gold standard. Of 694 surveyed children 2-59 months of age at 95 health facilities, 132 (19.0%) met survey criteria for pneumonia; HCWs gave recommended antibiotic treatment to 90 (68.2%). Of 723 children < 5 years of age, 222 (30.7%) had uncomplicated diarrhea; HCWs provided recommended treatment to 94 (42.3%). In multivariable analyses, caregivers' spontaneous report of children's symptoms was associated with recommended treatment of both pneumonia (odds ratio [OR]: 2.8, 95% confidence interval [CI]: 1.2-6.8, P = 0.023) and diarrhea (OR: 24.2, 95% CI: 6.0-97.0, P < 0001). Malaria diagnosis was negatively associated with recommended treatment (OR for pneumonia: 0.5, 95% CI: 0.2-1.0, P = 0.046; OR for diarrhea: 0.3, 95% CI: 0.1-0.6, P = 0.003). To improve quality of care, children should be assessed systematically, even when malaria is suspected. Renewed efforts to invigorate such a systematic approach, including HCW training, regular follow-up supervision, and monitoring HCW performance, are needed in Malawi.

BACKGROUND: Appropriate diagnosis and treatment are essential for reducing malaria mortality. A cross-sectional outpatient health facility (HF) survey was conducted in southern Malawi from January to March 2015 to determine appropriate malaria testing and treatment practices four years after implementation of a policy requiring diagnostic confirmation before treatment. METHODS: Enrolled patients were interviewed, examined and had their health booklet reviewed. Health workers (HWs) were asked about training, supervision and access to the 2013 national malaria treatment guidelines. HFs were assessed for malaria diagnostic and treatment capacity. Weighted descriptive analyses and logistic regression of patient, HW and HF characteristics related to testing and treatment were performed. RESULTS: An evaluation of 105 HFs, and interviews of 150 HWs and 2342 patients was completed. Of 1427 suspect uncomplicated malaria patients seen at HFs with testing available, 1072 (75.7%) were tested, and 547 (53.2%) tested positive. Testing was more likely if patients spontaneously reported fever (odds ratio (OR) 2.6; 95% confidence interval (CI) 1.7-4.0), headache (OR 1.5; 95% CI 1.1-2.1) or vomiting (OR 2.0; 95% CI 1.0-4.0) to HWs and less likely if they reported skin problems (OR 0.4; 95% CI 0.2-0.6). Altogether, 511 (92.7%) confirmed cases and 98 (60.3%) of 178 presumed uncomplicated malaria patients (at HFs without testing) were appropriately treated, while 500 (96.6%) of 525 patients with negative tests did not receive anti-malarials. Only eight (5.7%) suspect severe malaria patients received appropriate pre-referral treatment. Appropriate treatment was more likely for presumed uncomplicated malaria patients (at HFs without testing) with elevated temperature (OR 1.5/1 degrees C increase; 95% CI 1.1-1.9), who reported fever to HWs (OR 5.7; 95% CI 1.9-17.6), were seen by HWs with additional supervision visits in the previous 6 months (OR 1.2/additional visit; 95% CI 1.0-1.4), or were seen by older HWs (OR 1.1/year of age; 95% CI 1.0-1.1). CONCLUSIONS: Correct testing and treatment practices were reasonably good for uncomplicated malaria when testing was available. Pre-referral treatment for suspect severe malaria was unacceptably rare. Encouraging HWs to elicit and appropriately respond to patient symptoms may improve practices.

BACKGROUND: Correct treatment of potentially life-threatening illnesses (PLTIs) in children under 5 years, such as malaria, pneumonia, and diarrhea, can substantially reduce mortality. The Integrated Management of Childhood Illness (IMCI) strategy has been shown to improve treatment of child illnesses, but multiple studies have shown that gaps in health worker performance remain after training. To better understand factors related to health worker performance, we analyzed 9,330 patient consultations in Benin from 2001-2002, after training one of the first cohorts of 32 health workers in IMCI. METHODS: With data abstracted from patient registers specially designed for IMCI-trained health workers, we examined associations between health facility-, health worker-, and patient-level factors and 10 case-management outcomes for PLTIs. RESULTS: Altogether, 63.6 % of children received treatment for all their PLTIs in accordance with IMCI guidelines, and 77.8 % received life-saving treatment (i.e., clinically effective treatment, even if not exactly in accordance with IMCI guidelines). Performance of individual health workers varied greatly, from 15-88 % of patients treated correctly, on average. Multivariate regression analyses identified several factors that might have influenced case-management quality, many outside a manager's direct control. Younger health workers significantly outperformed older ones, and infants received better care than older children. Children with danger signs, those with more complex illnesses, and those with anemia received worse care. Health worker supervision was associated with improved performance for some outcomes. CONCLUSIONS: A variety of factors, some outside the direct control of program managers, can influence health worker practices. An understanding of these influences can help inform the development of strategies to improve performance.

BACKGROUND: The lack of high quality timely data for evidence-informed decision making at the district level presents a challenge to improving maternal and newborn survival in low income settings. To address this problem, the EQUIP project (Expanded Quality Management using Information Power) implemented a continuous household and health facility survey for continuous feedback of data in two districts each in Tanzania and Uganda as part of a quality improvement innovation for mothers and newborns. METHODS: Within EQUIP, continuous survey data were used for quality improvement (intervention districts) and for effect evaluation (intervention and comparison districts). Over 30 months of intervention (November 2011 to April 2014), EQUIP conducted continuous cross-sectional household and health facility surveys using 24 independent probability samples of household clusters to represent each district each month, and repeat censuses of all government health facilities. Using repeat samples in this way allowed data to be aggregated at six four-monthly intervals to track progress over time for evaluation, and for continuous feedback to quality improvement teams in intervention districts.In both countries, one continuous survey team of eight people was employed to complete approximately 7,200 household and 200 facility interviews in year one. Data were collected using personal digital assistants. After every four months, routine tabulations of indicators were produced and synthesized to report cards for use by the quality improvement teams. RESULTS: The first 12 months were implemented as planned. Completion of household interviews was 96% in Tanzania and 91% in Uganda. Indicators across the continuum of care were tabulated every four months, results discussed by quality improvement teams, and report cards generated to support their work. CONCLUSIONS: The EQUIP continuous surveys were feasible to implement as a method to continuously generate and report on demand and supply side indicators for maternal and newborn health; they have potential to be expanded to include other health topics. Documenting the design and implementation of a continuous data collection and feedback mechanism for prospective description, quality improvement, and evaluation in a low-income setting potentially represents a new paradigm that places equal weight on data systems for course correction, as well as evaluation.

To improve healthcare quality for hospitalized patients with malaria in Benin, a feasible and valid evaluation method is needed. Because observation of inpatients is challenging, chart abstraction is an attractive option. However, the quality of inpatient charts is unknown. We employed three methods in five hospitals to assess 11 signs of malaria and severe disease: 1) chart abstraction (probability sample of inpatients), 2) chart abstraction compared to interviews of inpatients and health workers (HWs), and 3) abstraction from charts of recently discharged inpatients compared to interviews with HWs. Method 1 showed that of 473 malaria signs (from 43 charts), 178 (38%, 95% confidence interval 24-51%) were documented. Method 2 showed that 96% (45 of 47) of documented signs were valid. Method 3 suggests that 65% (36 of 55) of non-documented signs were assessed (but not documented) by HWs. Chart abstraction was feasible and documented data were valid, but results should be interpreted cautiously in consideration of low levels of documentation.

BACKGROUND: A challenge for systematic reviews on improving health worker performance is that included studies often use different performance indicators, and the validity of comparing interventions with different indicators is unclear. One potential solution is to adjust comparisons by indicator category, with categories based on steps of the case-management process that can be easily recognized (assessment of symptoms, treatment etc.) and that might require different levels of effort to bring about improvements. However, this approach would only be useful if intervention effect sizes varied by indicator category. To explore this approach, studies were analyzed that evaluated the Integrated Management of Childhood Illness (IMCI) strategy. METHODS: Performance indicators were grouped into four categories: patient assessment, diagnosis, treatment and counseling. An effect size of IMCI was calculated for each indicator. Linear regression modeling was used to test for differences among the mean effect sizes of the indicator categories. RESULTS: Six studies were included, with data from 3136 ill child consultations. Mean effect sizes for 63 assessment indicators, 12 diagnosis indicators, 31 treatment indicators and 34 counseling indicators were 50.9 percentage-points (%-points), 44.7, 36.5 and 46.6%-points, respectively. After adjusting for baseline indicator value, compared with the assessment mean effect size, the diagnosis mean was 7.3%-points lower (P = 0.23), the treatment mean was 15.2%-points lower (P = 0.0004) and the counseling mean was 12.9%-points lower (P = 0.0027). CONCLUSION: Adjusting the results of systematic reviews for indicator category and baseline indicator value might be useful for improving the validity of intervention comparisons.

OBJECTIVE: Training health workers to use Integrated Management of Childhood Illness (IMCI) guidelines can improve care for ill children in outpatient settings in developing countries. However, even after IMCI training, important performance gaps exist. One potential reason is that the effect of training can rapidly wane. Our aim was to determine if the performance of IMCI-trained health workers deteriorated over 3 years. METHODS: We studied two departments in Benin. First, we performed a record review of 32 IMCI-trained health workers during the first year of IMCI implementation (2001-2002). Second, we analysed data from cross-sectional health facility surveys from 2001 to 2004 that represented the entire study area. Primary outcomes were the proportion of children under 5 years old with potentially life-threatening illnesses who received either recommended or adequate treatment, and among all children, an index of overall guideline adherence. Secondary outcomes reflected the treatment of individual diseases. Outcomes were calculated monthly, and time trends were evaluated with regression modelling. RESULTS: The record review included 9393 consultations, and the surveys included 411 consultations performed by 105 health workers. For both data sources, performance trends were essentially flat for nearly all outcomes. Absolute levels of performance revealed substantial performance gaps. CONCLUSIONS: We found no evidence that performance declined over 3 years after IMCI training. However, important performance gaps found immediately after IMCI training persisted and should be addressed. (2012 Blackwell Publishing Ltd.)

BACKGROUND: Health workers' malaria case-management practices often differ from national guidelines. We assessed whether text-message reminders sent to health workers' mobile phones could improve and maintain their adherence to treatment guidelines for outpatient paediatric malaria in Kenya. METHODS: From March 6, 2009, to May 31, 2010, we did a cluster-randomised controlled trial at 107 rural health facilities in 11 districts in coastal and western Kenya. With a computer-generated sequence, health facilities were randomly allocated to either the intervention group, in which all health workers received text messages on their personal mobile phones on malaria case-management for 6 months, or the control group, in which health workers did not receive any text messages. Health workers were not masked to the intervention, although patients were unaware of whether they were in an intervention or control facility. The primary outcome was correct management with artemether-lumefantrine, defined as a dichotomous composite indicator of treatment, dispensing, and counselling tasks concordant with Kenyan national guidelines. The primary analysis was by intention to treat. The trial is registered with Current Controlled Trials, ISRCTN72328636. FINDINGS: 119 health workers received the intervention. Case-management practices were assessed for 2269 children who needed treatment (1157 in the intervention group and 1112 in the control group). Intention-to-treat analysis showed that correct artemether-lumefantrine management improved by 23.7 percentage-points (95% CI 7.6-40.0; p=0.004) immediately after intervention and by 24.5 percentage-points (8.1-41.0; p=0.003) 6 months later. INTERPRETATION: In resource-limited settings, malaria control programmes should consider use of text messaging to improve health workers' case-management practices. FUNDING: The Wellcome Trust.

OBJECTIVE: Implementation of the Integrated Management of Childhood Illness (IMCI) strategy with an 11-day training course for health workers improves care for ill children in outpatient settings in developing countries. The 11-day course duration is recommended by the World Health Organization, which developed IMCI. Our aim was to determine if shortening the training (to reduce cost) reduces its effectiveness. METHODS: We conducted a systematic review to compare IMCI's effectiveness with standard training (duration ≥11 days) versus shortened training (5-10 days). Studies were identified from a search of MEDLINE, two existing systematic reviews, and by contacting investigators. We included published or unpublished studies that evaluated IMCI's effectiveness in developing countries and reported quantitative measures of health worker practices related to managing ill children under 5 years old in public or private health facilities. Summary measures were the median of effect sizes for all outcomes from a given study, and the percentage of patients needing oral antimicrobials or rehydration who were treated according to IMCI guidelines. FINDINGS: Twenty-nine studies were included. Direct comparisons from three studies showed little difference between standard and shortened training. Indirect comparisons from 26 studies revealed that effect sizes for standard training versus no IMCI were greater than shortened training versus no IMCI. Across all comparisons, differences ranged from -3 to +23 percentage-points, and our best estimate was a 2 to 16 percentage-point advantage for standard training. No result was statistically significant. After IMCI training (of any duration), 34% of ill children needing oral antimicrobials or rehydration were not receiving these treatments according to IMCI guidelines. CONCLUSIONS: Based on limited evidence, standard IMCI training seemed more effective than shortened training, although the difference might be small. As sizable performance gaps often existed after IMCI training, countries should consider implementing other interventions to support health workers after training, regardless of training duration.

BACKGROUND: Angola's malaria case-management policy recommends treatment with artemether-lumefantrine (AL). In 2006, AL implementation began in Huambo Province, which involved training health workers (HWs), supervision, delivering AL to health facilities, and improving malaria testing with microscopy and rapid diagnostic tests (RDTs). Implementation was complicated by a policy that was sometimes ambiguous. METHODS: Fourteen months after implementation began, a cross-sectional survey was conducted in 33 outpatient facilities in Huambo Province to assess their readiness to manage malaria and the quality of malaria case-management for patients of all ages. Consultations were observed, patients were interviewed and re-examined, and HWs were interviewed. RESULTS: Ninety-three HWs and 177 consultations were evaluated, although many sampled consultations were missed. All facilities had AL in-stock and at least one HW trained to use AL and RDTs. However, anti-malarial stock-outs in the previous three months were common, clinical supervision was infrequent, and HWs had important knowledge gaps. Except for fever history, clinical assessments were often incomplete. Although testing was recommended for all patients with suspected malaria, only 30.7% of such patients were tested. Correct testing was significantly associated with caseloads < 25 patients/day (odds ratio: 18.4; p < 0.0001) and elevated patient temperature (odds ratio: 2.5 per 1 degrees C increase; p = 0.007). Testing was more common among AL-trained HWs, but the association was borderline significant (p = 0.072). When the malaria test was negative, HWs often diagnosed patients with malaria (57.8%) and prescribed anti-malarials (60.0%). Sixty-six percent of malaria-related diagnoses were correct, 20.1% were minor errors, and 13.9% were major (potentially life-threatening) errors. Only 49.0% of malaria treatments were correct, 5.4% were minor errors, and 45.6% were major errors. HWs almost always dosed AL correctly and gave accurate dosing instructions to patients; however, other aspects of counseling needed improvement. CONCLUSION: By late-2007, substantial progress had been made to implement the malaria case-management policy in a setting with weak infrastructure. However, policy ambiguities, under-use of malaria testing, and distrust of negative test results led to many incorrect malaria diagnoses and treatments. In 2009, Angola published a policy that clarified many issues. As problems identified in this survey are not unique to Angola, better strategies for improving HW performance are urgently needed.

OBJECTIVE: In developing countries, supervision is a widely recognized strategy for improving health worker performance; and anecdotally, maintaining regular, high-quality supervision is difficult. However, remarkably little research has explored in depth why supervision is so challenging. METHODS: In the context of a trial to improve health worker adherence to Integrated Management of Childhood Illness (IMCI) guidelines and strengthen supervision in southeastern Benin, we used record reviews, focus group discussions, key informant interviews, and cross-sectional surveys to examine the supervision process. FINDINGS: Initially, little IMCI supervision occurred. The frequency increased substantially after implementing a series of workshops, but then deteriorated. Quantitative and qualitative data revealed obstacles to supervision at multiple levels of the health system. Based on supervisors' opinions, the main problems were: poor coordination; inadequate management skills and ineffective management teams; a lack of motivation; problems related to decentralization; health workers sometimes resisting IMCI implementation; and less priority given to IMCI supervision because of incentives for non-supervision activities, a lack of leadership, and an expectation of integrated supervision. To this list, based on our observations, we add: the increasing supervision workload, time required for non-supervision activities, project interventions not always being implemented as planned, and the loss of particularly effective supervisors. In terms of correctly completing steps of the supervision process, the quality of supervision was generally good. CONCLUSIONS: Managers should monitor supervision, understand the evolving influences on supervision, and use their resources and authority to both promote supervision and remove impediments to supervision. Support from leaders can be crucial, thus donors and politicians should help make supervision a true priority. As with front-line clinicians, supervisors are health workers who need support. We emphasize the importance of research to identify effective and affordable strategies for improving supervision frequency and quality. (ClinicalTrials.gov number NCT00510679.).

The global health community is interested in the health impact of the billions of dollars invested to fight malaria in Africa. A recent publication used trends in malaria cases and deaths based on health facility records to evaluate the impact of malaria control efforts in Rwanda and Ethiopia. Although the authors demonstrate the use of facility-based data to estimate the impact of malaria control efforts, they also illustrate several pitfalls of such analyses that should be avoided, minimized, or actively acknowledged. A critique of this analysis is presented because many country programmes and donors are interested in evaluating programmatic impact with facility-based data. Key concerns related to: 1) clarifying the objective of the analysis; 2) data validity; 3) data representativeness; 4) the exploration of trends in factors that could influence malaria rates and thus confound the relationship between intervention scale-up and the observed changes in malaria outcomes; 5) the analytic approaches, including small numbers of patient outcomes, selective reporting of results, and choice of statistical and modeling methods; and 6) internal inconsistency on the strength and interpretation of the data. In conclusion, evaluations of malaria burden reduction using facility-based data could be very helpful, but those data should be collected, analysed, and interpreted with care, transparency, and a full recognition of their limitations.