Study design
The study was a cluster-randomized trial implemented in 40 public and mission hospitals in Ghana between March 2014 and February 2017. Initial study sites included three regional hospitals, 38 district hospitals, and one polyclinic across Upper West, Central, and Western Regions of Ghana. Each facility was enrolled for 18 months. Data on institutional intrapartum stillbirths and 24-h newborn mortality were collected retrospectively for 6 months pre-intervention (i.e., baseline) and prospectively for 12 months post-intervention at each study site (Fig. 1).
All SBAs providing labor, birth, and immediate postnatal care at the selected facilities at the time of the first low-dose session were invited to participate. Four hundred and three SBAs individually consented and were enrolled in the study. All enrolled SBAs were registered or certified midwives.
Randomization and masking
Eligible facilities had at least three SBAs and an average of 30 or more births per month. These hospitals were stratified by region (three categories) and caseload (four categories), and then randomly assigned to one of four implementation waves. The pipeline randomization allowed for rigorous evaluation while the program was rolled out to all facilities. Waves 1 through 4 included eleven, seventeen, eight, and six facilities, respectively. Waves were unbalanced because of the stratification. Two hospitals were excluded at the time of implementation because they no longer met the inclusion criteria. Neither facilities nor participants were masked due to the nature of the intervention.
Procedures
Experienced SBAs were prepared as regional master mentors (MMs) through a course in targeted BEmONC and training skills. MMs conducted the onsite, low-dose sessions and follow-up mentorship activities in collaboration with project staff. Of the 40 study facilities, 19 had a MM working within the facility, while the remaining facilities received mentoring support from MMs via mobile phone and periodic visits. During the first low-dose session at each site, two to three SBAs were trained as peer practice coordinators (PPCs) to lead the high-frequency practice sessions and were oriented to their roles in a day-long session. One month later, the second 4-day low-dose session was conducted. In the month between low-dose sessions and for 11 months afterward, SBAs carried out weekly PPC-led practice sessions with simulators and participated in mMentoring, which included mobile phone–based mentoring and onsite coaching with MMs and automated SMS reminders and quiz questions about key concepts presented during the low-dose sessions (Fig. 2).
MamaNatalie® and NeoNatalie™ simulators were given to each site to enable regular practice of targeted skills. Each site also received newborn resuscitation equipment (NeoNatalie resuscitator, penguin suction [Laerdal], and basic delivery sets). SBAs and health information officers collected service delivery data. Orientation to facility-level data collection was included in the first low-dose session for all SBAs. An additional training was provided to each facility’s health information officer and maternity unit in-charge to ensure a common understanding of indicators and data collection, quality, and use. Monthly service statistics were extracted from routine health information systems and a supplementary delivery register introduced by the project. All outcome data were verified using register reviews at each study site.
Objective structured clinical examination (OSCE) tools and knowledge tests, based on global and national guidelines, were developed to assess SBAs in management of normal birth, immediate newborn care, and maternal and newborn complications. These assessments were used before and immediately after low-dose sessions, and again 1 year later, to evaluate knowledge and skill acquisition and retention. Master mentors and project staff conducted and collected all OSCE and knowledge test data. Assessments were scanned into Captricity (Oakland, CA, USA) for data capture and cleaning.
Outcomes
Twenty-four hour newborn mortality was defined as the death within 24 h, or before discharge, of a newborn who breathed at birth. The institutional 24-h newborn mortality rate was defined as deaths in the first 24 h per 1000 facility live births. Intrapartum stillbirth was defined as the death of a baby who had heart tones present at the time of the mother’s admission to the facility, was born with no signs of maceration, and did not breathe at birth, and on whom resuscitation attempts were unsuccessful. The institutional intrapartum stillbirth rate was defined as the proportion of all facility births that resulted in intrapartum stillbirth. Training results were the proportion of total test items (knowledge questions or skill steps) performed correctly.
Statistical analysis
We ran a simulation to assess statistical power to detect a reduction in the primary outcome, 24-h newborn mortality, in the original 42 study sites. In each simulation run, the study design was replicated, with four study entry time points (i.e., strata) for 42 facilities. A random effect Poisson model was fit to each study replicate to estimate the intervention effect on the number of newborn deaths, adjusting for the stratum, with the number of deliveries per facility as the offset term. Across the multiple iterations of the study, we calculated the empirical statistical power as the proportion of the simulation runs in which the p-value was less than 0·05 level of significance, indicating that the null hypothesis of no post-intervention difference in the number of deaths is rejected. Results indicated that we had 99% power to detect a 25% decrease in the number of newborn deaths after the intervention.
We assessed institutional mortality rates and SBA competency at all facilities post-implementation compared to pre-implementation and assessed differences between institutional mortality rates at a subset of intervention and non-intervention (i.e., control) facilities during three 6-month time intervals. The entire study period (March 2014 to February 2017) was subdivided into four intervention waves, based on the start date of the intervention, and six calendar periods. For each facility the following variables were defined: intervention stage (coded as 0 for baseline, 1 for months 1–6 after Low-Dose 1, and 2 for months 7–12 after Low-Dose 1), wave (from 1 to 4, depending on the start of the intervention), and calendar period (from 1 to 6). Exploratory analyses included comparison of the observed intrapartum stillbirth or newborn death rates by intervention stage, wave, calendar period, and region. Outcome rate trajectories were plotted over the calendar period by wave to assess for within-group trends. Baseline mortality rates were calculated as a facility-weighted (births or live births) average over the 6-month baseline period. Comparability of mortality rates at baseline was assessed using analysis of variance (ANOVA).
To estimate the effect of the intervention, we fit a series of generalized population-average linear models with negative binomial distribution and a log-link with exchangeable working correlation structure. The models were estimated using generalized estimating equations with robust variance estimates and a negative binomial distribution.
The base model included two indicator variables for intervention (months 1–6 and months 7–12) and adjustment for the region and level of the health facility (regional hospital or district hospital/polyclinic). Exponentiated beta coefficients for the two intervention variables were interpreted as the relative risk (RR) of the outcome post-intervention compared to the historical control (baseline) period.
To assess whether the effect of the intervention varied significantly by wave or region, the base model was modified to include the appropriate interaction terms. Secondary analyses were conducted using time as a continuous variable in a similar model to assess the monthly change in mortality rates prior to and after intervention. In addition, we used interaction terms in a modified model to assess the effect of the intervention comparing intervention and control facilities in the same calendar period, adjusting for effect of facility level on the outcome measures.
OSCE and knowledge test scores were calculated based on the number of steps correct in the OSCE or number of items correct on the knowledge test out of the total possible correct on each test. Scores were analyzed using ordinary least-squares linear regression that tested for a difference in individuals’ pre- and post-test assessment scores, and pre-test and 1-year assessment scores, adjusting for clustering of SBAs within the facility.
Data were analyzed using Stata Version 14 (StataCorp LLC, College Station, TX, USA).