Skip to main content

Advertisement

You are viewing the new article page. Let us know what you think. Return to old version

Economic evaluation of emergency obstetric care training: a systematic review

Abstract

Background

Training healthcare providers in Emergency Obstetric Care (EmOC) has been shown to be effective in improving their capacity to provide this critical care package for mothers and babies. However, little is known about the costs and cost-effectiveness of such training. Understanding costs and cost-effectiveness is essential in guaranteeing value-for-money in healthcare spending. This study systematically reviewed the available literature on cost and cost-effectiveness of EmOC trainings.

Methods

Peer-reviewed and grey literature was searched for relevant papers published after 1990. Studies were included if they described an economic evaluation of EmOC training and the training cost data were available. Two reviewers independently searched, screened, and selected studies that met the inclusion criteria, with disagreements resolved by a third reviewer. Quality of studies was assessed using the Consolidated Health Economic Evaluation Reporting Standards statement. For comparability, all costs in local currency were converted to International dollar (I$) equivalents using purchasing power parity conversion factors. The cost per training per participant was calculated. Narrative synthesis was used to summarise the available evidence on cost effectiveness.

Results

Fourteen studies (five full and nine partial economic evaluations) met the inclusion criteria. All five and two of the nine partial economic evaluations were of high quality. The majority of studies (13/14) were from low- and middle-income countries. Training equipment, per diems and resource person allowance were the most expensive components. Cost of training per person per day ranged from I$33 to I$90 when accommodation was required and from I$5 to I$21 when training was facility-based. Cost-effectiveness of training was assessed in 5 studies with differing measures of effectiveness (knowledge, skills, procedure cost and lives saved) making comparison difficult.

Conclusions

Economic evaluations of EmOC training are limited. There is a need to scale-up and standardise processes that capture both cost and effectiveness of training and to agree on suitable economic evaluation models that allow for comparability across settings.

Trial registration

PROSPERO_CRD42016041911.

Background

Improving maternal and newborn health has been at the forefront of the global health agenda for more than two decades. However, despite a 44% drop in maternal mortality ratio between 2000 and 2015, an estimated 300,000 women still die each year due to complications of pregnancy and childbirth [1]. In addition, an estimated 2.6 million babies are stillborn and 2.7 million newborns die within the first 28 days of life [2, 3]. Unlike many other public health concerns, maternal and newborn mortality is significantly influenced by institutionally-based clinical interventions [4, 5]. Evidence suggests that majority of these deaths could be prevented by timely and effective emergency obstetric care (EmOC) [6, 7]. However, recent evidence shows that more than half of all women with obstetric complications lack access to this life-saving intervention [8]. EmOC relies on the presence of suitably trained and competent healthcare providers. When carried out by a competent provider, it is estimated that EmOC can reduce intra-partum stillbirths by between 45% and 75%, [9] as well as reduce health facility-based maternal mortality by up to 50% [10].

In the early 1990s, there was wide acknowledgement globally that deficiency in the obstetric skills of healthcare providers was one of the reason for poor quality of care. In view of this, training for healthcare providers was recommended [11, 12]. At the time, EmOC training courses such as the Advanced Life Support in Obstetrics (ALSO) and Managing Obstetric Emergencies and Trauma (MOET) were developed to meet this need in high income settings [13, 14]. Since then, several other training programmes have been developed and implemented across the globe [15,16,17,18]. Studies have shown that in-service EmOC training is effective in increasing knowledge and skills of healthcare providers and can improve the quality and effectiveness of care [19,20,21,22,23]. However, despite this, very little is published about the costs and cost-effectiveness of training. Such information is usually obtained via economic evaluation studies. Partial economic evaluations (such as cost analysis, cost-description or outcome description), consider costs and/or consequences but do not compare different interventions or do not relate costs to benefits. Full Economic Evaluations (such as cost minimisation analysis (CMA), cost-effectiveness analysis (CEA), cost-utility analysis (CUA) and cost-benefit analysis (CBA)), compare both the costs and the consequences (benefits, effectiveness) of one or more interventions [24] (Table 1).

Table 1 Description of types of economic evaluation studies

In the era of the Sustainable Development Goals (SDGs), when competition for limited resources is high, information on the cost-effectiveness of existing and promising new interventions to improve health of mothers and their babies will be central to informing policy and practice [25]. It is important to understand the costs and cost-effectiveness of training packages in order to aid decision-makers on the most efficient use of resources and to assess value-for-money [25, 26]. The objective of this review is to systematically assess and summarise the evidence available on economic evaluations of in-service training in Emergency Obstetric Care (EmOC) for healthcare providers.

Methods

In designing the methods for this review, we borrowed critical insights on best practices for conducting systematic reviews of economic evaluations from experts from the Centre for Reviews and Dissemination and the Task Force on Community Preventive Services [27, 28].

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) approach was used in reporting the findings of the systematic search conducted for this review [29].

Search strategy

Multiple strategies were used to search articles in PubMed, Scopus, the Cochrane Library, Web of Knowledge, Google Scholar, CINAHL Plus, Global Health Archive, EconLit, Popline and African Journal Online. In searching, we combined medical subject headings (MeSH) and/or key words, using Boolean linkages “OR” within categories and “AND” between three categories.

  1. a)

    Emergency obstetric care: “obstetric emergenc*” OR “emergency obstetric care” OR “emergency obstetric and newborn care” OR EmOC OR EmONC

    AND

  2. b)

    Training: Training OR education OR “capacity building”

    AND

  3. c)

    Costs and economic evaluation: “cost*minimization” OR “cost*analysis” OR cost* OR “cost*effectiveness” OR “cost*utility” OR “cost*benefit” OR “economic evaluation”

The search terms used were based on the optimal search strategy for retrieving cost and economic studies in health services research [30].

The websites of non-government organisations and UN agencies were searched to identify grey literature, including John Snow International (JSI), Population Council, Averting Maternal Death and Disability (AMDD), Maternal Health Task Force (MHTF), United Nations Children’s’ Fund (UNICEF), United Nations Fund for Population (UNFPA) and World Health Organization (WHO). In addition to the automated search, relevant articles were identified through searching reference lists by hand and reviewing studies included in systematic reviews of training effectiveness.

The search was conducted for articles published from January 1990 to December 2016. The decision to include only studies published from 1990 was based on the recognition that in-service EmOC training was introduced at this time. We did not limit our search by language.

Two co-authors independently conducted the search and screened all retrieved records. Titles and abstracts were screened for relevance and eligibility, based on the set inclusion and exclusion criteria. Any discrepancies were resolved through discussion with the other co-authors. This was done to minimise selection bias.

Inclusion and exclusion criteria

Articles were included if these described any type of economic evaluation of an in-service (as opposed to pre-service) training in EmOC and provided results of the evaluation including costs data.

Articles (including letters, commentaries or editorials) that reported effectiveness data without any training cost data were excluded. In addition, articles that reported multiple implemented trainings without disentangling EmOC training specifically and any articles that described training of non-healthcare provider participants were also excluded.

Data extraction

Information was extracted pertaining to; study and training characteristics (including year of publication, country of training, cadre of training participants, number of training participants, training content, trainers/facilitators, duration of the training) as well as key findings on costs and cost-effectiveness of the training (economic evaluation type, overall study design (standalone evaluation versus nested in another study), the full breakdown of costs included for analysis by authors, reported or estimated total training implementation costs and currency in which costs were reported). Data was extracted by two reviewers independently and then checked for accuracy by a third reviewer.

Quality assessment

The 24-item Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist was used to assess the quality of reporting of the included full economic evaluations [31]. For partial economic evaluations, the relevant criteria in the CHEERS checklist were combined with those suggested by Stone et al. 2005 [32] (including provision of full costs breakdown and inclusion of opportunity costs) to create an eight-item costs focused quality checklist. The opportunity cost was taken to be the value of the best foregone alternative use of resources [24].

For each item, a score of 1 was awarded if the criterion was fully met, 0.5, if partially met, 0, if not met or only minimal information was provided and NA if not applicable. The total score achieved across all the criteria was then summed and converted to percentages. Since no standardised interpretation of the quality assessment tool exists, studies with 75% or more criteria fully met were graded as high quality, 50–74% as average quality and below 50% as poor quality [33]. Each included study was assessed independently by two co-authors.

Data synthesis

Following a brief description of the characteristics of the type of EmOC training reported, studies were classified as being either a partial or full economic evaluation.

For costs captured in both partial and full economic evaluations, all the cost data provided by the authors were retrieved. The different direct training costs (core costs including costs for central management, monitoring and research and dissemination of findings, implementation costs, overheads and external evaluation) were identified. For cost comparison across the included studies, only implementation costs for the training (costs incurred for actual delivery of the training) were selected and included. Examples of direct implementation costs included in the comparative analysis include cost of hiring a training venue, teaching materials, equipment costs, supervision costs, travel expenses, and/or, consultant fees for trainers. Opportunity costs such as costs of work that trainees could have being doing if they were not attending the training were excluded. Costs associated with start-up (such as cost of setting up an office for the training organisation), administration and capital projects were also excluded.

To allow for cost comparability, 2015 purchasing power parities conversion factors [34] were used to convert local currency of the country in which the training was conducted to International Dollar (I$) equivalents for the year the training took place [35]. Costs reported in US dollars using ‘market exchange rates’ were converted to local currency for the year the training was conducted, using official OANDA exchange rates (http://www.oanda.com/currency/historical-rates). The derived local currency value was subsequently converted to I$ equivalents for the same year. Purchasing power parity (as opposed to market exchange rates) allows one to estimate the amount it would have cost hypothetically to purchase the same market basket of goods in both countries, if their currencies were at par [35]. Based on the I$ equivalents, the cost per trainee per day were calculated for each study. When training was implemented over multiple years, we selected the last year of implementation when the training was completed.

For cost-effectiveness (which is only captured in full economic evaluations), the dimensions used to report effectiveness/utility/benefits in the included studies were identified. A narrative synthesis was used to summarise the available evidence [27].

Results

Two hundred thirty nine articles from both peer-reviewed and grey literature sources were screened by title and abstract for inclusion in the full-text review. Full-text of 42 articles were subsequently read, of which 12 articles met the inclusion criteria. An additional two articles were identified following hand searching, with a total of 14 studies included in the analysis (Fig. 1). Details of data extracted from the included studies and reasons for excluding the excluded studies are presented in the summary table (Additional file 1: Table S1: Summary of included studies).

Fig. 1
figure1

PRISMA diagram

Overview of studies

Nine studies were partial economic evaluations [36,37,38,39,40,41,42,43,44] and five studies were full economic evaluations [45,46,47,48,49] (four used CEA [45,46,47,48] and one used a mix of CEA and CUA [49]). No CBA study was retrieved. Ten studies were stand-alone economic evaluations [36, 37, 39,40,41,42,43,44,45], while four were reported as part of quasi-experimental studies of training effectiveness [38, 46, 47, 49]. All the articles retrieved were published in English.

Eight studies reported on EmOC training alone [37, 42,43,44,45,46, 48, 49], while the remaining six reported on EmOC training conducted along with other interventions such as healthcare facility renovation, improved availability of equipment and supplies, management information systems and reviewed policy or advocacy [36, 38,39,40,41, 47].

A total of 11 studies focused on the economic evaluation of a specific EmOC training intervention [36, 44, 47, 49], two studies compared cost-effectiveness of alternative EmOC training delivery approaches [45, 46], and one study compared the cost of training doctors and surgical technicians per surgery conducted after training [48].

The included studies reported on trainings conducted in eleven low- and middle-income countries including Bangladesh [36, 37], Ethiopia [41], Ghana [46], Indonesia [45], Kenya [47], Mozambique [38, 48], Nepal [39], Nigeria [40, 43], Tanzania [47], Zambia [49] and Zimbabwe [42]. One study was conducted in the United Kingdom (UK) [44].

Quality assessment

Based on the CHEERS checklist [31], one full economic evaluation was rated high quality [48], while the remaining four studies were rated average quality [45,46,47, 49]. Applying the costs quality criteria to all studies, two partial economic evaluation [39, 44] and all five full economic evaluations [45, 49] were assessed as high quality. Six partial economic evaluations were assessed as average quality [36,37,38, 40, 41, 43], and one partial economic evaluation was assessed as low quality [43] (Additional file 2: Table S2: Quality assessment of full economic evaluations, Additional file 3: Table S3. Quality assessment of cost analysis in partial and full economic evaluations).

Description of EmOC training for which economic evaluations have been conducted

Three studies reported on training conducted for midwives only [40, 45, 49], one for doctors only [36], eight for both doctors and midwives [37,38,39, 41, 42, 44, 46, 47], one for health aides and midwives [43], and one for surgical technicians, who provide EmOC [48].

Most studies described training that lasted between 1 and 30 days (seven studies) [38, 42,43,44,45,46, 49], three medium-term ranging from 45 to 180 days [37, 41, 46], and three described long-term training that ranged from one to three years [36, 37, 48]. Two studies did not specify the duration of the training [40, 47].

In the two studies that provided information on number of trainers and trainees, trainee/trainer ratio ranged from 3:1 to 7:1 [42, 49].

Seven of the trainings reported were facility-based [37, 38, 40,41,42, 44, 49], one was fully residential [39], while another four were of a mixed format (facility based with residential or facility-based with self-paced learning) [43, 45, 46, 48]. Two studies did not define the training site [36, 47].

Number of healthcare providers trained in the included studies ranged from 10 midwives in Nigeria [40] to 477 providers including doctors and midwives in the UK [44]. The number of trainees per session ranged from 18 to 28 [42, 49].

Costs of implementing EmOC training

The most commonly included costs for EmOC training were training materials (11 studies) [37,38,39, 41,42,43,44,45,46, 48], and travel expense and subsistence fees for facilitators (10 studies) [36,37,38, 41, 43,44,45,46,47,48,49].

The least commonly included costs were participant’s catering and boarding (three studies each) [43, 47, 49], and administration costs [9, 42, 47]. Two studies estimated the opportunity cost of the time spent by healthcare providers attending the training (instead of providing health care) [44, 46].

Three studies [40, 41, 46] included the actual component cost and/or percentage breakdown of the total training implementation costs. In Nigeria, the training equipment (52%) and obstetrician visits (30%) were the mostly expensive components of the training implementation cost [40]. Similarly, in Ethiopia, 67% of the total implementation cost was spent on training materials and equipment while the remaining 33% was spent on travel expenses and per diems [41]. However, in the training conducted in Ghana, per diem constituted the largest proportion of training implementation costs, making up 63% (self-paced learning) and 75% (residential) respectively. This was followed by resource person allowances (18% (self-paced learning) and 17% (residential)) [46].

It was possible to estimate the cost per trainee per day for nine of the included studies [36,37,38, 43,44,45,46, 48, 49] (Table 2). This ranged from I$5 to I$90 per trainee per day [36, 45]. Trainings that required boarding cost between I$33 and I$90, while those that were facility-based or, had some form of internship incorporated, cost between I$5 and I$21 per trainee per day. It was not possible to estimate costs per trainee per day for the other studies because the number of training days [40, 47] or the training implementation costs [39, 41, 42] were not provided.

Table 2 Cost of Emergency Obstetric Care training for included studies

Cost-effectiveness of EmOC training

One study used knowledge change of the group (mean score) for labour and delivery [46] and another used change in mean skills score [45]. Four studies reported effectiveness as the mean number of deliveries attended [45], performance change for managing obstetric and other complications [46], change in proportion of deliveries conducted by a skilled birth attendant [47] and change in number of major obstetric surgeries conducted [48]. One study used both number of lives saved and number of Disability Adjusted Life Years (DALYs) [49] (Table 3).

Table 3 Cost-effectiveness of training in economic obstetric care (EmOC)

Walker et al. reported cost-effectiveness/utility as cost per 1% increase in mean skills scores, by comparing the cost-effectiveness across three in-service training programmes in Indonesia [45]. Osei et al. comparing the cost-effectiveness of a traditional residential and a self-paced learning approach, used cost per unit improvement in participant knowledge and skills [46]. The authors noted that the most cost-effective approach to training was dependant on the specific knowledge or skill being taught, and whether opportunity costs were included along with implementation costs in the cost analyses. Osei et al. measured opportunity costs as the value of personnel time both in terms of trainers and participants. Although the self-paced learning approach cost more than the residential approach, this was considered to be more cost-effective than the residential approach with regard to improving knowledge, when direct implementation costs alone were considered (i.e. excluding opportunity cost). When both opportunity and implementation costs were considered, the residential approach proved more cost-effective [46].

Manasyan et al. reported the cost per DALY [49]. In this cost utility analysis, the cost per DALY averted was calculated from the cost per life saved (cost of training divided by the reduction in mortality) divided by the life expectancy in Zambia at the time of study. The authors estimated a ratio of US$5.24 per DALY averted, which when compared with a GDP per person in Zambia of about US$1500 (WHO cost-effectiveness threshold for the country [50]), suggests that the EmOC training intervention could be considered to be good value-for-money [49].

Discussion

Main findings

Overall, 14 studies were identified which conducted an economic evaluation of healthcare provider training in Emergency Obstetric Care. Of these, five were full and nine were partial economic evaluations. Training equipment, per diems and resource person allowances or facilitator fees were the most expensive cost components. When cost estimates were inflated to a constant price year, it cost between I$5 and I$90 to train a participant per day. Training that require participants to stay in accommodation (hotel or other) away from their place of work cost more compared to training which is health facility-based (Range I$33-I$90 vs I$5-I$21, respectively). Comparable effectiveness metrics such as Disability and Quality Adjusted Life Years were rarely used. The methods used in estimating cost-effectiveness varied considerably amongst studies.

Strengths and limitations

To the best of our knowledge, this is the first systematic review of economic evaluations of in-service training in Emergency Obstetric Care. We included both peer-reviewed and grey literature. Meaningful comparisons were possible with regards to costs per trainee per day, using the purchasing power parity equivalents of the training implementation costs. However, it is not possible to do the same for cost-effectiveness, because of the very different measures of effectiveness used in the included studies which ranged from immediate change in knowledge and skills of healthcare providers to the estimated number of lives saved.

Interpretation

Although many implementation programs include training in Emergency Obstetric Care [51,52,53], comparatively few studies have reported costs and/or cost-effectiveness of such trainings. This review illustrates that studies of average or low quality were those conducted as part of other studies. Similar observations were made in a systematic review [33]. The three main reasons for a low quality scores were a failure to; 1) provide a detailed breakdown of implementation costs, 2) include indirect and intangible costs (such as loss of productivity), or 3) describe the perspective of the economic study (government, society or healthcare provider). This information is critical for interpretation of economic evaluations and researchers should be encouraged to capture these details in future to improve the quality of published economic evaluation studies.

The way in which costs are measured and valued can have a substantial impact on the overall cost of an intervention and, therefore, the cost-effectiveness. In all the partial economic evaluations conducted, the costs of implementing the training were calculated using the financial definition of costs (the actual expenditure). Only three studies included ‘opportunity costs’ [44, 46, 48], which allows for more comprehensive economic analyses. Although this leads to higher overall costs, studies that fail to include opportunity costs are likely to report significantly favourable (rather than actual) cost-effectiveness [54]. With regard to in-service training of healthcare providers, this is particularly important. The opportunity costs associated with healthcare providers spending time away from providing clinical services, can be significant [55]. There is value in estimating both financial and economic costs since the former is the basis for budgeting and the latter is useful for robust full economic evaluations [56]. Finally, transparency regarding all of the costs will help researchers and policy makers to better identify areas where savings can be potentially made to reduce the overall cost of training can be reduced, thereby increasing the value-for-money.

Implications for practice

For the studies included in the review with relevant information available, it is clear that training equipment, per diems and resource person allowance account for the majority of the costs [40, 41, 46]. Possible cost saving strategies include sourcing training equipment in bulk centrally, establishing multi-purpose skills training rooms or laboratories which can be used to train multiple groups of healthcare providers [17, 57]. Introduction of a “no per diem” policy and paying only for subsistence costs would significantly reduce the cost of training in many settings [42]. There is a need to eradicate a culture where training has become “an opportunity to supplement income”, rather than an opportunity to build professional capacity [42]. Using volunteers, who have the requisite expertise to provide training and ensuring on-the-job training and continuous clinical education is an inherent part of senior staff roles and responsibilities, will save on ‘consultant’ fees [17].

Training which requires boarding of participants (residential or “hotel-based” training) costs significantly more compared to facility-based training. A recent systematic review on impact of Emergency Obstetric Care training showed that there is gathering interest in, and preference for, ‘facility-based’ or ‘on-site’ or ‘in-house’ training [53]. In addition to reducing the cost per trainee per day, this approach increases the potential for scale-up and is more likely to be sustainable.

The use of standard effectiveness measures allows for comparison of cost-effectiveness to be made across interventions. Although effectiveness measures of patient outcomes, are considered as the ‘real’ benefit of providing Emergency Obstetric Care training. [53, 58]. It is methodologically challenging to measure this [59, 60]. Similarly, especially where training is only one component of a larger implementation program, attribution is problematic [61, 62]. It may be methodologically correct to measure the primary outcome to the healthcare provider (knowledge, skills, competency) more systematically across settings for different types of training.

Conclusion

In-service training in Emergency Obsteric Care is considered to be an effective way to improve knowledge and skills of healthcare providers, which should improve performance, lead to better recognition and management of women who have complications during and after pregnancy or at the time of birth and can potentially reduce morbidity and mortality. In this respect, the wider health, social and economic benefits resulting from relatively small investments in training can be substantial, suggesting that these investments are likely to be good value-for-money [63]. The findings from this review underscore the need for more cost-effectiveness studies while strategically exploring approaches that maximise cost-savings for implementation.

Abbreviations

ALSO:

Advanced Life Support in Obstetrics

AMDD:

Averting Maternal Death and Disability

CBA:

Cost-Benefit analysis

CEA:

Cost-Effectiveness Analysis

CHEERS:

Consolidated Health Economic Evaluation Reporting Standards

CMA:

Cost Minimisation Analysis

CUA:

Cost-Utility Analysis

DALYs:

Disability Adjusted Life Years

JSI:

John Snow International

MHTF:

Maternal Health Task Force

MNH:

Maternal and Newborn Health

MOET:

Managing Obstetric Emergencies and Trauma

SDGs:

Sustainable Development Goals

UNFPA:

United Nations Fund for Population

UNICEF:

United Nations Children’s’ Fund

WHO:

World Health Organization

References

  1. 1.

    World Health Organization, UNICEF, UNFPA, World Bank Group, The United Nations Population Division. Trends in Maternal Mortality: 1990 to 2015. Geneva: World Health Organization; 2015. http://www.who.int/reproductivehealth/publications/monitoring/maternal-mortality-2015/en/

  2. 2.

    WHO. Neonatal mortality: situation and trends. Global Health Observatory (GHO) data. 2016. http://www.who.int/gho/en/

  3. 3.

    WHO. Stillbirths. Maternal, newborn, child and adolescent health; 2016. http://www.who.int/maternal_child_adolescent/en/

  4. 4.

    Maine D, Rosenfield A. The safe motherhood initiative: why has it stalled? Am J Public Health. 1999;89:480–2.

  5. 5.

    The Partnership for Maternal Newborn and Child Health. A Global Review of the Key Interventions Related to Reproductive, Maternal, Newborn and Child Health (RMNCH). Geneva: The Partnership for Maternal Newborn and Child Health; 2011. http://apps.who.int/medicinedocs/documents/s21666en/s21666en.pdf

  6. 6.

    Adegoke AA, van den Broek N. Skilled birth attendance-lessons learnt. BJOG. 2009;116:S33–40.

  7. 7.

    Paxton A, Maine D, Freedman L. The evidence for emergency obstetric care. Int J Gynaecol Obstet. 2005;88:181–93.

  8. 8.

    Holmer H, Oyerinde K, Meara J, Gillies R, Liljestrand J, Hagander L. The global met need for emergency obstetric care: a systematic review. BJOG. 2015;122:183–9.

  9. 9.

    Yakoob MY, Ali MA, Ali MU, Imdad A, Lawn JE, van den Broek N, et al. The effect of providing skilled birth attendance and emergency obstetric care in preventing stillbirths. BMC Public Health. 2011;11:S7.

  10. 10.

    Ni Bhuinneain G, McCarthy FA. Systematic review of essential obstetric and newborn care capacity building in rural sub-Saharan Africa. BJOG. 2015;122:174–82.

  11. 11.

    Penny S. Training initiatives for essential obstetric care in developing countries: a “state of the art” review. Health Policy Plann. 2000;15:386–93.

  12. 12.

    Ayres-de-Campos D, Deering S, Siassakos D. Sustaining simulation training programmes - experience from maternity care. BJOG. 2011;118:S22–6.

  13. 13.

    Beasley JW, Damos JR, Roberts RG, Nesbitt TS. The advanced life support in obstetrics course. A national program to enhance obstetric emergency skills and to support maternity care practice. Arch Fam Med. 1994;3:1037–41.

  14. 14.

    Johanson R, Cox C, O’Donnell E, Grady K, Howell C, Jones P. Managing obstetric emergencies and trauma (MOET): structured skills training using models and reality-based scenarios. Obstet Gynaecol. 1999;1:46–52.

  15. 15.

    Freeth D, Ayida G, Berridge EJ, Mackintosh N, Norris B, Sadler C, et al. Multidisciplinary obstetric simulated emergency scenarios (MOSES): promoting patient safety in obstetrics with teamwork-focused interprofessional simulations. J Contin Educ Heal Prof. 2009;29:98–104.

  16. 16.

    Frank K, Lombaard H, Pattinson R. Does completion of the essential steps in managing obstetric emergencies (ESMOE) training package result in improved knowledge and skills in managing obstetric emergencies? South African J Obstet. 2009;15:94–9.

  17. 17.

    Ameh CA, van den Broek N. Making it happen: training health-care providers in emergency obstetric and newborn care. Best Pract Res Clin Obstet Gynaecol. 2015;29:1077–91.

  18. 18.

    Shoushtarian M, Barnett M, McMahon F, Ferris J. Impact of introducing practical obstetric multi-professional training (PROMPT) into maternity units in Victoria, Australia. BJOG. 2014;121:1710–8.

  19. 19.

    van Lonkhuijzen L, Dijkman A, van Roosmalen J, Zeeman G, Scherpbier AA. Systematic review of the effectiveness of training in emergency obstetric care in low-resource environments. BJOG. 2010;117:777–87.

  20. 20.

    Evans CL, Johnson P, Bazant E, Bhatnagar N, Zgambo J, Khamis AR. Competency-based training “helping mothers survive: bleeding after birth” for providers from central and remote facilities in three countries. Int J Gynecol Obstet. 2014;126:286–90.

  21. 21.

    Bogne V, Kirkpatrick C, Englert Y. Simulation training in the management of obstetric emergencies. A review of the literature. Rev Med Brux. 2014;35:491–8.

  22. 22.

    Ameh CA, Kerr R, Madaj B, Mdegela M, Kana T, Jones S, et al. Knowledge and skills of healthcare providers in sub-Saharan Africa and Asia before and after competency-based training in emergency obstetric and early newborn care. PLoS One. 2016;11:e0167270.

  23. 23.

    Grady K, Ameh C, Adegoke A, Kongnyuy E, Dornan J, Falconer T, et al. Improving essential obstetric and newborn care in resource-poor countries. J Obstet Gynaecol. 2011;31:18–23.

  24. 24.

    Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care Programmes. Oxford: Oxford University Press; 2015.

  25. 25.

    Greenhill R, Prizzon A. Who foots the bill after 2015? Working paper 360: what new trends in development finance mean for the post-MDGs. London: ODI; 2012. https://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/7905.pdf

  26. 26.

    Banke-Thomas AO, Madaj B, Charles A, van den Broek N. Social return on investment (SROI) methodology to account for value for money of public health interventions: a systematic review. BMC Public Health. 2015;15:582.

  27. 27.

    Nixon J. Summarising economic evaluations in systematic reviews: a new approach. BMJ. 2001;322:1596–8.

  28. 28.

    Carande-Kulis VG, Maciosek MV, Briss PA, Teutsch SM, Zaza S, Truman BI, et al. Methods for systematic reviews of economic evaluations for the guide to community preventive services. Am J Prev Med. 2000;18:75–91.

  29. 29.

    Moher D, Liberati A, Tetzlaff J, Altman DG. The PRISMA group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.

  30. 30.

    Wilczynski NL, Haynes RB, Lavis JN, Ramkissoonsingh R, Arnold-Oatley AE. Optimal search strategies for detecting health services research studies in MEDLINE. Can Med Assoc J. 2004;171:1179–85.

  31. 31.

    Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated health economic evaluation reporting standards (CHEERS) statement. BJOG. 2013;120:765–70.

  32. 32.

    Stone GA, Hutchinson AB, Corso PS, Teutsch SM, Feilding JF, Carande-Kulis VG. Understanding and Using the Economic Evidence. In: Zaza S, Briss PA, Harris KW, editors. Task Force on Community Preventive Services. The Guide to Community Preventive Services. New York: Oxford University Press; 2005. p. 449–63.

  33. 33.

    Mangham-Jefferies L, Pitt C, Cousens S, Mills A, Schellenberg J. Cost-effectiveness of strategies to improve the utilization and provision of maternal and newborn health care in low-income and lower-middle-income countries: a systematic review. BMC Pregn Childb. 2014;14:243.

  34. 34.

    United Nations. Purchasing power parities (PPP) conversion factor, local currency unit to international dollar. Series data. 2015; http://unstats.un.org/unsd/mdg/Metadata.aspx?IndicatorId=0&SeriesId=699

  35. 35.

    Cheung Y-W. Purchasing power parity. In: Reinert KA, Rajan RS, Glass AJ, Davis LS, editors. The Princeton encyclopedia of the world economy. Princeton, NJ: Princeton University Press; 2009. p. 942–4.

  36. 36.

    Gill Z, Ahmed JU. Experience from Bangladesh: implementing emergency obstetric care as part of the reproductive health agenda. Int J Gynaecol Obstet. 2004;85:213–20.

  37. 37.

    Islam MT, Haque YA, Waxman R, Bhuiyan AB. Implementation of emergency obstetric care training in Bangladesh: lessons learned. Reprod Health Matters. 2006;14:61–72.

  38. 38.

    Santos C, Diante D, Baptista A, Matediane E, Bique C, Bailey P. Improving emergency obstetric care in Mozambique: the story of Sofala. Int J Gynaecol Obstet. 2006;94:190–201.

  39. 39.

    Rana TG, Chataut BD, Shakya G, Nanda G, Pratt A, Sakai S. Strengthening emergency obstetric care in Nepal: the Women’s right to life and health project (WRLHP). Int J Gynaecol Obstet. 2007;98:271–7.

  40. 40.

    Oyesola R, Shehu D, Lkeh A, Maru I. Improving emergency obstetric care at a state referral hospital, Kebbi state, Nigeria. Int J Gynecol Obstet. 1997;59:S75–81.

  41. 41.

    Mekbib T, Kassaye E, Getachew A, Tadesse T, Debebe A. The FIGO save the mothers initiative: the Ethiopia–Sweden collaboration. Int J Gynecol Obstet. 2003;81:93–102.

  42. 42.

    Crofts JF, Mukuli T, Murove BT, Ngwenya S, Mhlanga S, Dube M, et al. Onsite training of doctors, midwives and nurses in obstetric emergencies, Zimbabwe. Bull World Health Organ. 2015;93:347–51.

  43. 43.

    Chukudebelu W, Ikeme A, Okaro J, Egbuciem P, Onah B, Okeke V, et al. Involving the private sector in improving obstetric care, Anambra state, Nigeria. Int J Gynecol Obstet. 1997;59:S107–12.

  44. 44.

    Yau CWH, Pizzo E, Morris S, Odd DE, Winter C, Draycott TJ. The cost of local, multi-professional obstetric emergencies training. Acta Obstet Gynecol Scand. 2016;95:1111–9.

  45. 45.

    Walker D, McDermott JM, Fox-Rushby J, Tanjung M, Nadjib M, Widiatmoko D, et al. An economic analysis of midwifery training programmes in South Kalimantan, Indonesia. Bull World Health Organ. 2002;80:47–55.

  46. 46.

    Osei I, Garshong B, Banahene G, Gyapong J, Tapsoba P. Improving the Ghanaian Safe Motherhood Programme. Evaluating the effectiveness of alternative training models and other performance improvement factors on the quality of maternal care and client outcomes. Washington D.C.: Frontiers in Reproductive Health, Population Council; 2005.

  47. 47.

    Boulenger S, Dmytraczenko T. Cost of family care international’s skilled care initiative in Kenya and Tanzania. Bethesda, MD: Abt Associates; 2007. http://www.familycareintl.org/UserFiles/File/SCI%20Costing%20Analysis.pdf

  48. 48.

    Kruk ME, Pereira C, Vaz F, Bergström S, Galea S. Economic evaluation of surgically trained assistant medical officers in performing major obstetric surgery in Mozambique. BJOG. 2007;114:1253–60.

  49. 49.

    Manasyan A, Chomba E, McClure EM, Wright LL, Krzywanski S, Carlo WA. Cost-effectiveness of essential newborn care training in urban first-level facilities. Pediatrics. 2011;127:e1176–81.

  50. 50.

    Sachs JD, Ahluwalia IJ, Amoako KY, Aninat E, Cohen D, Diabre Z, et al. Macroeconomics and health: investing in health for economic development. Geneva: world health. Organization. 2001; http://apps.who.int/iris/bitstream/10665/42435/1/924154550X.pdf

  51. 51.

    Evans CL, Maine D, McCloskey L, Feeley FG, Sanghvi H. Where there is no obstetrician--increasing capacity for emergency obstetric care in rural India: an evaluation of a pilot program to train general doctors. Int J Gynaecol Obstet. 2009;107:277–82.

  52. 52.

    Ameh C, Adegoke A, Hofman J, Ismail FM, Ahmed FM, van den Broek N. The impact of emergency obstetric care training in Somaliland, Somalia. Int J Gynecol Obstet. 2012;117:283–7.

  53. 53.

    Bergh A-M, Baloyi S, Pattinson RC. What is the impact of multi-professional emergency obstetric and neonatal care training? Best Pract Res Clin Obstet Gynaecol. 2015;29:1028–43.

  54. 54.

    Posnett J, Jan S. Indirect cost in economic evaluation: the opportunity cost of unpaid inputs. Health Econ. 1996;5:13–23.

  55. 55.

    Campbell J, Dussault G, Buchan J, Pozo-Martin F, Guerra AM, Leone C, et al. A universal truth: no health without a workforce. Forum report, third global forum on human resources for health, Recife, Brazil. Geneva: Global Health workforce alliance and world health. Organization. 2013;

  56. 56.

    Saronga HP, Dalaba MA, Dong H, Leshabari M, Sauerborn R, Sukums F, et al. Cost of installing and operating an electronic clinical decision support system for maternal health care: case of Tanzania rural primary health centres. BMC Health Serv Res. 2015;15:132.

  57. 57.

    Utz B, Kana T, van den Broek N. Practical aspects of setting up obstetric skills laboratories - a literature review and proposed model. Midwifery. 2015;31(4):400–8. https://doi.org/10.1016/j.midw.2014.11.010.

  58. 58.

    Draycott TJ, Collins KJ, Crofts JF, Siassakos D, Winter C, Weiner CP, et al. Myths and realities of training in obstetric emergencies. Best Pract Res Clin Obstet Gynaecol. 2015;29:1067–76.

  59. 59.

    Arnesen T, Nord E. The value of DALY life: problems with ethics and validity of disability adjusted life years. BMJ. 1999;319:1423–5.

  60. 60.

    AbouZahr C. Disability adjusted life years (DALYs) and reproductive health: a critical analysis. Reprod Health Matters. 1999;7:118–29.

  61. 61.

    Calvert KL, McGurgan PM, Debenham EM, Gratwick FJ, Maouris P. Emergency obstetric simulation training: how do we know where we are going, if we don’t know where we have been? Aust New Zeal. J Obstet Gynaecol. 2013;53:509–16.

  62. 62.

    Merién AER, van de Ven J, Mol BW, Houterman S, Oei SG. Multidisciplinary team training in a simulation setting for acute obstetric emergencies. Obstet Gynecol. 2010;115:1021–31.

  63. 63.

    Collins KJ, Draycott TJ. Skills and drills: are they worth the effort? Obstet Gynaecol Reprod Med. 2015;25:372–4.

Download references

Funding

Department for International Development (DFID/UKAid) for the ‘Making it Happen’ programme (Grant number 202945–101). DFID/UKAid had no role in the design of the study, in collection, analysis, and interpretation of data and in writing the manuscript.

Availability of data and materials

The dataset(s) supporting the conclusions of this article is(are) included within the article (and its additional file(s)).

Author information

MWJ, BM and NvdB conceptualised the review. MWJ and ABT conducted literature searches. ABT, MWJ, BM and NvdB reviewed the included studies. Data synthesis was conducted by ABT, MWJ and NvdB. All authors were involved in writing the manuscript and approved the final version.

Correspondence to Nynke van den Broek.

Ethics declarations

Ethics approval and consent to participate

Ethics approval was not needed for this review.

Consent for publication

Not applicable.

Competing interests

BM and NvdB are involved in EmOC training programmes in 12 developing countries as part of the ‘Making it Happen’ programme.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1: Table S1.

Summary of included studies (DOCX 23 kb)

Additional file 2: Table S2.

Quality assurance of full economic evaluations (DOCX 19 kb)

Additional file 3: Table S3.

Quality assessment of cost analysis in partial and full economic evaluations (DOCX 15 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Keywords

  • Emergency obstetric care
  • Training
  • Economic evaluation
  • Cost analysis
  • Cost-effectiveness analysis
  • Cost-utility analysis
  • Cost-benefit analysis
  • Value-for-money