Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Indicators for monitoring maternal and neonatal quality care: a systematic review

BMC Pregnancy and Childbirth volume 19, Article number: 25 (2019) Cite this article

Abstract

Background

Research and different organizations have proposed indicators to monitor the quality of maternal and child healthcare, such indicators are used for different purposes.

Objective

To perform a systematic review of indicators for the central phases of the maternal and child healthcare continuum of care (pregnancy, childbirth, newborn care and postpartum).

Method

A search conducted using international repositories, national and international indicator sets, scientific articles published between 2012 and 2016, and grey literature. The eligibility criteria was documents in Spanish or English with indicators to monitor aspects of the continuum of care phases of interest.

The identified indicators were characterized as follows: formula, justification, evidence level, pilot study, indicator type, phase of the continuum, intended organizational level of application, level of care, and income level of the countries. Selection was based on the characteristics associated with scientific soundness (formula, evidence level, and reliability).

Results

We identified 1791 indicators. Three hundred forty-six were duplicated, which resulted in 1445 indicators for analysis. Only 6.7% indicators exhibited all requirements for scientific soundness. The distribution by the classifying variables is clearly uneven, with a predominance of indicators for childbirth, hospital care and facility level.

Conclusions

There is a broad choice of indicators for maternal and child healthcare. However, most indicators lack demonstrated scientific soundness and refer to particular continuum phases and levels within the healthcare system. Additional efforts are needed to identify good indicators for a comprehensive maternal and child healthcare monitoring system.

Peer Review reports

Background

Maternal and child health (MCH) has been a priority public health problem for decades [1]. Since the 1990s, the international community has implemented important initiatives to reduce the morbidity and mortality of mothers and newborn infants. We highlight the Millennium Development Goals (MDG) [2], as well as the current Sustainable Development Goals (SDG) that reinforce good health and well-being for women and children (Goal 3), coupled with the target of universal health coverage [3] and reflected in the renewed Global Strategy for Women’s, Children’s and Adolescents’ Health (2016–2030) [4]. MCH care frameworks to integrate and improve health system performance have simultaneously been proposed as the so-called continuum of care for maternal, newborn, and child health. This continuum has been presented as a rallying call to reduce the toll of maternal, newborn and child deaths, and has reached a solid international acceptance [5].

The initial main improvement strategy has focused on access to health services and increasing the number of births in health facilities, as well as the number of births attended by qualified health personnel. However, the improvement in accessibility has not yielded a similar reduction in the morbidity and mortality of mothers and newborns, thus bringing to the foreground the importance of the quality of care received [6, 7] and its measurement, which indicate the period around childbirth as the most important.

The emphasis on quality in several international initiatives has favored the construction and measurement of indicators [8,9,10,11,12]. Indicators of quality as “measurement tools that can be used to monitor, evaluate and improve the quality of patient care, organization and support services that affect patient outcomes” [13]. The consensus is that measurement of performance is essential to support improvement and accountability. However, this has unleashed a multitude of uncoordinated and often duplicative measurement and reporting initiatives [12]. Different international organizations, civil society groups, academics, and countries have proposed indicators. These indicators are used for different purposes and often examine only a component of the attention; rarely are the different initiatives integrated and coordinated [8, 9, 12, 14]. As a result, in the current situation, many initiatives co-exist; however, the total number and characteristics of the available indicators, as well as the particular technical components on which they were built, remain unknown [12, 14].

In this context, it appears relevant to perform a search and systematic review of the existing indicators for the most important components of the continuum of care (pregnancy, childbirth, newborn and puerperium), using explicit and homogeneous criteria in relation to their scientific soundness and unambiguous definitions for their application according to the levels of responsibility within the health system.

Methods

This study was conducted in two stages: 1) a systematic search for maternal and neonatal quality of care indicators related to four of the phases of the continuum of care: pregnancy, childbirth, puerperium and newborn up to the first two months of life; 2) characterization, classification and selection of the indicators with explicit criteria on scientific soundness, including complete description, validity with explicit level of evidence, tested reliability and feasibility, and proposed applicability.

Systematic search strategy and eligibility criteria

The searches were conducted using four sources: 1) international repositories of indicators; 2) compendiums or established national and international indicator sets; 3) grey literature; and 4) scientific articles, from 2012 to August 2016. We considered only indicators in Spanish and English. The search in repositories was conducted in the National Quality Measures Clearinghouse (NQMC) [15]; National Quality Forum (NQF) [16] and MEASURE Evaluation [17]. The selected compendiums or indicator sets were as follows: Inpatient Quality Measures [18] and Core Measures [19] of The Joint Commission; indicators of Medicare and Medicaid Services [20]; General Practice Quality and Outcomes Framework [21] and Clinical Commissioning Group Outcomes Indicator Set [22]; Key indicators of the national health system of Spain [23]; National quality indicators in the health system of Mexico [24]; European Core Health Indicators [25]; and indicators of the Organization for the Economic Cooperation and Development (OECD) [26]. The search of scientific articles was conducted in the PubMed database, whereas the grey literature search was performed through the search engine Google. The search terms used in the indicator compendiums, sets and repositories were as follows: Neonatal; Newborn; Neonate; Infant; Premature; Preterm; Birth; Childbirth; Delivery; Labor; ; Natal; Postnatal; Perinatal; Prenatal; Partum; Postpartum; Peripartum; Intrapartum; Prepartum; Antepartum; Pregnancy; Pregnant; Maternal; Gestation; and Gestational. The PubMed search strategy is in the Additional file 1. In Google, we used the terms “maternal health indicators”, “newborn care indicators”, “birth indicators”, “delivery indicators”, and “pregnancy control indicators”. In all searches, the homologous terms in Spanish were also employed.

To select scientific articles, we initially reviewed the titles and abstracts of all publications obtained in the search and maintained the articles that included indicators related to some aspect of the targeted phases of the continuum. Articles that addressed other phases of the continuum, did not describe the use of indicators, or were part of one of the analyzed compendiums and repertoires were discarded. In the grey literature, we extracted documents that included a set of indicators on MCH endorsed by organizations with international recognition, such as the WHO, UNICEF, and OECD. Additional file 2 shows the complete list of references used in the research by source (repositories, compendiums, scientific articles and grey literature).

Characterization and classification of the indicators

The reference framework is the phases of the continuum of care of interest, and the type of indicator and aspect of care measured [5, 27,28,29], as depicted in Fig. 1. To analyse, characterize and select them, we use the variables described in Table 1. The indicators found in the search were entered into a database that identified: full name; numerator and denominator or equivalent (format type if/then) [30]; type of indicator according to the Donabedian model [27] (structure, process, outcome) with an additional option for those not directly related to health care (determinants of health and demographic and social statistics) [28, 29]; phase of the continuum to which they refer [5]; referenced scientific justification; explicit level of evidence; existence of pilot study; organizational level of application within the health system; level of care for which the indicators are used; and the income level [31] of the country or countries for which they are proposed (Table 1).

Fig. 1
figure 1

Framework for grouping indicators according to type, aspect of care measured, and phase of the maternal and childcare continuum

Full size image
Table 1 Variables used for the characterization of indicators
Full size table

Two researchers independently extracted information on each indicator and discussed the discrepancies. In cases of unsolved discrepancies, a third senior researcher was consulted.

The analyzed indicators were grouped in relation to the intersection of the phase of the continuum to which they relate, and the main characteristics of the indicator: type of indicator and the activity or measurement objective (Fig. 1), level of care, application level, and income level of the country of intended use. In relation to the continuum, the indicators that applied to more than one of the four targeted phases were classified as “other”, whereas the indicators that also applied to infants older than 2 months of age (up to 18 years old) were considered a different category.

Selection of indicators

The steps for analysis and selection are summarized in Fig. 2b. Given our objective of analyzing indicators focused on the performance of the health services, and more specifically healthcare quality, we discarded the group of indicators not directly related to healthcare (health determinants and other social and demographic statistics) for further analysis. For the remaining indicators, we analyzed the presence of the desirable characteristics associated with scientific soundness [28, 29, 32] (complete description, referenced explicit evidence, and reliability), as well as feasibility demonstrated by pilot testing [33, 34]. The strength of evidence determine validity, which is the degree to which an indicator measures what it is intended to measure, and whether the results of a measurement corresponds to the true state of the phenomenon being measured [33]. Ultimately, validity determines the likelihood that improvement in the indicator will produce consistent and credible improvements in the quality of care [33]. The reliability of a measure is also necessary for validity [34]. Therefore, to assess the validity of an indicator we look for the explicit reference to the level of evidence, and proven reliability.

Fig. 2
figure 2

Systematic review process. a Systematic search of indicators. b. Review, selection and classification of indicators

Full size image

Consequently, we discarded first the indicators that did not have complete formula, and then, in the case of structure and process indicators, we discarded indicators without reference to support evidence and those without explicit level of evidence. Finally, we considered whether they had been pilot tested, documenting reliability and feasibility, and had an operational description on their application, calculation and interpretation.

For outcome indicators, we discarded also those without complete formula, then those without pilot test for feasibility and reliability, and then those that did not have the full operational description for measurement (for example, when the indicator mentions “low birth weight”, the weight that must be considered “low” for the gestational age is not identified).

In cases of similar (non-contradictory) definitions for the same type of indicator, we selected the most complete or up-to-date information and, preferably, those endorsed by internationally recognized organizations (i.e., the WHO, OECD and Eurostat). When we found different definitions for the same indicator (as it was the case for instance for neonatal mortality and also for perinatal mortality), we kept both for the analysis.

Finally, indicators that apply to infants up to 18 years old were also discarded to increase specificity.

The results of the review include evidence-based, reliable, feasible and pilot-tested indicators, which are grouped according to the main classification axes (type of indicator, activities or aspects measured, and phase of the continuum that apply), as well as by the level of care (primary, hospital, or both), organization within the health system (unit or service, health facility, supra-facility, population, or system), and economic level of the country for which they are proposed.

Results

The general outline of the study and the overall results are described in Fig. 2, depicted in a PRISMA Statement-based flow diagram [35]. From an initial selection of 1445 indicators, we identified 87 (6.7%, after discarding indicators on health determinants and statistics) that comply with the requirements of full description and empirically tested validity, reliability and feasibility.

Identification of indicators for analysis (Fig. 2a)

We identified in repositories a total of 3538 indicators (2528 in NQMC; 533 in NQF and 477 in MEASURE Evaluation), of which 782 were related to the phases of the continuum of care of interest. Of these indicators, 514 were duplicated, thus resulting in 268 indicators for further analysis (Fig. 2a).

National and international indicator sets yielded 837 indicators, of which 51 indicators evaluate the care of pregnancy, childbirth, puerperium and the newborn and were kept for analysis.

One hundred twenty-five of the 387 articles identified in the literature search were related to the population groups and topics of interest, according to their abstracts. After reviewing the full text, 58 articles were excluded because they did not describe indicators, the indicators were not related to the target phases of the continuum, they used indicators from the analyzed repositories, or they were duplicated. In some cases, the full text was in a language other than English or Spanish. Eventually, we identified 872 indicators in the 67 full text articles reviewed.

We identified 14 grey literature documents, and 600 indicators were extracted for analysis.

The initial database with the indicators extracted from all sources contained 1791 indicators. Three hundred forty-six were duplicated and discarded, thus resulting in 1445 indicators for further analysis (Fig. 2a).

Characteristics and classification of the analyzed indicators

Process indicators predominate in all phases of the continuum (Table 2). They represent 51% of the indicators and 75% of the indicators related to pregnancy. Childbirth and Pregnancy were the phases of the continuum for which more indicators were identified, i.e., 299 and 297 indicators respectively compared to 119 indicators related to the postpartum period and 277 indicators related to newborn care.

Table 2 Characteristics of the indicators included in the analysis by phase of the continuum
Full size table

In relation to the level of care, 43.1% of indicators are related to hospital care. Indicators applicable to primary care represent 24.2% of the indicators found, and are mainly for pregnancy care (58.6% of the total) and, to a lesser extent, puerperium (26.9%).

In relation to the organizational level, the majority of indicators are intended for use at health facility level, particularly in the phases related to puerperium (60.5%) and childbirth (53.9%), whereas the remaining indicators are intended for use at the supra-institutional or population level level (Table 2).

In relation to the income level of the countries in which they are applied, there is a slight predominance of middle and low-income countries in general (36.9%), even though high-income countries predominate in indicators related to pregnancy, puerperium, and newborn (Table 2).

According to the object of measurement (Table 3), preventive activities represent 16.2% of the total number of indicators and are the relative majority in the Pregnancy and Newborn phases of the continuum. Indicators related to the mortality and morbidity of newborns are more frequent in the outcomes group. It is relatively remarkable the number of structure indicators related to policy and the context of the system of health (7.3%), as well as human and material resources (9.4% of the total).

Table 3 Indicators included in the analysis according to the phase of the continuum and the object of measurement
Full size table

Selection of valid, reliable and pilot-tested indicators

After the application of the criteria on validity, reliability, feasibility and pilot testing (Fig. 2b, Table 4), 87 of the 1445 analyzed indicators were selected. Databases with all indicators analyzed are available online.

Table 4 Classification of the indicators selected in the review (n = 87)
Full size table

We initially discarded 146 indicators not directly related to health care (social determinants and demographic statistics). More than half (781, 60%) of the remaining 1299 indicators did not report their formula, which made their reproducibility impossible. However, the differences between the indicators extracted from different sources are noteworthy: all indicators from repositories and 89.1% of indicators extracted from systems or official indicator sets describe their formulas compared with less than half of the indicators identified from the grey literature and only 32.3% of the indicators obtained from scientific publications.

In the groups of structure and process indicators with formulas, 73 (19.7%) indicators were discarded because there was no reference to support their evidence, and 261 (an additional 70.4%) indicators were discarded because the level of evidence was not explicitly stated. Referenced and explicit evidence are the first and key criteria for validity.

In the outcome group, 39 (26.5%) of the 147 indicators with formula were duplicated or overlapped, and 26 (24.1%) of the remaining indicators were discarded because of the absence of a pilot study to demonstrate feasibility and reliability, or clear operational description of their application, calculation and interpretation.

Finally, to increase specificity we discarded 26 indicators that applied to an age range of 0 to 18 years. The final selection includes 31 process indicators and 56 outcome indicators (Fig. 2b, Table 4).

A more detailed analysis of the selected indicators, in view of their potential use for monitoring the quality of maternal and perinatal care, indicates certain imbalances and gaps (Table 4). Thus, they do not cover all possible categories according to the type of indicator and the object of measurement in any of the considered phases of the continuum. Most indicators relate to childbirth or newborns; however, very few relate to puerperium. The majority are applicable at the hospital level, and only a limited number exclusively relate to primary care. In addition, the majority are proposed for assessing quality at the facility level and for high-income countries, with a limited number for middle and low-income environments only.

The low proportion of indicators selected (6.7% of the analyzed indicators, after discarding indicators on health determinants and statistics) reflects a low level of scientific soundness and proven feasibility of the published indicators. This proportion is, nevertheless, uneven according to the phase of the continuum and the level of application within the health system. Calculating these proportions using data from Table 4 and Table 2, they are somehow higher for the indicators that apply to childbirth (9.7%), hospital care (8.7%), and department or service unit (12.2%). Indicators related to pregnancy are one of the largest group of extracted and analyzed indicators (Table 2); however, they are also one of the lowest percentages of indicators that comply with the criteria used in the review (2.7%). Puerperium, primary healthcare, and supra-institutional or system level indicators yield the lowest percentage of scientifically sound indicators. In particular, for primary health care, only 6 of the 311 analyzed (1.9%) indicators were selected.

Additional file 3 provides the description and reference documents of the 87 selected indicators in English and Spanish, ordered by the phase of the continuum.

Discussion

This review provides an abundance of initiatives, frameworks and indicators to monitor the quality of pregnancy, childbirth, postpartum and newborn care. However, in terms of proven scientific soundness, the quality is largely poor. Moreover, indicators that meet the requirements do not cover all aspects expected and only relate to parts of the continuum, levels of care, such as primary care, and levels of application within the system, such as the supra-institutional, where there is a dearth of indicators with the necessary guarantees to consider their adaptation and routine use.

Abundance of indicators, but limited scientific rigor

Doctors, administrators, policy makers and patients require reliable and valid information to perform comparative evaluations, make judgments, determine priorities and improve the quality of care [33]. Therefore, there is a need for indicators that are easy to interpret, reasonable, validated and adapted to the characteristics of the context of each country [8]. In the pursuit of the achievement of the MDG, as well as the SDG, the need to assess the quality of care has been strongly acknowledged [8, 9, 11, 36]. This has motivated and given momentum to the development and proposal of indicators [8, 9, 37,38,39,40]. However, according to our review, more than half of the indicators are not properly reported, and the vast majority of the remaining indicators lack the necessary methodological rigor. Another review, that included only indicators proposed by global multi-stakeholder groups, found that 25% of all indicators is either under development or requires a clear definition and methodology [40]. A common feature is the use of consensus techniques to select indicators, which is to the detriment of proven and explicit criteria to ensure their scientific robustness. These criteria are often described as desirable and a guide for the selection; however, this selection is eventually left to the opinion of the consulted experts. Consensus may be important as a final step to guarantee acceptance; however, we suggest that the initial selection should be made on solid scientific grounds. We have determined that this is not the most frequent case.

Three other issues of relevance for the creation of monitoring systems for MCH are worth highlighting because they are not clearly addressed: 1) the desirable integration of the indicators in a coordinated set which takes into account the different levels of responsibility within the health system [41]; 2) the limited attention paid to the importance of the quality of data; and 3) the scarcity of indicators based on data reported by patients. We have shown, as indicated in other publications [9, 12], that the level of detail on the processes addressed for monitoring is often not suitable for routine information systems. Most valid indicators are intended for use at the micro level (service unit or facility) (Table 4), whereas other indicators intended for the system level ignore lower levels in the organization. Metrics may need to vary at different levels of the healthcare system; however, all sets should be aligned. This is not visible in the current situation. In addition, some indicators seem to address coverage of services (i.e. number of antenatal services) rather than direct quality of care (i.e. the right contents of antenatal care or the right clinical decisions for particular MCH conditions).

The quality of the data is not directly addressed in our review. However, the lack of explicit reference to the data required for the calculation of the indicators (the first criterion for discarding them; refer to Fig. 2b), as well as the indicators proposed without pilot tests (particularly among outcome indicators), may be considered as proxies for the limited attention paid to the quality of data required for implementing the indicators.

Patient-reported indicators are considered by international organizations, such as the OECD, as the next generation of health statistics [42], and survey-based indicators on patient experience have a salient place in the current monitoring systems of quality of care in general. However, they are barely present in the final selection in our review. We found 28 indicators measuring satisfaction (see Table 3), but most of them were complex, including more than one phase of the continuum, and they did not have the required criteria for being included in the final selection.

It is striking that indicators identified from the scientific literature are of poorer quality in terms of a lower compliance with the selection criteria established than the extracted indicators from repositories or established systems of measurement (Fig. 2b). This issue primarily arises because authors fail to accurately describe how to measure them, as well as the reduced attention given to explicitly state the level of evidence that supports them. Specialized repositories (NQMC and NQF) are good sources for well-described indicators, as well as indicators developed by agencies, such as the WHO [9, 10], USAID [43] and other agencies [44]. However, we have not identified an extensive and systematic review as the one presented here.

A comprehensive set with significant gaps

As indicated in the total column in Table 4, there are indicators for all considered aspects of the continuum, as well as all levels of attention and levels of responsibility within the health system. However, a complete and coordinated set must be constructed. Incomplete proposals, which referred to a single phase of the continuum or very specific aspects within the same phase, are frequent. The distribution of the indicators by groups is also very uneven. As a result, although the selection we present forms the basis, or first step, to determine and adapt a relatively comprehensive indicator set, there are many significant gaps. There is a persistent need to provide additional efforts to build a good indicator set for MCH care and monitor progress in the SDG [45]. The lack of indicators related to the postnatal period for the mother and, relatively, pregnancy within the continuum is particularly noticeable, which is likely associated with the shortage of fully validated indicators applicable to primary health care and the population level. The apparent abundance of indicators hides the likely need of further progress in the construction and validation of empirically tested and scientifically solid indicators to build a comprehensive and hierarchical system to monitor the quality of MHC care, in all phases of the continuum and at all levels of the health system. Recent initiatives such as the WHO Quality of Care Network for Maternal, newborn, child and adolescent health [46] may contribute to this endeavor.

The search was limited to sources in Spanish and English. There may be repositories and publications in other languages. Moreover, it is possible to search other databases, such as Embase, and attempt to enlarge the grey literature search using the snowball technique. However, we believe that most references for indicators are included in our search and that our results provide a fair view of the current situation.

Conclusions

There is a broad panorama of indicators available internationally for the evaluation of the quality of maternal and newborn care; however, a critical analysis shows that most indicators are not readily suitable for adaptation and implementation. Only a minority of published indicators comply with the requirements of scientific validity, usefulness and feasibility empirically tested.

The indicators identified and selected in our search may comprise a good starting point; however, it is likely that they should be supplemented by new indicators to cover the needs of a comprehensive monitoring system. Our study indicates the specific aspects and levels of care and responsibility for which there is a likely need to make additional efforts in the construction and validation of quality indicators to monitor the continuum of maternal and newborn health.

Abbreviations

MCH:

Maternal and child health

MDG:

Millennium Development Goals

NQF:

National Quality Forum

NQMC:

National Quality Measures Clearinghouse

OECD:

Organization for the Economic Cooperation and Development

SDG:

Sustainable Development Goals

UNICEF:

United Nations International Children’s Emergency Fund

WHO:

World Health Organization

References

  1. World Health Organization. World health report 2005: make every mother and child count. Geneva: World health; 2005.

    Book  Google Scholar 

  2. United Nations. The millennium development goals report 2015. 2015. http://www.undp.org/content/dam/undp/library/MDG/english/UNDP_MDG_Report_2015.pdf. Accessed 24 Sep 2017.

    Google Scholar 

  3. United Nations. Sustain Dev goals. Goal 3: Good health and well-being. http://www.undp.org/content/undp/en/home/sustainable-development-goals/goal-3-good-health-and-well-being.html. Accessed 15 Sep 2017.

  4. World Health Organization. The global strategy for Women’s, Children’s and adolescents health (2016 - 2030). United Nations. 2015. http://www.who.int/life-course/partners/global-strategy/ewec-globalstrategyreport-200915.pdf?ua=1. Accessed 16 Sep 2017.

  5. Kerber KJ, de Graft-Johnson JE, Bhutta ZA, Okong P, Starrs A, Lawn JE. Continuum of care for maternal, newborn, and child health: from slogan to service delivery. Lancet. 2007;370(9595):1358–69.

    Article  Google Scholar 

  6. Scott KW, Jha AK. Putting quality on the Global Health agenda. N Engl J Med. 2014;371(1):3–5.

    Article  CAS  Google Scholar 

  7. Graham WJ, Varghese B. Quality, quality, quality: gaps in the continuum of care. Lancet. 2012;379(9811).

  8. World Health Organization. Standards for improving quality of maternal and newborn care in health facilities. Geneva: World Health Organization; 2016. http://apps.who.int/iris/bitstream/handle/10665/249155/9789241511216-eng.pdf?sequence=1. Accessed 10 Jun 2016

    Google Scholar 

  9. World Health Organization. Consultation on improving measurement of the quality of maternal, newborn and child care in health facilities. Geneva: World Health Organization; 2014.

    Google Scholar 

  10. Countdown to 2015, Health metrics network, UNICEF, World Health Organization. Monitoring maternal, newborn and child health: understanding key progress indicators. Geneva: World Health Organization; 2011.

  11. Murray CJL. Choosing indicators for the health-related SDG targets. Lancet. 2015;386(10001):1314–7.

    Article  Google Scholar 

  12. Institute of Medicine. Vital signs: core metrics for health and health care progress. Washington, D.C.: National Academies Press; 2015.

    Google Scholar 

  13. Saturno P, Martinez-Nicolas I, Robles-Garcia I, López-Soriano F, Angel-García D. Development and pilot test of a new set of good practice indicators for chronic cancer pain management. Eur J Pain. 2015;19:28–38 https://doi.org/10.1002/ejp.516. Accessed 15 Jan 2018.

  14. The Partnership for Maternal Newborn and Child Health. Consensus for Maternal, Newborn and Child Health. The Partnership for Maternal, Newborn and Child Health; 2009. http://www.who.int/pmnch/topics/maternal/consensus_12_09.pdf. Accessed 15 Jan 2018.

  15. Agency for Healthcare Research and Quality (AHRQ). National Quality Measures Clearinghouse. https://www.qualitymeasures.ahrq.gov/. Accessed 15 May 2016.

  16. National Quality Forum. Find Measures. http://www.qualityforum.org/QPS/QPSTool.aspx. Accessed 20 May 2016.

  17. MEASURE Evaluation. Family planning and reproductive health indicators database. Summary List of Indicators https://www.measureevaluation.org/prh/rh_indicators/indicator-summary. Accessed 26 Jul 2018.

  18. The Joint Commission. Performance Measurement. Specifications Manual for National Hospital Inpatient Quality Measures. 2016. https://www.jointcommission.org/performance_measurement.aspx. Accessed 26 Jun 2016.

  19. The Joint Commission. Performance Measurement. Specifications Manual for Joint Commission National Quality Core Measures. 2016. https://www.jointcommission.org/performance_measurement.aspx. Accessed 30 Nov 2016.

  20. Centers for Medicare & Medicaid Services. Core Measure. 2016. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityMeasures/Core-Measures.html. Accessed 3 Jun 2016.

  21. National Institute for Health and Care Excellence (NICE). Standards and Indicators. Single menu of indicators. 2016. https://www.nice.org.uk/standards-and-indicators. Accessed 15 Jun 2016.

  22. NHS Digital. Clinical Commissioning Group Outcomes Indicator Set (CCG OIS). https://indicators.hscic.gov.uk/webview/. Accessed 17 Jun 2016.

  23. Ministerio de Sanidad Servicios Sociales e Igualdad (MSSSI). Indicadores Clave Sistema Nacional de Salud - Versión 1.0. 2016. http://inclasns.msssi.es/main.html. Accessed 10 Jun 2016.

  24. Secretaría de Salud. Dirección General de Calidad y Educación en Salud. Sistema Nacional de Indicadores de Calidad en Salud. 2016. http://dgces.salud.gob.mx/INDICASII/. Accessed 2 Jun 2016.

  25. European Commission. Indicators. European Core Health Indicators (ECHI). 2016. https://ec.europa.eu/health/indicators_data/echi_en. Accessed 20 Jun 2016.

  26. Organisation for Economic Co-operation and Development. OECD health statistics 2015. List of indicators. 2016 http://www.oecd.org/els/health-systems/health-data.htm. Accessed 24 Jun 2016.

  27. Donabedian A. The quality of care. How can it be assesed. J Am Med Assoc. 1988;260(12):1743–8.

    Article  CAS  Google Scholar 

  28. Arah OA, Westert GP, Hurst J, Klazinga NS. A conceptual framework for the OECD health care quality indicators project. Int J Qual Health Care. 2006;18(Suppl 1):5–13.

    Article  Google Scholar 

  29. Carinci F, Van Gool K, Mainz J, Veillard J, Pichora EC, Januel JM, et al. Towards actionable international comparisons of health system performance: expert revision of the OECD framework and quality indicators. Int J Qual Health Care. 2015;27(2):137–46.

    CAS  PubMed  Google Scholar 

  30. Shekelle PG, MacLean CH, Morton SC, Wenger NS. Assessing care of vulnerable elders: methods for developing quality indicators. Ann Intern Med. 2001;135(8 I):647–52.

    Article  CAS  Google Scholar 

  31. World Bank. World Bank: country and lending groups. World Bank Data Website. 2016; https://datahelpdesk.worldbank.org/knowledgebase/articles/906519. Accessed 4 Sep 2016.

  32. World Health Organization. Global Reference List of 100 Core Health Indicators. Geneva; 2015. http://apps.who.int/iris/bitstream/handle/10665/173589/WHO_HIS_HSI_2015.3_eng.pdf?sequence=1. Accessed 9 Jun 2016.

  33. Mainz J. Developing evidence-based clinical indicators: a state of the art methods primer. Int J Qual Health Care. 2003;15(Suppl 1):i5–11.

    Article  Google Scholar 

  34. Rubin HR, Pronovost P, Diette GB. From a process of care to a measure: the development and testing of a quality indicator. Int J Qual Health Care. 2001;13(6):489–96.

    Article  CAS  Google Scholar 

  35. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009 Oct;62(10):e1–34.

    Article  Google Scholar 

  36. Gabrysch S, Civitelli G, Edmond KM, Mathai M, Ali M, Bhutta ZA, et al. New signal functions to measure the ability of health facilities to provide routine and emergency newborn care. PLoS Med. 2012;9(11).

  37. Tunçalp Ӧ, Were W, MacLennan C, Oladapo O, Gülmezoglu A, Bahl R, et al. Quality of care for pregnant women and newborns-the WHO vision. BJOG An Int J Obstet Gynaecol. 2015;122(8):1045–9.

    Article  Google Scholar 

  38. Brenner S, De Allegri M, Gabrysch S, Chinkhumba J, Sarker M, Muula AS. The quality of clinical maternal and neonatal healthcare a strategy for identifying “routine care signal functions”. PLoS One. 2015;10(4):e0123968.

    Article  Google Scholar 

  39. Sibanda T, Fox R, Draycott TJ, Mahmood T, Richmond D, Simms RA. Intrapartum care quality indicators: a systematic approach for achieving consensus. Eur J Obstet Gynecol Reprod Biol. 2013;166(1):23–9.

    Article  Google Scholar 

  40. Moller A-B, Newby H, Hanson C, Morgan A, El Arifeen S, Chou D, et al. Measures matter: a scoping review of maternal and newborn indicators. PLoS One. 2018;13(10):e0204763. https://doi.org/10.1371/journal.pone.0204763.

  41. Saturno-Hernández PJ. Métodos y herramientas para la realización de ciclos de mejora de la calidad en servicios de salud. Cuernavaca, Morelos: Instituto Nacional de Salud Pública; 2015.

    Google Scholar 

  42. OECD. PaRIS: Patient reported indicators survey. The next generation of OECD health Statistics 2016. http://www.oecd.org/els/health-systems/PaRIS-Booklet.pdf. Accessed 4 Jul 2017.

  43. Safe Motherhood and Reproductive Health Working Group and CORE Group. Maternal and Newborn Standards and Indicators Compendium. Washington, D.C; 2004. https://www.mchip.net/sites/default/files/Maternal_and_Newborn_Standards_and_Indicators_Compendium_2004.pdf. Accessed 26 May 2016.

  44. International Federation of Red Cross and Red Crescent Societies. Maternal, newborn and child health Framework 2013. https://www.ifrc.org/PageFiles/93927/1232600-MNCH%20Framework%20report_LR%20(2).pdf. Accessed 26 May 2016.

  45. Marchant T, Bryce J, Victora C, Moran AC, Claeson M, Requejo J, et al. Improved measurement for mothers, newborns and children in the era of the sustainable development goals. J Glob Health. 2016;6(1):010506.

    Article  Google Scholar 

  46. World Health Organization. Maternal, child and adolescent health. Quality of care network. http://www.who.int/maternal_child_adolescent/topics/quality-of-care/network/en/. Accesed 30 Sept 2018.

Download references

Acknowledgements

Not applicable

Funding

This study receive external funding by Consejo Nacional de Ciencia y Tecnología (CONACYT), Project #S0008–2015-2: “Mejora de la calidad en la atención al parto institucional a través de la lista de verificación de parto seguro”. The funder played no role in the execution or writing of this paper.

Availability of data and materials

The database with the indicators identified and analyzed in the search is available in Figshare repository, https://doi.org/10.6084/m9.figshare.7148855 (Tab 1 includes 1791 indicators and Tab 2 includes 1445 indicators).

Author information

Authors and Affiliations

  1. Centro de Investigación en Evaluación y Encuestas, Instituto Nacional de Salud Pública, Universidad No. 655 Colonia Santa María Ahuacatitlán, C.P 62100, Cuernavaca, Morelos, Mexico

    Pedro J. Saturno-Hernández, Ismael Martínez-Nicolás, Estephania Moreno-Zegbe, María Fernández-Elorriaga & Ofelia Poblano-Verástegui

Authors
  1. Pedro J. Saturno-Hernández
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ismael Martínez-Nicolás
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Estephania Moreno-Zegbe
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. María Fernández-Elorriaga
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ofelia Poblano-Verástegui
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

P.J.S.H. developed the study protocol; extracted data from the included studies, documents and indicators; interpreted the results; and edited the manuscript. I.N.M. developed the study protocol; developed the search strategy; conducted the literature search; screened articles, documents and indicators; extracted data from the included studies, documents and indicators; interpreted the results; and drafted the manuscript. E.M.Z. developed the search strategy; conducted the literature search; screened articles, documents and indicators; extracted data from the included studies, documents and indicators; interpreted the results; and edited the manuscript. M.F.E. drafted the manuscript. O.P.V. interpreted the results; and drafted the manuscript. All authors reviewed the article for intellectual content and approved the final version.

Corresponding author

Correspondence to Estephania Moreno-Zegbe.

Ethics declarations

Ethics approval and consent to participate

The research protocol Mejora de la calidad en la atención al parto institucional a través de la “lista de verificación de parto seguro” (#1306), in which the review is nested was approved by the Research and Ethics Committees of the National Institute of Public Health.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

PubMed search strategy. Search strategy in PubMed. (DOCX 14 kb)

Additional file 2:

References all sources. Complete list of references used in the research by source: repositories, compendiums, scientific articles (abstract discarded, full text articles excluded, and full text articles included) and grey literature. (DOCX 82 kb)

Additional file 3:

Description 87 indicators. 87 indicators selected in English and Spanish ordered by each phase of the continuum with reference. (DOCX 112 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saturno-Hernández, P.J., Martínez-Nicolás, I., Moreno-Zegbe, E. et al. Indicators for monitoring maternal and neonatal quality care: a systematic review. BMC Pregnancy Childbirth 19, 25 (2019). https://doi.org/10.1186/s12884-019-2173-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12884-019-2173-2

Keywords

BMC Pregnancy and Childbirth

ISSN: 1471-2393

Contact us