Developing quality indicators for assessing quality of birth centre care: a mixed- methods study
BMC Pregnancy and Childbirth volume 17, Article number: 259 (2017)
Birth centres are described as settings where women with uncomplicated pregnancies can give birth in a home-like environment assisted by midwives and maternity care assistants. If complications arise or threaten, the woman is referred to a maternity unit of a hospital where an obstetrician will take over responsibility. In the last decade, a number of new birth centres have been established in the Netherlands, based on the assumption that birth centres provide better quality of care since they offer a better opportunity for more integrated care than the existing system with independent primary and secondary care providers. At present, there is no evidence for this assumption. The Dutch Birth Centre Study is designed to present evidence-based recommendations for organization and functioning of future birth centres in the Netherlands. A necessary first step in this evaluation is the development of indicators for measuring the quality of the care delivered in birth centres in the Netherlands. The aim of this study is to identify a comprehensive set of structure and process indicators to assess quality of birth centre care.
We used mixed methods to develop a set of structure and process quality indicators for evaluating birth centre care. Beginning with a literature review, we developed an exhaustive list of determinants. We then used a Delphi study to narrow this list, calling on experts to rate the determinants for relevance and feasibility. A multidisciplinary expert panel of 63 experts, directly or indirectly involved with birth centre care, was invited to participate.
A panel of 42 experts completed two Delphi rounds rating determinants of the quality of birth centre care based on their relevance (to the setting) and feasibility (of use). A set of 30 determinants for structure and process quality indicators was identified to assess the quality of birth centre care in the Netherlands.
We identified 30 determinants for structure and process quality indicators concerning birth centre care. This set will be validated during the evaluation of birth centres in the Dutch Birth Centre Study.
Internationally, birth centres are described as settings where women with uncomplicated pregnancies can give birth in a home-like environment. In the Netherlands, women with uncomplicated pregnancies can choose where they want to give birth: at home, in a birth centre or in a hospital . At any location, community midwives are responsible for care during labour and birth as long as it stays uncomplicated. When additional medical assistance is required, the women will receive specialist care under responsibility of an obstetrician at an obstetric unit. Birth care in a birth centre is provided by community midwives, assisted by maternity care assistants. The community midwife accompanies the woman to the centre when labour has started. A maternity care assistant assists the midwife during labour and birth and provides postnatal care to the woman and new-born. Most birth centres do not have a permanent staff of midwives and maternity care assistants. They are only present at the centre when accompanying a woman in labour.
Birth centres have been present in the Netherlands since the nineteenth century , but not until the year 2000 did the number of these centres begin to grow considerably. This appeared to be a reaction to a severe shortage of maternity care providers, especially primary care midwives and maternity care assistants but also obstetric nurses in hospitals. Birth centres were seen as a solution, because they reduce the pressure on hospital maternity wards by providing women who do not want to give birth at home with a safe and home-like alternative. And because birth centres allow midwives to supervise multiple births simultaneously, they also reduce the pressure on community midwives. These birth centres were typically built right next to, or within, the walls of a hospital. However, most of them disappeared again when the problem of the shortage of maternity care providers was alleviated by a dropping birth rate following the millennium baby boom.
In recent years perceptions about the safety of the maternity care system in the Netherlands began to change. An important cause for this was the publication of the Euro-Peristat data, alarming the Netherlands because of its relatively high perinatal mortality compared to other European countries . It was suggested that this might be related to the strict division between primary and secondary care in the Dutch maternity care system [4,5,6,7]. The basic feature of this system is that for healthy women community midwives or general practioners are the responsible care providers (primary care), and for women with pre-existing and emerging pathology obstetricians are the responsible care providers (secondary care) . Media attention given to the Euro-Peristat data and the report from a special committee set up by the Minister of Health (Steering Group “Pregnancy and Childbirth”)  may have attributed to a change in the attitudes and behaviour of Dutch women and their care providers with an increasing number of women choosing, or being referred to, a hospital to give birth : in 2000 30.3% of all births took place at home but this fell to 13.1% in 2015 . More and more healthy women are opting for a hospital birth because they do not feel safe at home, or are asking for referral to receive treatment (i.e. pain medication) that cannot be provided in primary care . Birth centres can be seen as an opportunity to keep these healthy women away from the clinical setting, to provide a safe and home-like alternative, but to be close enough to a hospital to be able to take them in quickly when referral is warranted. In their report, the Steering Group recommended more integration in maternity care, by improved cooperation between primary and secondary care and the introduction of birth centres with close links to hospitals. They also recommended further research on the added value of birth centres . In recent years, following these recommendations, a number of new birth centres have been established in the Netherlands, based on the assumption that birth centres provide better quality of care – as measured by perinatal and maternal outcomes – since they offer a better opportunity for more integrated care than the existing system with independent primary and secondary care providers . At present, there is no evidence for this assumption because there is no reliable way to measure degree and quality of integration in care provision. The Dutch Birth Centre Study is designed to present evidence-based recommendations for organization and functioning of future birth centres in the Netherlands, based on careful assessment of existing birth centres . A necessary first step in this process is development of indicators for measuring the quality of the care delivered in birth centres in the Netherlands.
Although formulated in 1990, the definition of quality of care provided by the Institute of Medicine (IOM) is still widely accepted: “quality of care is the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge” . Usually three dimensions of quality of care are distinguished: structure (the capacity to provide high quality care), process, and outcome . Measures of these three dimensions are called indicators. To assess quality of care, indicators should be developed for the seven domains of quality identified by the IOM: effectiveness, safety, timeliness, efficiency, equity, accessibility and patient-centeredness . Internationally, standards for birth centres are available and can provide a tool for measuring the quality of service provided to childbearing families in birth centres [18, 19], but these standards must be adjusted for specific settings of these centres, in our case, the unique maternity care system in the Netherlands. A number of outcome quality indicators are available to assess birth centre care (i.e. perinatal and maternal mortality and morbidity) [20,21,22,23,24,25,26,27], but structure and process indicators, specifically developed for birth centre care, are scarce.
In this article we describe a set of determinants for structure and process indicators for assessing the quality of birth centre care and we explain the approach we used to develop this set. We only describe the development of determinants for structure and process indicators, because a newly validated Optimality Index (OI-NL2015) and a Composite Adverse Outcome Score (CAOS) were used to evaluate outcomes of birth centre care .
In order to develop a comprehensive set of structure and process quality indicators to evaluate birth centre care, we used mixed methods. Three phases were followed in the development process: 1) identification of existing quality indicators in birth care, 2) translating these structure and process indicators into determinants, 3) determinant selection by Delphi consultation. The study was conducted in the first half of 2013 as part of the Dutch Birth Centre Study .
Identification of existing quality indicators in birth care
In the first phase of the study, we used various sources to find existing quality indicators in birth care. We began with an Internet search for documents from Dutch Institutes that had developed quality indicators for maternity care. Documents that described the (development of) quality indicators by midwives, obstetricians and maternity care assistants were obtained. Next, we reviewed international scientific literature about birth centres in order to identify existing quality indicators. We searched PubMed and the Cochrane Library using the Mesh terms: “birthing centres”, “quality indicator”, “health care” and search terms “quality” and “birth centre”. We used references from these articles to find other relevant articles and documents related to quality indicators in maternity care.
Translating indicators into determinants
In the second phase we translated the structure and process indicators that we had identified into determinants (or topics): elements that identify the nature of the indicator. We used a framework based on the seven domains of quality according to the IOM (effectiveness, safety, timeliness, efficiency, equity, accessibility and patient-centeredness.) We added an eighth domain, “Law on the Accessibility of Healthcare Facilities”, because of obligations placed on healthcare facilities by this law in the Netherlands. The research group used their experience to add topics that were missing in the resulting list. No outcome indicators were included. We then created a questionnaire that members of an expert panel could complete in a minimum amount of time in order to maximize our response rate.
Determinant selection by Delphi consultation
We initiated an online Delphi study with the goal of obtaining consensus among a group of experts. The online Delphi technique is an anonymously structured approach, in which information is gathered from a group of participants through a number of Delphi rounds. The web-based anonymous nature of the Delphi technique ensures that a single individual cannot dominate the consensus formation. Moreover all participants are equally able to change their opinion in the course of the process [28, 29]. Our Delphi study consisted of two online questionnaires.
We selected participants for the expert panel from the Research Advisory Group of the Dutch Birth Centre Study , participants of former panels of developing indicators for maternity care in the Netherlands, professionals from different disciplines who are working with or in a birth centre with several years of experience, representatives of health insurance companies, policymakers, clients and advisors in birth care. Of the care providers, only experts who are actually involved in birth (centre) care were invited and all health care disciplines related to birth (centre) care were represented. We included professionals in our heterogeneous expert panel: (11 (community and clinical) midwives, 2 general practitioners, 5 maternity care assistants, 6 obstetricians, 4 paediatricians, 5 obstetrics and gynaecology nurse specialists, 7 managers from birth centres, 5 representatives from health insurance companies, 3 representatives from clients and 15 other experts (i.e. policymakers, advisors and research experts). We limited the number of participating clients, because their view on quality of birth centre care is examined in another part of the study .
Rating determinants by experts: First Delphi round
In May 2013, we sent a link to an online questionnaire by e-mail to the expert panel. The experts were instructed to rate the determinants on relevance (to the setting) and feasibility (of use) and, if necessary, to comment on them. Each determinant was rated on a seven-point Likert scale (1 = not at all relevant/feasible; 4 = neutral; 7 = very much relevant/feasible). Finally, experts were encouraged to suggest additional relevant subjects that should be taken into consideration in the assessment of the quality of birth centre care. All ratings from the first Delphi round were analysed in Excel and distributions of scores were presented in median scores for each determinant. We considered determinants with a median score of ≥6 with agreement to be relevant and feasible to collect and accepted these immediately. Agreement was defined when 80% or more of the ratings were within a range of three (i.e. 5–6-7 of 4–5-6). Determinants that scored with a median score of ≤2 were rejected. Median scores of >3 and <6 with agreement or ≥6 without agreement were discussed again in the second Delphi round. Furthermore, all the comments on determinants from the first round were analysed and the descriptions of determinants were re-phrased in cases of ambiguity. All proposed new determinants from the first round were categorized in domains. New determinants were coded and two researchers of our research group decided, using a consensus method, which determinants should be submitted in the second round. Items the research group already had decided to include in the overall study (i.e. professional experiences and topics related to integration) were not included in the second round.
Rating determinants by experts: Second Delphi round
In the second Delphi round, the experts were informed about the median scores on relevance and feasibility of the total expert group, their own scores and the comments of the respondents regarding determinants for which no consensus was reached in the first round. They were instructed to re-consider their rating of the determinants presented in the first round as well as to rate and comment on the new elements the same way as in the first round. This was done to allow experts to revise their opinion of the first round while considering the ratings and comments provided by the other members of the expert panel. The link to the personalized online questionnaire was sent by email 10 days after the first round. Again, the median scores and the degree of agreement were calculated. Only scores ≥6 with agreement were adopted into the list. Determinants with scores for relevance ≥6 with agreement, but feasibility between 3 and 6 were presented to the research group for a final decision.
Figure 1 shows the total process that led to the selection of structure and process quality indicators of birth centres, and the number of determinants (topics) at each step.
Identification of existing quality indicators in birth care
Two hundred fifteen indicators were derived from Dutch sources, 145 from international literature. We eliminated duplication and excluded all outcome indicators. Indicators that clearly do not determine quality of care in a birth centre (because it is clear that this kind of care does not occur in birth centres, i.e. caesarean section) were also excluded from this list. Finally 66 structure and process indicators were identified.
Translating indicators into determinants
These 66 literature-based indicators were divided into seven themes matching the seven domains of quality according to the United States Institute of Medicine (IOM). The research group added a domain “Law Accessibility of Healthcare Facilities”. In these eight domains, 22 topics were identified, because several indicators appeared to relate to the same topic, albeit with different wordings. The research group added another 13 topics that they missed, based on their experience. After this process, the topics were formulated as 35 determinants to be included in the first questionnaire for the online Delphi panel.
Determinant selection by Delphi consultation
The questionnaire in the first Delphi round was completed by 48 experts (response rate of 76%). 42 of them also completed the questionnaire in the second round (response rate of 88%). During the first round, 24 of the 35 determinants were accepted for inclusion, none were rejected right away, leaving 11 topics without consensus. 22 experts mentioned 52 new topics they missed in the questionnaire. These topics were labelled and categorized, after which two researchers of our research group decided, based on consensus, that 8 of them would be included in the second Delphi round. In the second round, the 11 topics from the first round on which no consensus was reached and the 8 new topics were presented to the expert panel. This resulted in the acceptance of another five determinants and the rejection of 13 determinants. One determinant was presented to the research group because of low feasibility according to the experts. The research group accepted this determinant, so finally 30 determinants resulted from the Delphi consultation. Table 1 shows the selected determinants per IOM quality domain. Table 2 shows all determinants included in the Delphi procedure with the number or rated scores on the Likert Scale.
In this study, part of the Dutch Birth Centre Study, we identified a set of 30 determinants, to be translated into 30 structure and process quality indicators that can be used to assess the quality of birth centre care in the Netherlands. The new developed determinants are derived from existing quality indicators in maternity care in the Netherlands (used to measure quality of care by midwives, obstetricians and maternity care assistants) and indicators derived from international documents concerning birth centre care. The experts selected 5 determinants that are used by Laws in the research on characteristics and practices of birth centres in Australia  and 4 determinants derived from Dutch existing quality indicators. They also selected 3 determinants which were formulated in a quality framework of birth centre care, proposed by the Royal Dutch Organization of Midwives (KNOV). Ten selected determinants are used by different organizations to assess quality of care (e.g. maternity care assistance, emergency care) [30,31,32,33,34]. Finally, 7 new determinants were selected by the experts. The final set of indicators will be included in the on-going study to evaluate birth centre care in the Netherlands.
A strength of the development of this set of determinants for indicators is that it is developed in collaboration with all parties involved in birth centre care, and is based on consensus. Therefore it can be expected that all professionals in the field will accept assessing the quality of birth centre care using this set of indicators.
We are aware that the set of determinants for indicators we developed has its limitations. Firstly, to assess care in general, structure, process and outcome indicators should be used. However, because there are already a large number of quality indicators to assess outcomes of birth centre care, this set contains only structure and process indicators . The expert panel chose 19 structure and 11 process indicators to asses birth centre care. For the same reason the set we developed does not include indicators of women’s experiences of care, because they can be regarded as outcome indicators . Thirdly, this set only consists of determinants for indicators. The process for developing structure and process quality indicators for birth centres still needs to be described. Also, we do not yet know whether this set of determinants for indicators will be able to differentiate between birth centres or not. It has yet to prove itself in practice: the Dutch National Birth Centre Study will be the first to use these indicators to assess the quality of care.
Finally, although our study was focused on Dutch birth centres, we expect that this set of determinants for indicators will be applicable in other settings where birth centres are used.
We used an online Delphi-method to develop a list of thirty determinants for structure and process indicators to measure quality of birth centre care. We will describe the process for developing quality indicators from these determinants and evaluation of the validity and reliability of these indicators as part of the Dutch Birth Centre Study in a later paper. It is important to underscore that indicators are part of an on-going cycle of quality improvement. Indicators should never be static. Changes in evidence or clinical relevance, a consistently high performance or a low variation in achievement, new developments and demographic changes in the population of childbearing women, all may be criteria for removing an indicator or adding a new one in a future list of determinants for quality indicators for birth centre care.
Institute of Medicine
Royal Dutch Organization of Midwives
the Netherlands Organisation for Health Research and Development ZonMw
Wiegers T, de Graaf H, van der Pal K. De opkomst van geboortecentra en hun rol in de zorg. Tijdschrift voor gezondheidswetenschappen 2012. 2012;90(8):475–8.
de Graaf JP, Merkus H, Ravelli A, Bonsel G, Steegers E. Short history of Dutch obstetric care and birth centres in particular. In: Perinatal and maternal health inequalities: effects of places of residence and delivery. 2013;14
Zeitlin J, Mohangoo A, Cuttini M. The European perinatal health report: comparing the health and care of pregnant women and newborn babies in Europe. J Epidemiol Community Health. 2009;63(9):681–2.
Evers AC, Brouwers HA, Hukkelhoven CW, Nikkels PG, Boon J, van Egmond-Linden A, et al. Perinatal mortality and severe morbidity in low and high risk term pregnancies in the Netherlands: prospective cohort study. BMJ. 2010;341
Tuffnell D. Place of delivery and adverse outcomes. BMJ. 2010;341
van der Kooy J, Poeran J, de Graaf JP, Birnie E, Denktas S, Steegers EA, et al. Planned home compared with planned hospital births in the Netherlands: intrapartum and early neonatal death in low-risk pregnancies. Obstet Gynecol. 2011;118(5):1037–46.
de Jonge A, Baron R, Westerneng M, Twisk J, Hutton EK. Perinatal mortality rate in the Netherlands compared to other European countries: a secondary analysis of euro-PERISTAT data. Midwifery. 2013;29:1011–8.
Verloskunde C. Verloskundig Vademecum 2003. College voor Zorgverzekeringen: Diemen; 2003.
Stuurgroep Zwangerschap en Geboorte. Een goed begin - Advies Stuurgroep zwangerschap en geboorte. Den Haag: 2009.
de Vries RG, Paruchuri Y, Lorenz K, Vedam S. Moral science: ethical argument and the production of knowledge about place of birth. J Clin Ethics. 2013 Fall;24(3):225–38.
Nederland SPR. Perinatale Zorg in Nederland 2015. Stichting Perinatale Registratie Nederland: Utrecht; 2016.
Anthony S, Amelink-Verburg M, Jacobusse G, van der Pal-de Bruin, KM. De thuisbevalling in Nederland 1995-2002. Rapportage over de jaren 2001–2002 (Home births in the Netherlands 1995–2002. Report for the years 2001-2002). PRN/TNO rapport KvL/JPB 2005;2005.
Bonsel GJ, Birnie E, Denktas S, Poeran J, Steegers EAP. Lijnen in de Perinatale Sterfte, Signalementstudie Zwangerschap en Geboorte 2010. Rotterdam:Erasmus MC,2010.
Hermus MA, Wiegers TA, Hitzert MF, Boesveld IC, van den Akker-van Marle ME, et al. The Dutch birth Centre study: study design of a programmatic evaluation of the effect of birth centre care in the Netherlands. BMC Pregnancy and Childbirth. 2015;15(1):148.
Lohr KN, Schroeder SA. A strategy for quality assurance in Medicare. N Engl J Med. 1990;322(10):707–12.
Donabedian A. Explorations in Quality Assessment and Monitoring: Ill.: Health Administration Press; 1980.
Institute of Medicine (US). Committee on Quality of Health Care in America. Crossing the quality chasm: A new health system for the 21st century. National Academies Press; 2001.
American Association of Birth Centers. Standards for birth centers. Perkiomenville: AABC; 2008.
The Royal College of Midwives. Standards for birth Centres in England: a standards document. London: RCM; 2009.
Laws PJ, Lim C, Tracy S, Sullivan EA. Characteristics and practices of birth centres in Australia. Aust N Z J Obstet Gynaecol. 2009;49(3):290–5.
Laws PJ, Tracy SK, Sullivan EA. Perinatal outcomes of women intending to give birth in birth centers in Australia. Birth. 2010;37(1):28–36.
Stapleton SR, Osborne C, Illuzzi J. Outcomes of care in birth centers: demonstration of a durable model. Journal of Midwifery & Women’s Health. 2013;
Rooks JP, Weatherby NL, Ernst EK, Stapleton S, Rosen D, Rosenfield A. Outcomes of care in birth centers. The National Birth Center Study. N Engl J Med. 1989;321(26):1804–11.
Stewart M, McCandlish R, Henderson J, Brocklehurst P. Review of evidence about clinical, psychosocial and economic outcomes for women with straightforward pregnancies who plan to give birth in a midwife-led birth centre, and outcomes for their babies. Updated July: Report of a structured review of birth centre outcomes; 2005.
Waldenstrom U, Nilsson C, Winbladh B. The Stockholm birth centre trial: maternal and infant outcome. BJOG. 1997;104(4):410–8.
Byrne JP, Crowther CA, Moss JR. A randomised controlled trial comparing birthing centre care with delivery suite care in Adelaide, Australia. Aust N Z J Obstet Gynaecol. 2000;40(3):268–74.
Hollowell J, Puddicombe D, Rowe R, Linsell L, Hardy P, Stewart M. The Birthplace national prospective cohort study: perinatal and maternal outcomes by planned place of birth. Birthplace in England research programme. Final report part 4: NIHR service delivery and organisation programme, 2011 .
Keeney S, McKenna H. Hasson F. The Delphi technique in nursing and health research: John Wiley & Sons; 2010.
Skulmoski GJ, Hartman FT, Krahn J. The Delphi method for graduate research. J Inf Technol Educ. 2007;6:1.
Kooistra M, Waelput A, Offerhaus P, et al. Kwaliteitsindicatoren voor de eerstelijnsverloskunde. RIVM rapport 260101005. 2009.
Zichtbare Zorg Eerstelijnsverloskunde. Zorginhoudelijke indicatoren zichtbare zorg eerstelijnsverloskunde. 2012.
Zorginstituut Nederland (2012). Kwaliteitsindicatoren kraamzorg.
Zorgverzekeraars Nederland (2013). Rapport Kwaliteitsvisie spoedeisende zorg,
KNOV. Kwaliteitskader geboortecentra (2010) 2010. Accessed 20 Februar 2013. http://www.knov.nl/werk-en-organisatie/tekstpagina/268/geboortecentra/
Wiegers TA, Keirse MJNC, Berghs GAH, van der Zee J. An approach to measuring quality of midwifery care. J Clin Epidemiol. 1996;49(3):319–25.
van der Kooy J, Valentine NB, Birnie E, Vujkovic M, de Graaf JP, Denktaş, et al. Validity of a questionnaire measuring the world health organization concept of health system responsiveness with respect to perinatal services in the Dutch obstetric care system. BMC Health Serv Res. 2014;14(1):1.
The authors gratefully acknowledge the other members of the Project Group ‘Birth Centre Study’ for their input in the project. The authors would like to thank the members of the Expert panel for their time and energy spent in the panel. We are also grateful for the helpful comments of Theo Haitjema and Ömer Aydogan for their useful comments on the draft version of this article.
This study was funded by ZonMw (the Netherlands Organisation for Health Research and Development (ZonMw) in the context of the research program Pregnancy and Childbirth, [grant no. 50–50,200–98-102].
The authors declare that the funding body had no role in the design of the study, collection, analysis and interpretation of data and writing the manuscript.
Availability of data and materials
The dataset supporting the conclusions of this article is included within the article (and its additional file).
Ethics approval and consent to participate
Design and planning of the study were presented to the Medical Ethics Committee of the UMCU (University Medical Centre Utrecht) who confirmed that an official ethical approval of this study is not required.
Consent for publication
This study was funded by ZonMw (the Netherlands Organisation for Health Research and Development (ZonMw) in the context of the research program Pregnancy and Childbirth, [grant no. 50–50,200–98-102]. The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Boesveld, I.C., Hermus, M.A.A., de Graaf, H.J. et al. Developing quality indicators for assessing quality of birth centre care: a mixed- methods study. BMC Pregnancy Childbirth 17, 259 (2017). https://doi.org/10.1186/s12884-017-1439-9