An evaluation of the quality, suitability and impact on equity of clinical practice guidelines relevant to preterm birth for use in Aotearoa New Zealand

Background Preterm birth is a leading cause of perinatal morbidity and mortality and a defining event for pregnant people, infants, and whānau (extended families). Recommendations have been made for a national preterm birth prevention initiative focusing on equity in Aotearoa New Zealand, including the development of a national best practice guide. An understanding of the number and quality of guidelines, and consideration of their suitability and impact on equity is required. Methods Guidelines were identified through a systematic literature search, search of professional bodies websites, and invitation to regional health services in Aotearoa New Zealand. Obstetric and midwifery clinical directors were invited to report on guideline use. Identified guidelines were appraised by a 23-member trans-disciplinary Review Panel; quantitatively using the AGREE-II instrument and qualitatively using modified ADAPTE questions. The quality of guidelines available but not in use was compared against those in current use, and by health services by level of maternity and neonatal care. Major themes affecting implementation and impact on equity were identified using Braun and Clarke methodology. Results A total of 235 guidelines were included for appraisal. Guidelines available but not in use by regional health services scored higher in quality than guidelines in current use (median domain score Rigour and Development 47.5 versus 18.8, p < 0.001, median domain score Overall Assessment 62.5 versus 44.4, p < 0.001). Guidelines in use by regional health services with tertiary maternity and neonatal services had higher median AGREE II scores in several domains, than those with secondary level services (median domain score Overall Assessment 50.0 versus 37.5, p < 0.001). Groups identified by the Review Panel as experiencing the greatest constraints and limitations to guideline implementation were rural, provincial, low socioeconomic, Māori, and Pacific populations. Identified themes to improve equity included a targeted approach to groups experiencing the least advantage; a culturally considered approach; nationally consistent guidance; and improved funding to support implementation of guideline recommendations. Conclusions We have systematically identified and assessed guidelines on preterm birth. High-quality guidelines will inform a national best practice guide for use in Taonga Tuku Iho, a knowledge translation project for equity in preterm birth care and outcomes in Aotearoa. Supplementary Information The online version contains supplementary material available at 10.1186/s12884-024-06415-0.

Methods Guidelines were identified through a systematic literature search, search of professional bodies websites, and invitation to regional health services in Aotearoa New Zealand.Obstetric and midwifery clinical directors were invited to report on guideline use.Identified guidelines were appraised by a 23-member trans-disciplinary Review Panel; quantitatively using the AGREE-II instrument and qualitatively using modified ADAPTE questions.The quality of guidelines available but not in use was compared against those in current use, and by health services by level of maternity and neonatal care.Major themes affecting implementation and impact on equity were identified using Braun and Clarke methodology.
Results A total of 235 guidelines were included for appraisal.Guidelines available but not in use by regional health services scored higher in quality than guidelines in current use (median domain score Rigour and Development 47.5 versus 18.8, p < 0.001, median domain score Overall Assessment 62.5 versus 44.4,p < 0.001).Guidelines in use by regional health services with tertiary maternity and neonatal services had higher median AGREE II scores in several domains, than those with secondary level services (median domain score Overall Assessment 50.0 versus 37.5, p < 0.001).Groups identified by the Review Panel as experiencing the greatest constraints and limitations to guideline implementation were rural, provincial, low socioeconomic, Māori, and Pacific populations.Identified themes to improve equity included a targeted approach to groups experiencing the least advantage; a culturally An evaluation of the quality, suitability and impact on equity of clinical practice guidelines relevant to preterm birth for use in Aotearoa New Zealand Briar Hunter 1* , Lisa Dawes 1,2,3 , Makayla Wadsworth 1 , Lynn Sadler 2,3,4 , Liza Edmonds 2,5,6 , Judith McAra-Couper 2,7 , Tina Allen-Mokaraka 2 and Katie M. Groom 1,2,3

Background
Preterm birth is a leading cause of perinatal morbidity and mortality and may lead to lifelong disability and poor health [1].The risk of preterm birth varies both between and within populations, often due to differences in individuals' risk status, but also by factors such as ethnic and cultural background, place of residence, and socioeconomic status [2][3][4][5][6].This is clearly seen in Aotearoa New Zealand with a lower rate of preterm birth, as well as higher rates of survival following extreme preterm birth, for European and Asian babies than for their Māori (Aotearoa New Zealand Indigenous/First Nations), Pacific and Indian counterparts [3,7].Improving quality and consistency of care with equity as a focus should result in less variation in practice and lead to improved outcomes, especially for those currently experiencing the least advantage.
Clinical practice guidelines have the potential to improve practice and outcomes by summarising and synthesising complexities of evidence, thereby increasing speed and efficiency of clinical decision-making [8].Furthermore, when applied consistently in practice, clinical practice guidelines should support improved equity in care and outcomes.In order to achieve this, they should contain and/or consider a systematic review of all relevant evidence and be developed in partnership with key stakeholders including the populations most affected by the clinical issue [9,10].If equity is to be effectively addressed it must be considered from the outset of a clinical practice guideline's development, and factored into every step including through implementation and evaluation.Without this consideration, there is a risk that there will be no reduction in differences seen by equity factors, or even worsening of equity issues by only improving care and outcomes for those already experiencing system and other advantage [11].
Improving care and outcomes for preterm birth poses a challenge due to the diverse number and type of conditions contributing to both spontaneous and providerinitiated preterm birth.Clinical practice guidance must consider this broad range of conditions and include prediction and prevention of preterm birth, as well as, preparation and management when preterm birth is inevitable or expected.
In Aotearoa New Zealand, preterm birth and equity of access to care have been identified as priority areas for maternity care practice improvement [3,[12][13][14][15].A stakeholder-led initiative, the Carosika Collaborative, has been funded through the New Zealand Health Research Council to undertake a knowledge translation project for equity in preterm birth care and outcomes in Aotearoa.This project, named Taonga Tuku Iho (Te Reo Māori referring to the legacy of preterm babies) includes the development of a national best practice guide to support consistent and equitable practice.This best practice guide will be developed using the ADAPTE framework; adopting, contextualising and adapting recommendations across all aspects of preterm birth care, using existing high-quality clinical practice guidelines relevant to Aotearoa New Zealand, where available, and formulating new recommendations where no high quality guidelines exist, to provide a single point of reference for national use.This requires an understanding of the quality, suitability and impact on equity of the guidelines that are currently in use or are appropriate for use in Aotearoa New Zealand.

Methods
This study aims to systematically appraise the quality of clinical practice guidelines relating to preterm birth and to assess their perceived impact on equity and suitability for use in the Aotearoa New Zealand context.

Guideline selection
Clinical practice guidelines were identified and considered for inclusion from the published literature, relevant professional bodies, and the previously named Aotearoa New Zealand District Health Boards.District health boards were the name given to the regional health services in Aotearoa New Zealand up until July 2022.Before they were disestablished, there were 20 district health boards across the country.

Systematic literature search
A search strategy was developed and refined (Appendix A).Searches were undertaken of EMBASE, MED-LINE and CINAHL databases (Appendix B).Titles and abstracts were screened by two independent reviewers using the Covidence systematic review reference management tool [16], then full-text articles of references thought to be potentially relevant were retrieved and reviewed.A third reviewer was available if consensus was not able to be reached after initial discussion of any discrepancies.

Professional bodies search
Relevant professional bodies, colleges and societies were identified by the investigator team alongside an internet search.Professional bodies were included if they were involved with any aspect of preterm birth care or the management of its major risk factors.Included professional bodies were Aotearoa New Zealand specific, Australasian or international with relevance to countries with similar resources to Aotearoa New Zealand.A manual search of the websites of each professional body was undertaken by a single investigator to retrieve potentially eligible guidelines.Two reviewers then independently assessed the full text guidelines for inclusion, with a third reviewer available if consensus was not reached after initial discussion of any discrepancies.

District health board search
Invitations were sent to the obstetric and midwifery clinical directors at each of Aotearoa New Zealand's 20 district health boards (assigned regional health services until 2022).Clinical directors were contacted by email and invited to complete a questionnaire about guidelines in use at their institution and requested to contribute these guidelines to the study.If a local guideline was explicitly reported as being based on a national or international guideline then the national/international guideline was also considered to be in use.Written informed consent was obtained from all participants.Non-responders were contacted via email two weeks later and any ongoing non-responders were contacted once by telephone a further two weeks later.Ethical approval was obtained.

Inclusion criteria
Guidelines could be regional, national, bi-national (Australasian), or international (defined as those developed for use in more than one country and with similar resources and health context to Aotearoa New Zealand).Guidelines were included if they contained recommendations on prediction or prevention of preterm birth, preparation for preterm birth, or the acute management of preterm labour.Guidelines on both spontaneous and provider-initiated preterm birth were included, as well as the conditions and major risk factors that contribute to preterm birth.Guidelines were in English and must have been published, reviewed or updated within the last ten years.If multiple versions of a guideline existed, only the latest version was included.Where further guideline documents were referenced, these were retrieved and assessed alongside their relevant guideline.

Appraisal
Descriptive data were collected for each guideline by a single investigator.Guideline appraisals were undertaken by 23 members of the transdisciplinary Taonga Tuku Iho Review Panel including midwives, obstetricians, neonatologists, radiologists, consumers, Māori, Pacific and Indian representatives, and individuals with experience in clinical practice guideline development.Review Panel members were invited as representatives of a professional college, society or group including consumer groups and national guideline groups or due to expertise in specific areas such as preterm birth and Māori health.All members provided a disclosure of interest declaration with information on their professional affiliations and previous involvement in guideline development.An on-line meeting provided background information and a briefing on the appraisal process, including provision of the Review Panel Terms of Reference, ADAPTE Toolkit [17], AGREE Trust training exercises [18], AGREE II Instrument User's Manual [19] and an AGREE II tutorial.
Each guideline was allocated to four reviewers representing a variety of stakeholder groups, with two reviewers (BH and MW) undertaking reviews on every guideline.Guidelines were not allocated to reviewers employed or associated with the organisation responsible for the development of a guideline or involved in its development.
Assessment of quality was made using the AGREE II tool [19] via the online My AGREE Plus appraisal platform [20].This tool was chosen as it is suggested to be the most commonly applied and comprehensively validated guideline appraisal tool worldwide, and the accepted standard for evaluating clinical practice guidelines [21][22][23][24].The AGREE II checklist includes 23 key items organised into six domains.Each guideline was scored by domain, with an overall assessment score and recommendation for use in clinical practice.
Assessment of applicability and/or acceptability for use in Aotearoa New Zealand was undertaken using a modification of Tool 15 from the ADAPTE Resource Toolkit [17] answered on a secure google form.This tool was chosen as it is specifically designed for evaluating guidelines, as well as being in keeping with use of the ADAPTE framework for the overall development of the national best practice guide.It was modified to allow a broader exploration of perceived impact on equity.

Outcomes
The primary outcome was the overall number and quality of guidelines relevant to preterm birth in Aotearoa New Zealand; firstly those in current use and and secondly those available but not in use.

Secondary outcomes
• The proportion of guidelines in use in Aotearoa New Zealand, that are considered to be high-quality.• The proportion of guidelines that are available, but not currently in use in Aotearoa New Zealand, that are considered to be high quality.
Of the 235 identified guidelines, 151 were reported to be currently in use by district health boards.Another 84 were available but not currently in use in Aotearoa New Zealand.The majority of these were national or international guidelines not identified as being utilised by district health board clinical directors.
The AGREE II quality scores are reported in Table 2.A greater proportion of guidelines that were reported to be available but not in use met the pre-defined criteria for high-quality across every category, than those currently in use (Table 2).
Guidelines reported to be in use by district health boards with tertiary maternity and neonatal services, had higher median scores in the domains Rigour of Development, Clarity of Presentation and Applicability, as well as in Overall Assessment Score, compared with guidelines reported to be in use by district health boards with secondary services (Table 3).There was a greater proportion of guidelines in use by district health boards providing tertiary services that met the high-quality criteria for Overall Assessment Score and recommendation for use in clinical practice or use with modifications compared with district health boards providing secondary services (Table 3).Consistency in AGREE II scores between appraisers was moderate or good for (153/235) 65.1% of guidelines and poor for the remainder.
Responses to questions regarding equity and implementation constraints are shown in Table 4. Identified constraints and limitation themes included; cultural considerations in care; service capacity, access to tests, interventions and specialist services; funding to support implementation of recommendations, and staff shortages across the maternity sector (Table 5).Groups identified as experiencing the greatest constraints and limitations to guideline implementation were rural, provincial, low socioeconomic, Māori, and Pacific populations.Major themes of factors that had negative or positive impact on equity were; nationally consistent guidance; culturally considered approach and feasibility of guideline implementation (Table 5).Identified themes to improve equity included a targeted approach to groups experiencing the least advantage; a culturally considered approach; nationally consistent guidance; and improved funding to support implementation of guideline recommendations (Table 5).

Discussion of important findings
Guidelines that were available but not currently in use by district health boards scored higher in quality than guidelines reported to be in current use.This is a potential major concern but may be justified in some instances as guidelines that were available but not in use did not always cover the same topics as the guidelines in use by district health boards.Furthermore, some guidelines with higher quality scores were from international organisations, such as the World Health Organization, and may have been judged by clinicians as less applicable to the local Aotearoa New Zealand clinical environment.However, similar results were found in an appraisal of clinical practice guidelines on reduced fetal movements in the United Kingdom; with consistently lower AGREE II scores in local maternity unit guidelines compared with established national guidelines [26].Overall this makes a compelling argument for a high-quality national best practice guide that is prepared for country-specific context allowing easy adoption and implementation by all local health services.
Guidelines reported to be in use by district health boards with tertiary maternity and neonatal services had significantly higher median AGREE II scores in several domains, than those with secondary level services.Again, this is a potential major concern and may contribute to differences seen in care and outcomes by equity factors, in particular by region of residence [27].Similar findings have been reported in an appraisal of Australasian antenatal corticosteroid guidelines, where institutions providing secondary neonatal services had lower scoring guidelines, than institutions providing tertiary services [28].Our Review Panel identified that rural and provincial areas have greater constraints and limitations to implementing guideline recommendations.Again this strongly supports the development of a highquality national best practice guide that is suitable for use for all.
Despite 61.5% of clinical practice guidelines being developed in Aotearoa New Zealand, and 80.0% being developed in either Aotearoa New Zealand or Australasia, Indigenous involvement in guideline development was extremely low (reported in 4.2%).This low rate of inclusion of Māori as tangata whenua (people of the land), combined with a low rate of reporting equity tool use during guideline development (4.2%), is in contrast with the established roles of district health boards and Aotearoa New Zealand professional bodies to uphold the principles of Te Tiriti o Waitangi, a founding document of Aotearoa New Zealand [29].Having a culturally considered approach to guideline development and implementation was identified by our Review Panel as a key factor to support the ability for guideline recommendations to reduce differences by equity factors.This key factor has been noted in previous studies incorporating equity in guideline implementation [30].Inclusion of a Including 25 duplicates identified across > 1 search strategies members of the most impacted populations and use of an equity tool from the outset of guideline development have been identified as essential for guideline success in addressing equity issues [30], and have been studied specifically in the Aotearoa New Zealand context [31].It is possible that broader Māori representation occurred than was reported and that some institutions may detail guideline development processes in supplementary documents that were not provided to our investigator team, however full disclosure of the guideline development processes and stakeholder involvement is a long-established recommendation for clinical practice guideline reporting [9,19].Further themes identified in the analysis of qualitative comments on enablers to improve equity were nationally consistent guidance, and improved funding to support implementation of guideline recommendations.The latter theme has been identified elsewhere as a major equity consideration, likely to be relevant worldwide [32].
Although this study was carried out in the Aotearoa New Zealand context, findings are likely to be relevant and of interest to other countries with similar health  resources.This is particularly important given the apparent exponential growth in clinical guidelines in recent years and increasing awareness of consideration of equity in all areas of clinical care including the development and use of guidelines.

Strengths and limitations
This study included a systematic search covering prediction, prevention, preparation and management of preterm birth.The use of a multi-pronged search strategy allowed for identification of a broad range of guidelines, not limited to those formally published in peer-reviewed journals.Inclusion of regional clinical practice guidelines and a questionnaire on guideline utilisation by maternity care providers, has not been captured in previous appraisals of New Zealand's maternity sector guidelines [33].District health board engagement was good, increasing the generalisability of the findings.The AGREE II instrument was chosen as a systematic, validated, and globally-recognised clinical practice guideline appraisal tool, which is appropriate for use across a wide range of guideline subject matters [21][22][23][24].The use of the online My AGREE Plus platform made for convenient coordination of appraisals and straightforward collaboration of a large panel of reviewers.While the platform and built-in user's manual made the appraisals easy to complete, the platform was limited in its ability to report results; generating individual data files for each guideline.For a project of this size, this resulted in a large amount of manual processing of results, introducing the potential for human error in transcribing data.
A limitation of AGREE II is the lack of established threshold criteria for quality, which is left to individual research groups to determine based on their research question.Domain 3 'Rigour of Development' was chosen as a priority area for this study, as we were interested in identifying guidelines that had methodological strength and a firm evidence basis that could be trusted for use in developing a national best practice guide.'Overall Assessment Score' was chosen as another quality criteria to provide a global assessment score of the guideline.Guideline recommendation for clinical use provided a pragmatic indicator of value in clinical practice, for guidelines that may have practical value that is not necessarily reflected in their AGREE II domain ratings.
A further major limitation of the AGREE II tool is the lack of comment or consideration of equity in guideline implementation, for which it has received criticism previously [10,34].For this reason, we included additional questions modified from the ADAPTE Handbook, specifically to look at guideline applicability, acceptability, and impact on equity [17].This enabled identification of the major themes of constraints and limitations to guideline implementation and identification of guideline factors that impact on equity.Including a qualitative appraisal component allowed the wealth and breadth of experience of our Review Panel to be appreciated and added a depth to the assessment that would not have been possible from quantitative appraisal alone.These insights were invaluable in identifying enablers and barriers for developing a national best practice guide on this topic.
All appraisers underwent training on the AGREE II tool, however despite this, there was significant variation seen between the scores given by different appraisers, with 35% of guidelines having low consistency in the scores given by different appraisers.This is in part a reflection of the subjective nature of AGREE II scoring, with no definitive guidance provided informing the score that should be given when different criteria are met.Variation likely also arose due to the wide range of backgrounds of the Review Panel and the large number of appraisers, although this broad representation is also a strength of the appraisal.

Conclusions
This was a comprehensive review of clinical practice guidelines on preterm birth available for use and in use in the Aotearoa New Zealand public health system.The mixed methods appraisal had strengths in identifying guidelines of high methodological rigour, as well as potential constraints and limitations to guideline implementation.Major factors contributing to impact on equity were identified; providing invaluable insight for developing a national best practice guide on this topic to be used across Aotearoa New Zealand.
Development of a national best practice guide for equity in preterm birth care is now underway, in partnership with the groups most impacted in Aotearoa New Zealand; with a culturally considered approach and "For women in rural areas it may be harder for them to access care when they do start bleeding and therefore do not receive interventions until later "Availability of emergency transfer and retrieval services and neonatal services" "Access to specialist scanning and MFM specialists if required" "BP monitoring more difficult for women living rurally or with transport difficulties.LMCs may have limited ability to visit and perform frequent checks" "Access to ultrasound where demand is exceeding supply and cost [is a] barrier" "Ideally a pre-pregnancy health optimisation consultation would take place for high risk and obese women but the scope and funding for this is certainly not prioritized and more often than not is opportunistic rather than planned.Our highest risk patients are the least likely to present for such consultations secondary to financial social and educational constraints" "Many peripheral centres do not have the neonatal expertise to care for the extreme preterm and women need to be transferred" Funding "LMCs not resourced to deliver [anti-D] in the community which may contribute to women not receiving prophylactic dosing" "Funding of LMCs to perform frequent BP monitoring" Staff shortages "Maternity services all around New Zealand are facing critical shortages and this means that optimal care is not being provided to women.This is more so in the most unwell women" "National midwifery shortage makes it challenging to perform frequent outpatient BP monitoring as well as undertake staff intensive interventions such as MgSO4 or IV antihypertensive therapy" "There is a shortage of specialist Maternal Fetal Medicine doctors and midwives for the population and this may hinder the advice, support and management that is provided to women" Education and care coordination "Consistent availability of treatment recommendations in all main centres" "Staff education around implementation of guideline" "Skill expertise, legislation and policy, and organisational barriers" Factors that had a positive impact on equity Quality of guidelines "Administration of steroids [following binational guideline] to improve outcomes for preterm babies.Universal administration would ideally reduce differences for all" "If recommendations [from binational Magnesium Sulphate guideline] are implemented across all areas, [it] has the potential to reduce differences in PTB outcomes across populations" Nationally consistent guidance "The clear guidance for what to do for women who are less than 24 weeks -all women should get [the] same talk and options for their baby" "Good to have clear indications/parameters about when to consider delivery at different gestations [for women with antepartum haemorrhage]" "Good to have explicit information about lines of communication and responsibility plus clear information about funding and support for women and whanau" "Good to have [a] clear step by step guide of how to arrange transfers and who is responsible at each step to streamline this potentially stressful event" Culturally considered approach "Good that Māori/Pacific Island ethnicity scores a point as an Amber warning score [on the sepsis guideline] to account for women of these ethnicities often presenting later and having worse outcomes" Factors that had a negative impact on equity Quality of guidelines "Very low quality guideline with no referencing" "Extremely brief guideline, no background, no references, no comment about management" Nationally consistent guidance "Recommend more consistency in practice that is likely to vary by clinician and unit" Culturally considered approach "There is a lack of cultural safety in how this guideline would apply to Māori and Pasifika patients.As a result, it may be less successful for Māori and Pasifika patients if there is an absence of a more holistic approach, as well as the absence of upstream approaches" "No culturally diverse information/advice provided" "Unless population specific (culturally appropriate) support available then differences between Māori and non-Māori smoking cessation rates are likely to increase"

Fig. 1
Fig. 1 Identified clinical practice guidelines.CINAHL, Cumulative Index to Nursing and Allied Health Literature; RANZCOG, Royal Australian and New Zealand College of Obstetricians and Gynaecologists; NZCOM, New Zealand College of Midwives; NZMFMN, New Zealand Maternal Fetal Medicine Network; MOH, Ministry of Health; WHO, World Health Organization

Table 1
Description of included guidelines

Table 2
Quality of guidelines by district health board use

Guidelines available but not in use by DHBs N = 84 Guidelines in current use by DHBs N = 151 P value AGREE II domains Median scaled domain score a
DHB: District Health Board a Scaled score calculated as: (obtained score -minimum possible score)/(maximum possible score -minimum possible score) b Excludes guidelines already included in the group > 50% of appraisers recommended yes for clinical use

Table 3
Quality of guidelines by type of district health board

for high quality Number of guidelines meeting the criteria for high quality N (%)
DHB: District Health Board a Includes 25 guidelines that are in use by DHBs with secondary and tertiary maternity and neonatal services b Scaled domain score calculated as: (obtained score -minimum possible score)/(maximum possible score -minimum possible score) c Excludes guidelines already included in the group > 50% of appraisers would recommend yes for clinical use

Table 4
Responses to modified ADAPTE questions

constraints and limitations to implementing the recommendations
Limited service capacity "Constraints on [antenatal clinic] appointments to have women seen in an appropriate timeframe" "Bed capacity to offer admission for all women electing for expectant management [of preterm rupture of membranes]" "Bed availability for inpatient insulin and blood sugar management [for women with diabetes]" "NICU cot availability at local tertiary hospital and need to transfer elsewhere" Culturally considered care "Diabetes is over-represented in Māori and Pasifika populations who are also the demographics that suffer disproportion-

Table 5
Themes identified from modified ADAPTE questions