Skip to main content

Implementation of a first-trimester prognostic model to improve screening for gestational diabetes mellitus

Abstract

Background

Improvement in the accuracy of identifying women who are at risk to develop gestational diabetes mellitus (GDM) is warranted, since timely diagnosis and treatment improves the outcomes of this common pregnancy disorder. Although prognostic models for GDM are externally validated and outperform current risk factor based selective approaches, there is little known about the impact of such models in day-to-day obstetric care.

Methods

A prognostic model was implemented as a directive clinical prediction rule, classifying women as low- or high-risk for GDM, with subsequent distinctive care pathways including selective midpregnancy testing for GDM in high-risk women in a prospective multicenter birth cohort comprising 1073 pregnant women without pre-existing diabetes and 60 obstetric healthcare professionals included in nine independent midwifery practices and three hospitals in the Netherlands (effectiveness-implementation hybrid type 2 study). Model performance (c-statistic) and implementation outcomes (acceptability, adoption, appropriateness, feasibility, fidelity, penetration, sustainability) were evaluated after 6 months by indicators and implementation instruments (NoMAD; MIDI).

Results

The adherence to the prognostic model (c-statistic 0.85 (95%CI 0.81–0.90)) was 95% (n = 1021). Healthcare professionals scored 3.7 (IQR 3.3–4.0) on implementation instruments on a 5-point Likert scale. Important facilitators were knowledge, willingness and confidence to use the model, client cooperation and opportunities for reconfiguration. Identified barriers mostly related to operational and organizational issues. Regardless of risk-status, pregnant women appreciated first-trimester information on GDM risk-status and lifestyle advice to achieve risk reduction, respectively 89% (n = 556) and 90% (n = 564)).

Conclusions

The prognostic model was successfully implemented and well received by healthcare professionals and pregnant women. Prognostic models should be recommended for adoption in guidelines.

Peer Review reports

Background

Gestational diabetes mellitus (GDM) is a common disorder in pregnancy with an estimated incidence of five to 10 % [1, 2]. GDM is associated with short- and long-term complications affecting both mother and child, such as macrosomia and diabetes mellitus type 2 [3,4,5]. Timely diagnosis and treatment of GDM improves pregnancy outcomes, therefore, improvement in the accuracy of identifying women who are at risk to develop GDM is warranted [6,7,8,9].

Many countries recommend universal testing for GDM by means of an oral glucose tolerance test (OGTT) in the second or third trimester of pregnancy as part of standard obstetric care [1]. However, in low-risk populations testing is often performed selectively in women with one or more risk factors for GDM to prevent many women without GDM being subjected to a burdensome OGTT stressing healthcare budgets and logistics. However, GDM diagnoses could be missed with this strategy [1].

Prognostic models for GDM have the potential improve selective testing [10]. Such models, which weigh and combine readily available predictors, have already been externally validated and outperform current risk factor based selective screening approaches [10,11,12]. Implementation of a prognostic model has the potential to reduce the number of women unnecessarily undergoing a burdensome oral glucose tolerance test (OGTT) and, consequently, healthcare expenditure. However, at present there is little evidence about the performance, impact and usefulness of prognostic models for GDM in day-to-day obstetric care [13].

The implementation of prognostic models in healthcare is challenging and complex. Contrary to current GDM screening approaches, prognostic models require additional action when predictors have to be entered into the model to generate a woman’s risk [14, 15]. Professionals have to integrate the model into their daily work routine while being influenced by group processes, social conventions, organisational factors and social structures they operate in (normalisation process) [16]. It is important to study outcomes that measure implementation success to advance understanding of implementation processes [17]. Ideally, such an evaluation to assess whether a prognostic model for GDM is feasible for daily clinical use, should be performed before adoption in clinical guidelines [13]. After all, an innovation is not likely to be effective when it is not implemented properly.

Therefore, the aim of this study was to evaluate the implementation of an externally validated prognostic model to improve selective screening for GDM through the assessment of implementation outcomes and model performance involving both obstetric healthcare professionals (OHP) and pregnant women. The identified barriers and facilitators may also provide insights into enhancing implementation of healthcare innovations in other settings.

Methods

Design and study population

We performed an effectiveness-implementation hybrid type 2 study in a prospective multicentre regional birth cohort to evaluate both implementation outcomes and performance of a first-trimester prognostic model for GDM. The study was performed between December 2016 to January 2018 in nine midwifery practices and three hospitals in the central region of the Netherlands (Risk EStimation for PrEgnancy Complications to provide Tailored care study part two (RESPECT2): implementation of a prognostic model for GDM).

The implementation was evaluated among OHP from the participating centres. All obstetric healthcare professionals, including midwives working in independent midwifery practices (primary care) or clinical midwives, residents in obstetrics and obstetricians employed in hospitals (secondary or tertiary care) who used the model were asked to participate. All pregnant women in the participating centres received the replaced standard care (i.e. the prognostic model for GDM). Pregnant women ≥18 years old, were asked to provide informed consent for use of medical data for research purposes and to fill out questionnaires. Exclusion criteria were < 18 years old, inability to provide informed consent or to respond to online questionnaires in Dutch. The RESPECT2 study was approved by the medical ethics committee of the University Medical Centre Utrecht on 15 November 2016 (protocol 16/741).

Selective screening for GDM by the prognostic model

All participating centres replaced their current screening approach in which testing is performed selectively in women with one or more risk factors for GDM [14], by the prognostic model for GDM as standard care during the study. The model contained the following predictors: maternal age, body mass index (BMI), ethnicity, first-degree relative with any type of diabetes mellitus, history of GDM and first-trimester venous glucose. The full equation plus information on threshold consensus is shown in Box S1 in Additional file 1. An implementation plan was set up using the refined compilation of implementation strategies as defined by Powell et al. 2015 (Box S2 in Additional file 1) [18].

The prognostic model was applied by OHP through a secured online data-collection platform (Research Online©). The result was displayed as either low- or high risk with subsequent directive recommendations for care. A two-page leaflet to take home was discussed with high-risk women. This included general information on GDM, risk of GDM, preventive measures and their care pathway. The care pathway included testing for GDM at 24–28 weeks of gestation with an OGTT and an ultrasound examination to check for macrosomia or polyhydramnios at 30–32 weeks of gestation. In the part ‘What can I do?’, healthy gestational weight gain, exercise (at least 30 min a day) and healthy nutrition with referral to the website of Netherlands Nutrition Centre for pregnant women was advised. Also, consultation with a dietician was suggested. Low-risk women only received information on their risk for GDM. Note that all pregnant women, regardless of their GDM risk-status, underwent an OGTT when they developed symptoms of GDM at any point in pregnancy. See Box S3 in Additional file 1 for the diagnosis and treatment of GDM.

Implementation outcomes

Implementation outcomes as defined by Proctor et al. (adoption, acceptability, appropriateness, feasibility, fidelity, penetration, sustainability) were assessed 6 months after implementation using the validated NoMAD and MIDI instruments plus additional questions and indicators [17, 19,20,21,22]. The NoMAD is based on the Normalization Process Theory, which states that normalization is a process of embedding and integrating healthcare innovations in routine care as a product of action of individuals and groups [20]. The four constructs in this theoretic framework are: coherence, cognitive participation, collective action and reflexive monitoring [19, 20]. The MIDI is developed to identify factors influencing the use of an implemented intervention comprising of four constructs measuring determinants associated with the innovation, user, organisation and socio-political context [22].

In Table 1 it is shown how the implementation outcomes were defined in the context of our study and through which items they were assessed. The full subconstructs and details about scaling of the survey administered to OHP are provided in Table S1 in Additional file 1.

Table 1 Definition of implementation outcomes and their assessment

Data collection

OHP received a survey 6 months after implementation (reminders after 3, 7 and 10 days). Collected demographics included age in years, sex, profession (midwife employed in an independent midwifery practice, midwife employed in a hospital, resident in obstetrics, obstetricians), and years of OHP experience.

Characteristics of pregnant women were documented by OHP via the model and pregnancy outcomes were extracted from medical records by the research team in a standardised case report form (Box S4 in Additional file 1). Pregnant women received a questionnaire at 22 weeks of gestation and 4 weeks after their estimated date of delivery including questions and statements on risk communication and lifestyle in relation to pregnancy outcome (reminders after seven and 14 days).

Statistical analysis

Women with a multiple pregnancy, fetal demise < 16 weeks of gestation or who were lost to follow-up were excluded for analysis. Characteristic and outcomes of pregnant women were assessed for the overall study population as well as stratified for risk (high-risk versus low-risk) and GDM diagnosis. A responder versus non-responder analysis was performed for both OHP and pregnant women; a pregnant woman was defined as a responder when she filled out one or both questionnaire(s).

Model performance was assessed by c-statistics and calibration-plots. The sensitivity and specificity of the prognostic model were compared to the formerly used selective screening approach [14]. A reclassification plot illustrated how the proportions of women who were true positive, true negative, false positive or false negative for GDM altered by using the prognostic model compared to the former approach.

The subconstructs of the NoMAD and MIDI instruments were scored on a 5-point Likert scale from 0 (strongly disagree) to 5 (strongly agree). Subconstructs to which ≥20% of the healthcare professionals responded with ‘strongly disagree’ or ‘disagree’ were considered barriers, and those to which ≥80% responded ‘strongly agree’ or ‘agree’ were considered facilitators [23]. A single composite score for a subconstruct consisting of multiple statements, total scores for the instruments and the combined total score were calculated by averaging the statements, the scores of all constructs and the total scores of both instruments, respectively. Internal consistency of the constructs and total scores was assessed with Cronbach’s Alpha for all subconstructs and when the least coherent subconstruct was dropped (α: < 0.50 ‘unacceptable’, 0.50–0.59 ‘poor’, 0.60–0.69 ‘questionable’, 0.70–0.79 ‘acceptable’, 0.80–0.89 ‘good’, ≥0.90 ‘excellent’).

Analyses were performed in SPSS statistics version 25 (SPPS Inc., Chicago, IL) and R version 3.5.1 for windows using ‘rms’ and ‘calibrate’ packages (http://cran.r-project.org).

Results

Study population

The majority of the 60 OHP was female (n = 58; 97%) and worked as midwifes in primary care (n = 45; 75%) (Table 2). The number of responders to the survey 6 months after implementation was 42 (70%). No significant differences in demographics were observed between responders and non-responders.

Table 2 Determinants of healthcare professionals employed at the implementation sites overall and stratified by responder type

A total of 1940 pregnant women were assessed for eligibility of whom 798 met one of the exclusion criteria or did not provide informed consent. Another 69 women were excluded because they had a multiple pregnancy (n = 27), fetal demise < 16 weeks of gestation (n = 11) or were lost to follow-up (n = 31), leaving 1073 women for analysis. The age and BMI of the study population were 31.4 ± 4.3 years and 23.7 (IQR 21.4–27.3) kg/m2, respectively. The prognostic model classified 352 women (32.8%) as high-risk for GDM. Gestational diabetes mellitus was diagnosed in 81 women (7.5%) of whom 71 (87.7%) were classified as high-risk. Characteristics and outcomes of the study population overall and stratified for risk for and diagnosis of GDM are described in Table 3. Rates of high-risk status for and diagnosis of GDM were similar between the 672 responders (62.6%) and 401 non-responders (37.4%) to the questionnaires, however, non-responders were slightly younger and more often of non-Caucasian ethnicity (Table S2 in Additional file 1).

Table 3 Characteristics of the study population overall, and stratified for risk for and diagnosis of gestational diabetes mellitus

Model performance

The prognostic model showed good discrimination, illustrated by a c-statistic of 0.85 (95%CI 0.81–0.90), and adequate calibration (Figure S1). The prognostic model classified 4.3% fewer women as high-risk for GDM (32.8% compared to 37.1%) and classified 5.3% more women correctly than the former approach (Fig. 1). Moreover, the prognostic model outperformed the former approach on all test characteristics illustrated by a sensitivity and specificity of 87.7 (95%CI 30.0–35.7) and 71.7 (95%CI 68.8–74.5) compared to 81.5 (95%CI 71.3–89.2) and 66.5 (95%CI 63.5–69.5).

Fig. 1
figure1

Reclassification plot of the former approach for selective screening for gestational diabetes mellitus compared to a prognostic model

Implementation outcomes

OHP were predominantly positive about the model with an overall score on implementation outcomes of 3.7 (IQR 3.3–4.0) (Table S3 in Additional file 1). Scores were not associated with the respondent’s age (p = 0.735), sex (p = 0.571), profession (p = 0.736), or number of years employed in obstetrics (p = 0.862). The internal consistency of the MIDI and NoMAD total scores were good to excellent (0.86 ≤ alpha≤91 and 0.85 ≤ alpha≤0.88, respectively) (Table S3 in Additional file 1).

Adoption

All centres that made the initial decision to adopt the prognostic model for GDM also started the implementation process. On an individual level, the majority (n = 34; 81%) of OHP was open to start working with the model (Fig. 2; CP3).

Fig. 2
figure2

Distribution of responses of obstetric healthcare professionals on NoMAD (a) and MIDI (b) subcontructs 6 months after the implementation of a prognostic model for gestational diabetes mellitus

Acceptability

Most OHP scored positive on the procedural clarity of the prognostic model, adequate knowledge to use it and on how the model affects the nature of their work (n = 33; 79%, n = 34; 81%, n = 36; 86%, respectively) (Fig. 2; I1, U17, C3). However, some reported no awareness of scientific reports about the effects of the innovation (n = 15; 36%), and reported no improvement client relationships as a personal drawback (n = 10; 24%) (Fig. 2; RM1; U8).

Appropriateness

OHP scored neutral to positive on all subconstructs concerning their perceived fit, relevance or compatability of the prognostic model. They found the prognostic model relevant for their clients (n = 30; 71%) and had confidence in their colleagues’ ability to use the model (n = 34; 81%) (Fig. 2; I7, CA3). OHP expected that high-risk women would accept testing on GDM (n = 38; 91%), whereas they scored neutral on the outcome expectations regarding effects of lifestyle advice (Fig. 2; U9). OHP scored a 7.5 (IQR 6.5–8.1), on a scale from 0 = ‘not al all’ to 10 = ‘completely’, on the question “When you use the prognostic model, how familiar does it feel?” (Table S3 in Additional file 1; past normality).

Pregnant women appreciated first-trimester information on GDM risk-status and lifestyle advice to achieve risk reduction, in both the second trimester (n = 556; 89% and n = 564; 90%) and postpartum (n = 437; 95% and n = 433; 94%). These results were similar for low- and high-risk women.

Feasibility

OHP ability to discuss lifestyle advice with high-risk women was identified as a facilitator (n = 36; 86%). They also scored positive on self-efficacy to apply the prognostic model as intended (n = 33; 79%), and agreed that they could count on adequate assistance from colleagues when needed (n = 32; 76%) (Fig. 2; U13, U16). OHP scored a 7.0 (IQR 5.1–8.2), on a scale from 0 = ‘not al all’ to 10 = ‘completely’, on the question “Do you feel that the prognostic model is currently a normal part of your work?” (Table S3 in Additional file 1; current normality). Barriers included lack of staff capacity (n = 10; 24%), time available (n = 13; 31%), training (n = 11; 26%), resources (n = 11; 26%), support (n = 12; 29%) and formal ratification by the management (n = 27; 64%), compatibility to (n = 8; 19%) and easy integration (n = 9; 21%) into their existing work, as well as the presence of concurrent innovations (n = 37; 88%) (Fig. 2; O21, O23, CA5, CA6, CA7, O19, I5, CA1, O26).

Fidelity

Testing for GDM by the prognostic model was performed as intended in 1021 women (95%) (Fig. 3).

Fig. 3
figure3

Flowchart of testing for gestational diabetes mellitus after implementation of a prognostic model

In the low-risk group, 694 (96%) women received care conforming to protocol. In 24 (3%) low-risk women where the advice of the prognostic model was violated, the former selective testing approach for GDM was applied. Other reasons were: hyperglycaemia-inducing medication (n = 1), wrong risk-status assigned due to administrative error (n = 1), or unknown (n = 1).

In the high-risk group, 327 (93%) women received care conforming to protocol. The reasons for the 25 (7%) protocol violations were: testing refused by pregnant women (n = 3), not advised by OHP (n = 3), wrong risk-status assigned due to administrative error (n = 7), or unknown (n = 12). Withdrawal for risk factor based testing for GDM was not considered as a protocol violation in seven high-risk women, because they had a preterm birth (n = 1) or already received an OGTT because of symptoms (n = 6) before testing was planned.

The impact of the prognostic model in clinical practice, combining model performance and fidelity, resulted in 84.0% safety (n = 68/81) and 70.3% efficiency (n = 697/992).

Penetration

The prognostic model was filled out for 1038 (97%) women. OHP thought that most of their direct colleagues applied the model as intended and also appointed them as the most important influencers on whether or not to apply the prognostic model (Fig. 2; U15; U14).

Low-risk women reported less often that they were informed about their risk for GDM compared to high-risk women, 269 (62%) and 168 (86%) women, respectively. From the women who did receive information, 350 (80%) women remembered their risk correctly, 36 (8%) wrong and 51 (12%) did not remember their risk for GDM; these percentages were not different for low-risk compared to high-risk women (p = 0.199).

Sustainability

A total of 27 (64%) OHP would like to continue using the prognostic model; ten (24%) of them responded neutral. Eight (19%) OHP did not find the effect of using the model clearly observable, 13 (31%) of them reported negative on replacement when staff leaves and 14 (33%) on performance feedback (Fig. 2; I6, O22, O28). With regard to the prognostic model becoming a normal part of their work in the future, OHP scored an 8.0 (IQR 7.0–9.0) on a scale from 0 (not at all) to 10 (completely) (Table S3 in Additional file 1; future normality). They preferred that the model would be integrated in the electronic patient file (n = 36; 85%) and felt that feedback can be used to improve the prognostic model (n = 34; 81%).

Discussion

Main findings

In a prospective regional cohort of 1073 pregnant women, we showed that a first-trimester prognostic model was successfully implemented as a directive clinical prediction rule, classifying women as low- or high-risk for GDM with distinctive care pathways. This was illustrated by the excellent fidelity, good model performance and impact on selective screening for GDM. Moreover, the application of the model and the subsequent care pathways were well received by OHP and pregnant women.

Strengths and limitations

The major strength of this study is the multifaceted approach, since it is advocated that the impact of prognostic models should be evaluated as complex interventions because the introduction of a model with subsequent management recommendations consists of multiple interacting factors (i.e. model performance, implementations outcomes, stakeholders’ perspectives) [25, 26]. Furthermore, comprehensive evaluation of implementation outcomes in studies introducing a prognostic model are scarce [26], especially using validated quantitative NoMAD and MIDI instruments. Additionally, our prospective study of a large multicentre cohort of pregnant women recruited in all types of obstetric care facilities with little missing data of pregnancy outcomes had the preferable design to measure the implementation process [19, 22].

The inclusion of a control group would have been informative with regard to assessing improvements in medical outcomes, i.e. a step-wedged cluster randomized controlled trial, however, such a design is far more time-consuming and does often not reflect ‘real’ clinical practice [26]. The use of online surveys enabled us to reach many OHP and pregnant women, although combining this data with qualitative deepening of the reasons behind responses could have provided even more insights into factors influencing the implementation process. Also, subjects with a positive attitude towards the model may have been more eager to respond, and may thus be overrepresented. Although response rates were acceptable and no major differences in characteristics were found between responders and non-responders this could, to some extent, have led to a non-response bias. The primary aim of this study was to evaluate the implementation of a prognostic model to improve selective testing for GDM, therefore it was not possible to apply universal testing at the same time. This may have resulted in some missed GDM cases in the low-risk group. We assume that this concerns a small proportion, since women who developed symptoms of GDM at any point in pregnancy did receive an OGTT. Nevertheless, these false-negatives might have led to overestimation of model performance. Although a formal combined diet and exercise intervention was not part of the care pathway of high-risk women, it is possible that the included lifestyle advice may have decreased the GDM incidence in this group to some extent, which might have led to underestimation of model performance.

Interpretation

This is one of few studies evaluating the introduction of a prognostic model in obstetric care [13]. By our knowledge there has only been one previous study that evaluated the impact of prediction tools for several adverse pregnancy outcomes, including GDM, on perinatal outcomes and costs in a before-after study [27]. This study found a significant reduction of a composite of perinatal outcomes in nulliparous women and was cost-effective [27]. However, it is unknown to what degree the prognostic model for GDM contributed to these results and neither implementation outcomes nor stakeholders’ perspectives were evaluated.

Our study identified several facilitators for the implementation of a prognostic model for GDM in routine obstetric care. We found that the field was ready for its introduction and that OHP had confidence in client cooperation, positive outcome expectations, enough knowledge and understanding how the model affected their work. Most importantly, OHP had confidence in their colleagues’ and their own ability to apply the model for every woman and to discuss lifestyle advice with high-risk women. This is important since self-efficacy has been found to be the strongest predictor for completeness of use [22, 28, 29]. In case of non-adherence, we identified similar reasons as in a recent review on impact analysis studies of clinical prediction rules, including: fear of missing the diagnosis, preference for own clinical judgement and patient request [30]. Finally, it is encouraging that OHP think that feedback about the model and corresponding care pathways can be used to improve these, as this is a commonly reported facilitator that could enhance further integration and maintenance in clinical practice [21, 31,32,33,34,35].

Despite scores on self-efficacy, which were high, a number of OHP did not find it easy to integrate the model into their routine and reported lack of training and awareness of reports about model effectiveness. In contrast to personal knowledge and skills, the other barriers concerned external factors regarding organisation or management, such as insufficient time, staff capacity, management support, formal ratification and resources. Nevertheless, these impeding factors might comprise more than only the implementation of the prognostic model, since 88% of OHP reported that other changes in the organisation affected the implementation with the subsequent stress and heavy workload hindering implementation [36]. The identified themes are in line with previously identified barriers regarding the use of clinical prediction rules in practice [26]. Most barriers were found in the constructs ‘collective action’ (NoMAD) and ‘organisation’ (MIDI) concerning feasibility. Moreover, lack of formal ratification by management was found to be strongly associated with unsuccessful implementation in previous studies [22, 29]. These findings may indicate that the implementation strategy was predominantly a bottom-up approach and that improvements can be made with more emphasis on how OHP work together within an organisation to enact the model in their daily routine and by more comprehensive involvement of the management [20, 21].

The performance of the implemented model in our study was higher than previous development, external validation and update studies with a c-statistic of 0.85 (95%CI 0.81–0.90) compared to c-statistics ranging from 0.70 (95%CI 0.68–0.73) to 0.80 (95%CI 0.76–0.84) [10,11,12, 37]. In clinical practice, this resulted in less pregnant women requiring testing for GDM while more GDM cases were timely identified, which may reduce perinatal morbidity and subsequent healthcare expenditure [7, 9].

Conclusions

A first-trimester prognostic model for GDM was successfully implemented into obstetric care and was well received by OHP and pregnant women. We therefore recommended that prognostic models for GDM should be considered for adoption in obstetric guidelines. All areas using selective risk factor based testing for GDM could potentially benefit from using a prognostic model [38, 39]. Generalisability should be examined before implementing a prognostic model in a new population as model updating might be required to account for differences such as disease prevalence and predictor distribution [10, 40]. Furthermore, this study aggregated knowledge on how we could possible enhance our implementation strategy. The identified facilitators and barriers may as well provide insights into enhancing implementation of other (healthcare) innovations in various settings. Future studies are needed to broadly validate and improve quantitative implementation instruments in various healthcare fields. Additionally, normative data and directives for reporting are needed in order to classify what scores define a successful implementation and to make comparisons possible, serving both implementation research and practice.

Availability of data and materials

The datasets analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

BMI:

Body mass index

GDM:

Gestational diabetes mellitus

MIDI:

Measurement Instrument for Determinants of Innovations instrument

NoMAD:

Normalization MeAsure Development Instrument

OGTT:

Oral glucose tolerance test

OHP:

Obstetric healthcare professional(s)

RESPECT2:

Risk EStimation for PrEgnancy Complications to provide Tailored care study part two

References

  1. 1.

    Chiefari E, Arcidiacono B, Foti D, Brunetti A. Gestational diabetes mellitus: an updated overview. J Endocrinol Investig. 2017;40(9):899–909. https://doi.org/10.1007/s40618-016-0607-5.

    CAS  Article  Google Scholar 

  2. 2.

    Eades CE, Cameron DM, Evans JMM. Prevalence of gestational diabetes mellitus in Europe: a meta-analysis. Diabetes Res Clin Pract. 2017;129:173–81. https://doi.org/10.1016/j.diabres.2017.03.030.

    Article  PubMed  Google Scholar 

  3. 3.

    Farrar D, Simmonds M, Bryant M, Sheldon TA, Tuffnell D, Golder S, et al. Hyperglycaemia and risk of adverse perinatal outcomes: systematic review and meta-analysis. BMJ. 2016;354:i4694.

    Article  Google Scholar 

  4. 4.

    Burlina S, Dalfra MG, Lapolla A. Short- and long-term consequences for offspring exposed to maternal diabetes: a review. J Matern Fetal Neonatal Med. 2019;32(4):687–94. https://doi.org/10.1080/14767058.2017.1387893.

    CAS  Article  PubMed  Google Scholar 

  5. 5.

    Retnakaran R. Hyperglycemia in pregnancy and its implications for a woman’s future risk of cardiovascular disease. Diabetes Res Clin Pract. 2018;145:193–9. https://doi.org/10.1016/j.diabres.2018.04.008.

    Article  PubMed  Google Scholar 

  6. 6.

    Alwan N, Tuffnell DJ, West J. Treatments for gestational diabetes. Cochrane Database Syst Rev. 2009;2009(3):CD003395. https://doi.org/10.1002/14651858.CD003395.pub2.

  7. 7.

    Landon MB, Spong CY, Thom E, Carpenter MW, Ramin SM, Casey B, et al. A multicenter, randomized trial of treatment for mild gestational diabetes. N Engl J Med. 2009;361(14):1339–48. https://doi.org/10.1056/NEJMoa0902430.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  8. 8.

    Viana LV, Gross JL, Azevedo MJ. Dietary intervention in patients with gestational diabetes mellitus: a systematic review and meta-analysis of randomized clinical trials on maternal and newborn outcomes. Diabetes Care. 2014;37(12):3345–55. https://doi.org/10.2337/dc14-1530.

    Article  PubMed  Google Scholar 

  9. 9.

    Crowther CA, Hiller JE, Moss JR, McPhee AJ, Jeffries WS, Robinson JS, et al. Effect of treatment of gestational diabetes mellitus on pregnancy outcomes. N Engl J Med. 2005;352(24):2477–86. https://doi.org/10.1056/NEJMoa042973.

    CAS  Article  PubMed  Google Scholar 

  10. 10.

    van Hoorn F, Koster M, Naaktgeboren CA, Groenendaal F, Kwee A, Lamain-de Ruiter M, Franx A, Bekker MN. Prognostic models versus single riskfactor approach in first-trimester selective screening for gestational diabetes mellitus: a prospective population-based multicentre cohort study. BJOG. 2021;128(4):645–54. https://doi.org/10.1111/1471-0528.16446. Epub 2020 Sep 1.

  11. 11.

    Teede HJ, Harrison CL, Teh WT, Paul E, Allan CA. Gestational diabetes: development of an early risk prediction tool to facilitate opportunities for prevention. Aust N Z J Obstet Gynaecol. 2011;51(6):499–504. https://doi.org/10.1111/j.1479-828X.2011.01356.x.

    Article  PubMed  Google Scholar 

  12. 12.

    Lamain-de Ruiter M, Kwee A, Naaktgeboren CA, de Groot I, Evers IM, Groenendaal F, et al. External validation of prognostic models to predict risk of gestational diabetes mellitus in one Dutch cohort: prospective multicentre cohort study. BMJ. 2016;354:i4338.

    Article  Google Scholar 

  13. 13.

    Kleinrouweler CE, Cheong-See FM, Collins GS, Kwee A, Thangaratinam S, Khan KS, et al. Prognostic models in obstetrics: available, but far from applicable. Am J Obstet Gynecol. 2016;214(1):79–90.e36. https://doi.org/10.1016/j.ajog.2015.06.013.

    Article  PubMed  Google Scholar 

  14. 14.

    Nederlandse Vereniging voor Obstetrie & Gynaecologie. Diabetes mellitus en zwangerschap [Guideline]. 2010 [updated 04-06-2010]. Available from: https://www.nvog.nl/wp-content/uploads/2018/02/Diabetes-mellitus-en-zwangerschap-2.0-04-06-2010.pdf.

    Google Scholar 

  15. 15.

    National Institute for Health and Care Excellence. Diabetes in pregnancy: management from preconception to the postnatal period [Guideline]: National Collaborating Centre for Women’s and Children’s Health; 2015. [updated August 2015]. Available from: https://www.nice.org.uk/guidance/ng3

    Google Scholar 

  16. 16.

    May CR, Mair F, Finch T, MacFarlane A, Dowrick C, Treweek S, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009;4(1):29. https://doi.org/10.1186/1748-5908-4-29.

    Article  PubMed  PubMed Central  Google Scholar 

  17. 17.

    Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Admin Pol Ment Health. 2011;38(2):65–76. https://doi.org/10.1007/s10488-010-0319-7.

    Article  Google Scholar 

  18. 18.

    Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21. https://doi.org/10.1186/s13012-015-0209-1.

    Article  PubMed  PubMed Central  Google Scholar 

  19. 19.

    Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, et al. Improving the normalization of complex interventions: part 2 - validation of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18(1):135. https://doi.org/10.1186/s12874-018-0591-x.

    Article  PubMed  PubMed Central  Google Scholar 

  20. 20.

    Rapley T, Girling M, Mair FS, Murray E, Treweek S, McColl E, et al. Improving the normalization of complex interventions: part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18(1):133. https://doi.org/10.1186/s12874-018-0590-y.

    Article  PubMed  PubMed Central  Google Scholar 

  21. 21.

    Vis C, Ruwaard J, Finch T, Rapley T, de Beurs D, van Stel H, et al. Toward an objective assessment of implementation processes for innovations in health care: psychometric evaluation of the Normalization Measure Development (NoMAD) questionnaire among mental health care professionals. J Med Internet Res. 2019;21(2):e12376. https://doi.org/10.2196/12376.

    Article  PubMed  PubMed Central  Google Scholar 

  22. 22.

    Fleuren MA, Paulussen TG, Van Dommelen P, Van Buuren S. Towards a measurement instrument for determinants of innovations. Int J Qual Health Care. 2014;26(5):501–10. https://doi.org/10.1093/intqhc/mzu060.

    Article  PubMed  PubMed Central  Google Scholar 

  23. 23.

    Verberne LM, Kars MC, Schepers SA, Schouten-van Meeteren AYN, Grootenhuis MA, van Delden JJM. Barriers and facilitators to the implementation of a paediatric palliative care team. BMC Palliat Care. 2018;17(1):23. https://doi.org/10.1186/s12904-018-0274-8.

    Article  PubMed  PubMed Central  Google Scholar 

  24. 24.

    Hoftiezer L, Hof MHP, Dijs-Elsinga J, Hogeveen M, Hukkelhoven C, van Lingen RA. From population reference to national standard: new and improved birthweight charts. Am J Obstet Gynecol. 2019;220(4):383.e1–e17.

    Article  Google Scholar 

  25. 25.

    Kappen TH, van Klei WA, van Wolfswinkel L, Kalkman CJ, Vergouwe Y, Moons KGM. Evaluating the impact of prediction models: lessons learned, challenges, and recommendations. Diagn Progn Res. 2018;2(1):11. https://doi.org/10.1186/s41512-018-0033-6.

    Article  PubMed  PubMed Central  Google Scholar 

  26. 26.

    Cowley LE, Farewell DM, Maguire S, Kemp AM. Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature. Diagn Progn Res. 2019;3(1):16. https://doi.org/10.1186/s41512-019-0060-y.

    Article  PubMed  PubMed Central  Google Scholar 

  27. 27.

    van Montfort P, Scheepers HCJ, Dirksen CD, van Dooren IMA, van Kuijk SMJ, Meertens LJE, Wijnen EJ, Zelis M, Zwaan IM, Spaanderman MEA, Smits LJM. Impact on perinatal health and cost-effectiveness of risk-based care in obstetrics: a before-after study. Am J Obstet Gynecol. 2020;223(3):431.e1-431.e18. https://doi.org/10.1016/j.ajog.2020.02.036. Epub 2020 Feb 26.

  28. 28.

    Klaassen LA, Friesen-Storms J, Bours G, Dirksen CD, Boersma LJ, Hoving C. Perceived facilitating and limiting factors for healthcare professionals to adopting a patient decision aid for breast cancer aftercare: a cross-sectional study. Patient Educ Couns. 2020;103(1):145–51. https://doi.org/10.1016/j.pec.2019.07.024.

    Article  PubMed  Google Scholar 

  29. 29.

    van der Kleij RM, Crone MR, Reis R, Paulussen TG. Critical stakeholder determinants to the implementation of intersectoral community approaches targeting childhood obesity. Health Educ Res. 2016;31(6):697–715. https://doi.org/10.1093/her/cyw047.

    Article  PubMed  Google Scholar 

  30. 30.

    Wallace E, Uijen MJ, Clyne B, Zarabzadeh A, Keogh C, Galvin R, et al. Impact analysis studies of clinical prediction rules relevant to primary care: a systematic review. BMJ Open. 2016;6(3):e009957. https://doi.org/10.1136/bmjopen-2015-009957.

    Article  PubMed  PubMed Central  Google Scholar 

  31. 31.

    Stewart D, Maclure K, Newham R, Gibson-Smith K, Bruce R, Cunningham S, Maclure A, Fry S, Mackerrow J, Bennie M. A cross-sectional survey of the pharmacy workforce in general practice in Scotland. Fam Pract. 2020;37(2):206–12. https://doi.org/10.1093/fampra/cmz052.

  32. 32.

    Elf M, Nordmark S, Lyhagen J, Lindberg I, Finch T, Aberg AC. The Swedish version of the Normalization Process Theory Measure S-NoMAD: translation, adaptation, and pilot testing. Implement Sci. 2018;13(1):146. https://doi.org/10.1186/s13012-018-0835-5.

    Article  PubMed  PubMed Central  Google Scholar 

  33. 33.

    Bryce C, Fleming J, Reeve J. Implementing change in primary care practice: lessons from a mixed-methods evaluation of a frailty initiative. BJGP Open. 2018;2(1):bjgpopen18X101421.

    Article  Google Scholar 

  34. 34.

    Gillespie BM, Harbeck E, Lavin J, Gardiner T, Withers TK, Marshall AP. Using normalisation process theory to evaluate the implementation of a complex intervention to embed the surgical safety checklist. BMC Health Serv Res. 2018;18(1):170. https://doi.org/10.1186/s12913-018-2973-5.

    Article  PubMed  PubMed Central  Google Scholar 

  35. 35.

    Sturgiss EA, Elmitt N, Haesler E, van Weel C, Douglas K. Feasibility and acceptability of a physician-delivered weight management programme. Fam Pract. 2017;34(1):43–8. https://doi.org/10.1093/fampra/cmw105.

    Article  PubMed  Google Scholar 

  36. 36.

    Goodridge D, Rana M, Harrison EL, Rotter T, Dobson R, Groot G, et al. Assessing the implementation processes of a large-scale, multi-year quality improvement initiative: survey of health care providers. BMC Health Serv Res. 2018;18(1):237. https://doi.org/10.1186/s12913-018-3045-6.

    Article  PubMed  PubMed Central  Google Scholar 

  37. 37.

    Meertens LJE, Scheepers HCJ, van Kuijk SMJ, Roeleveld N, Aardenburg R, van Dooren IMA, et al. External validation and clinical utility of prognostic prediction models for gestational diabetes mellitus: a prospective cohort study. Acta Obstet Gynecol Scand. 2020;99(7):891–900. https://doi.org/10.1111/aogs.13811.

    Article  PubMed  PubMed Central  Google Scholar 

  38. 38.

    Farrar D, Simmonds M, Bryant M, Lawlor DA, Dunne F, Tuffnell D, et al. Risk factor screening to identify women requiring oral glucose tolerance testing to diagnose gestational diabetes: a systematic review and meta-analysis and analysis of two pregnancy cohorts. PLoS One. 2017;12(4):e0175288. https://doi.org/10.1371/journal.pone.0175288.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  39. 39.

    Benhalima K, Van Crombrugge P, Moyson C, Verhaeghe J, Vandeginste S, Verlaenen H, et al. Risk factor screening for gestational diabetes mellitus based on the 2013 WHO criteria. Eur J Endocrinol. 2019;180(6):353–63. https://doi.org/10.1530/EJE-19-0117.

    CAS  Article  PubMed  Google Scholar 

  40. 40.

    Moons KG, Kengne AP, Grobbee DE, Royston P, Vergouwe Y, Altman DG, et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart. 2012;98(9):691–8. https://doi.org/10.1136/heartjnl-2011-301247.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge dr. Henk F van Stel, who unfortunately passed away in 2018, for enthusiastically sharing his expertise in implementation science and his invaluable contribution to the design of this study. We would like to express our gratitude and appreciation to all women and obstetric healthcare professionals who participated in the RESPECT study part two. Special thanks go to all members of the RESPECT 2 study group (Anjoke J.M. Huisjes, Marije Lamain-de Ruiter, Eva Stekkinger, Simone A. Vankan-Buitelaar, Mariska A.A.W. Vonk). We would like to thank Maartje Snoep and Laura van Zijtveld for their contributions on data collection. We would like to acknowledge the University of Utrecht for supporting open access publication.

Funding

The RESPECT study was conducted with the support of the Netherlands Organisation for Health Research and Development (project number 209020004 and 2090100031). The funding source had no role in the design, conduct, analyses, or reporting of the study or in the decision to submit the manuscript for publication.

Author information

Affiliations

Authors

Consortia

Contributions

All authors (A.F., A.K., F.G., F.H., M.N.B., M.P.H.K.) were involved in the study design. F.H. coordinated the acquisition of data. F.H. and M.P.H.K. performed data-analysis. F.H., M.N.B., M.P.H.K. were involved in the initial interpretation of data and drafting of the manuscript. All authors were responsible for interpretation of data and critical revision of the manuscript. All authors agreed on the final version to be published. All authors had full access to the data (including statistical reports and tables) in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. The guarantors (A.F., M.N.B.) accept full responsibility for the work and/or the conduct of the study, had access to the data, and controlled the decision to publish. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Corresponding author

Correspondence to Mireille N. Bekker.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the medical ethics committee of the University Medical Centre Utrecht on 15 November 2016 (protocol 16/741). Written informed consent was obtained from all pregnant women included in the study. All methods were performed in accordance with the relevant guidelines and regulations (Declaration of Helsinki).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Table S1.

Full statements of the implementation survey administered to obstetric healthcare professionals to evaluate the implementation of a prognostic model for gestational diabetes mellitus including the NoMAD (a), MIDI (b), and extra evaluation (c). Table S2. Characteristics and outcomes of pregnant women stratified by responder type. Table S3. Median scores and internal consistency of MIDI and NoMAD instruments. Table S4. List of participating investigators of the RESPECT2 study. Box S1. Equation of the prognostic model for gestational diabetes mellitus. Box S2. Implementation strategies. Box S3. Diagnosis and treatment of gestational diabetes mellitus. Box S4. Definitions of characteristics and pregnancy outcomes. Figure S1. Receiver-operator-curve (a) and calibration plot (b) of the implemented prognostic model for gestational diabetes mellitus.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

van Hoorn, F., Koster, M.P.H., Kwee, A. et al. Implementation of a first-trimester prognostic model to improve screening for gestational diabetes mellitus. BMC Pregnancy Childbirth 21, 298 (2021). https://doi.org/10.1186/s12884-021-03749-x

Download citation

Keywords

  • Prognostic model
  • Clinical prediction rule
  • Gestational diabetes mellitus
  • Selective screening
  • Implementation
  • Impact analysis
  • NoMAD
  • MIDI
  • Normalization process theory
  • Risk communication