- Open Access
Development and internal validation of a model predicting severe maternal morbidity using pre-conception and early pregnancy variables: a population-based study in Ontario, Canada
BMC Pregnancy and Childbirth volume 21, Article number: 679 (2021)
Improvement in the prediction and prevention of severe maternal morbidity (SMM) - a range of life-threatening conditions during pregnancy, at delivery or within 42 days postpartum - is a public health priority. Reduction of SMM at a population level would be facilitated by early identification and prediction. We sought to develop and internally validate a model to predict maternal end-organ injury or death using variables routinely collected during pre-pregnancy and the early pregnancy period.
We performed a population-based cohort study using linked administrative health data in Ontario, Canada, from April 1, 2006 to March 31, 2014. We included women aged 18–60 years with a livebirth or stillbirth, of which one birth was randomly selected per woman. We constructed a clinical prediction model for the primary composite outcome of any maternal end-organ injury or death, arising between 20 weeks’ gestation and 42 days after the birth hospital discharge date. Our model included variables collected from 12 months before estimated conception until 19 weeks’ gestation. We developed a separate model for parous women to allow for the inclusion of factors from previous pregnancy(ies).
Of 634,290 women, 1969 experienced the primary composite outcome (3.1 per 1000). Predictive factors in the main model included maternal world region of origin, chronic medical conditions, parity, and obstetrical/perinatal issues – with moderate model discrimination (C-statistic 0.68, 95% CI 0.66–0.69). Among 333,435 parous women, the C-statistic was 0.71 (0.69–0.73) in the model using variables from the current (index) pregnancy as well as pre-pregnancy predictors and variables from any previous pregnancy.
A combination of factors ascertained early in pregnancy through a basic medical history help to identify women at risk for severe morbidity, who may benefit from targeted preventive and surveillance strategies including appropriate specialty-based antenatal care pathways. Further refinement and external validation of this model are warranted and can support evidence-based improvements in clinical practice.
Severe maternal morbidity (SMM) covers a range of conditions along the continuum to maternal death during pregnancy or within 42 days after delivery . Maternal morbidity is a substantial public health concern  whose incidence is rising in Canada and the US . These patterns are driven by multiple risk factors including delaying childbearing, use of assisted reproductive technologies, rising rates of obesity, and Caesarean delivery . Because of the low prevalence of maternal mortality in many industrialized countries, data covering several years are required to compute precise estimates of prevalence and risk factors, thus complicating the use of maternal mortality as a population health indicator [2, 5]. Consequently, SMM has received increasing attention as an indicator of perinatal health and obstetric care [6, 7].
As the focus in industrialized countries such as Canada has shifted towards ‘near miss’ events as a means to improving the health and quality of care for pregnant women , prediction of SMM has been identified as a critical research gap in obstetrics . Many maternal characteristics are known pre-conception or early in pregnancy and are strong risk factors for the development of SMM [2, 8]. Therefore, a combination of such factors may reliably predict its onset, enabling evidence-based and rational early triage of high-risk women for enhanced surveillance and subspecialty-based care.
Advances in maternal morbidity risk prediction include a US obstetric comorbidity index , which was externally validated within a Canadian population, resulting in modest discrimination (C-statistic of 0.66, 95% confidence interval [CI] 0.65–0.67) . That index included variables that both preceded, and were simultaneous with, the onset of SMM, making it a useful research tool for identifying the burden of morbidity but less so for clinical prediction. Others have developed models focused on specific subtypes of maternal morbidity, such as cardiovascular-related conditions . Models predicting maternal mortality include the Collaborative Integrated Pregnancy High-dependency Estimate of Risk (CIPHER) model (C-statistic 0.82, 95% CI 0.81–0.84) and the Maternal Severity Index (C-statistic 0.83, 95% CI 0.80–0.85) , both developed among women either already critically ill or hospitalized, and mostly later in gestation.
Since SMM predominantly arises around birth or early postpartum , the ideal timeframe for prediction is before or early in pregnancy to facilitate effective preventive strategies such as referral to high-risk centres or shared-care antenatal care pathways [12, 13]. Existing models do not enable these latter steps, nor do they account for important pre-pregnancy factors, such as maternal infertility and its treatment, which are associated with SMM . Additionally, existing prediction efforts did not consider prior adverse pregnancy outcomes among parous women. We therefore undertook the current study to develop and internally validate a clinical prediction model of SMM, defined as a composite of maternal end-organ injury or death, using readily available factors ascertained pre-pregnancy and prior to 20 weeks’ gestation in a population-based study in Ontario – Canada’s most populous and multi-ethnic province.
The use of data in this project was authorized under section 45 of Ontario’s Personal Health Information Protection Act, which does not require review by a Research Ethics Board. We followed the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guideline for reporting of prediction studies .
Population and data sources
All women with a pregnancy lasting beyond 20 weeks’ gestation, and who delivered within an Ontario hospital between April 1, 2006 and March 31, 2014, were identified within the Better Outcomes Registry & Network (BORN) databases . Data beyond 2014 were not available in these datasets. The BORN registry captures over 99% of hospital births in the province, and has been validated for data completeness and accuracy [17, 18]. We used the Registered Persons Database, the Immigration, Refugees and Citizenship Canada’s Permanent Resident Database, the Ontario Health Insurance Plan (OHIP) outpatient claims database, and the Canadian Institute for Health Information (CIHI) Discharge Abstract Database to capture maternal demographics, pre-existing health conditions and diagnoses and procedures documented during a hospitalization (see Table S1 for variables and diagnostic codes used to develop the study cohort). The datasets were linked using unique encoded identifiers and analysed at ICES – a not-for-profit provincial research entity that houses a large network of health administrative databases (https://www.ices.on.ca/).
We excluded ectopic pregnancies, pregnancies resulting in abortion or miscarriage or ending before 20 weeks’ gestation. We randomly sampled one birth (live- or stillbirth) per woman to avoid potential within-person correlations among women with multiple pregnancies (Table S1; Figure S1).
The primary composite outcome was maternal end-organ injury or death arising between 20 weeks’ gestation and up to 42 days after the index birth hospital discharge date. The list of conditions used to define maternal end-organ injury was based on the model developed by Bateman  and validated by Metcalfe , comprising 20 diagnoses and procedures, and consistent with Canadian perinatal surveillance definitions for SMM and death [19,20,21] (Table S1).
A secondary outcome was all-cause maternal mortality, from birth until 365 days postpartum, since previous work has shown a persistent increase in mortality risk beyond the early postpartum period [22, 23].
Candidate predictors, variable selection, and coding
Demographic, medical and obstetric factors known to be associated with an increased risk of SMM were considered as candidate predictors. These included: estimated maternal age at conception (continuous, categorical, and squared terms); residential income quintile; world region of origin (Table S2), as a proxy for both maternal birthplace and ethnicity; attendance at a first-trimester prenatal care visit; pre-pregnancy body mass index (BMI); parity; multiple gestation; infertility; infertility treatment; placental disorders (e.g., placenta praevia, placenta accreta); and pre-existing medical conditions coded within 12 months before the estimated date of conception (Table S1). Substantial missing data were noted only for the variable pre-pregnancy BMI (63.8%). We tested models in which BMI was modelled as a continuous variable and where missing values were assigned the median BMI (24.2 kg/m2). We further tested models in which BMI was divided into the following categories: < 18.5 kg/m2, 18.5–24.9 kg/m2 (reference category), 25–29.9 kg/m2, > 30 kg/m2, and missing. Certain categorical variables with a low frequency in the cohort were combined with other similar variables (e.g., pre-existing cardiovascular conditions; placental conditions and anomalies). Variables were also assessed for collinearity by checking the variance inflation factor (VIF), and where collinear (VIF > 5), the most commonly reported variable was selected .
In the model restricted to the sub-cohort of parous women, in addition to the above variables, we included complications coded in any previous pregnancy as predictors (Table S1).
Possible interactions between variables were assessed and included if statistically significant at alpha = 0.10 .
We used standardized differences to contrast births with and without the primary composite outcome of maternal end-organ injury or death, with a value > 0.10 indicating an important difference in baseline characteristics .
Among the entire cohort, a logistic regression model was fit using the final selected variables to predict the primary composite outcome of maternal acute end-organ injury or death from 20 weeks’ gestation until 42 days postpartum. A backward elimination method was applied for variable selection, with predictor evaluation based on a balance of the model’s C-statistic, clinical influence, and statistical significance. For continuous predictor variables such as age in which non-linear associations with the outcome were observed, a quadratic (squared) term was added to the model. Model discrimination was expressed as a C-statistic and its 95% CI, as well as visual detection of a receiver operating curve (ROC). We considered a C-statistic of < 0.5 to be not useful, 0.5 to 0.6 poor, 0.6 to 0.7 moderate, and ≥ 0.7 as good .
Model internal validation
To arrive at an optimism-corrected C-statistic, we used a bootstrapping approach, with 500 bootstrap samples selected from the original cohort, with replacement  – an approach known to produce stable estimates with low risk of bias . The optimism-corrected C-statistic was defined as the C-statistic from the original data minus the optimism value .
Model calibration was assessed by visual inspection of calibration plots of observed vs. expected probabilities of the outcome, where a 45-degree line denotes good calibration, and a slope of 1 indicates perfect agreement between observed and expected events .
We used a risk classification table and computed likelihood ratios (LRs)  with associated 95% CI to assess the main model’s ability to stratify the population as low or high-risk. We divided the population into five groups of predicted probability: very low risk (< 1.5 per 1000), low risk (1.5 to 3 per 1000), intermediate risk (3 to 5 per 1000), high risk (5 to 15 per 1000) and very high risk (> 15 per 1000). These cut-offs were chosen based on the overall incidence of our primary outcome of 3.1 per 1000, which we assumed to reflect the risk among the majority of the cohort. Positive LRs of > 5 and > 10 were interpreted as moderately or very useful “rule-in” tests, while values between 0.2 and 0.5, and < 0.1 were considered moderately and very useful “rule-out” tests .
This study was supported by funding from the Canadian Institutes of Health Research (grant number 15139).
After sampling one birth per woman from among 853,517 eligible births, the total cohort comprised 634,290 births (Figure S1). The primary outcome of end-organ injury or death from 20 weeks’ gestation up to 42 days postpartum occurred in 1969 women (3.1 per 1000), including 62 deaths (0.1 per 1000). Women who experienced the primary outcome were older, more likely to have a pre-existing medical condition, and to have had infertility treatment (Table 1).
The most frequent factors contributing to end-organ injury or death were acute heart failure (40.6%), need for assisted ventilation (29.2%), acute renal failure (12.0%) and shock (10.1%) (Table 2).
Model discrimination and internal validation
In the overall cohort (n = 634,290), variables significantly associated with the composite outcome of maternal end-organ injury or death included maternal age, low income, world region of origin, high BMI, pre-existing medical conditions, and placental disorders (Table S3), which contributed to the final model. Attendance at a first-trimester antenatal visit and parity were inversely associated with the composite outcome. The corresponding model C-statistic was 0.68 (95% CI 0.66–0.69) (Fig. 1). There was minimal overfitting of the model, with mean optimism of 0.0055 (95% CI 0.0050–0.0061), and an optimism-corrected C-statistic of 0.67 (95% CI 0.66–0.68). Model discrimination was unchanged when BMI was included, either as a categorical variable with “missing” as a separate category, or as a continuous variable imputed with the median value for BMI. We tested 300 pairwise interactions, of which 13 interactions were statistically significant. The main model including interaction terms resulted in similar model discrimination as the main model (C-statistic 0.69, 95% CI 0.68–0.70), however this model included unstable estimates. Therefore, the model without interaction terms was chosen as the most balanced and efficient model.
All-cause mortality from birth until 365 days postpartum occurred in 194 women over the study time period (0.3 per 1000). The final multivariable model for all-cause mortality no longer retained world region of origin, parity, previous spontaneous abortion, and several medical comorbidities (Table S4). Major psychiatric conditions and alcohol and substance use newly emerged as predictors. The corresponding C-statistic was 0.70 (95% CI 0.66–0.74) (Figure S2). However, this model was slightly over-fitted, and the optimism-corrected C-statistic was 0.67 (95% CI 0.63–0.71).
The risk classification table for the main model, dividing the cohort according to the five categories of predicted risk of acute end-organ injury or death (Table 3) demonstrated the capacity of this model to classify women who are at very low risk (−LR 0.41, 95% CI 0.33–0.52) and those at very high risk of the outcome (+LR 8.58, 95% CI 7.32–10.05), but was less useful in classifying women in intermediate risk categories.
Sub-cohort of parous women
In the sub-cohort of 333,435 parous women, the aforementioned variables significantly associated with end-organ injury or death persisted, as did the addition of an unplanned Caesarean delivery and severe organ injury in a previous birth (Table S7). The C-statistic was 0.61 (95% CI 0.59–0.63) when limited to variables from the index pregnancy (Fig. 2a), rising to 0.69 (95% CI 0.67–0.70) after adding pre-pregnancy predictors (Fig. 2b), and 0.71 (95% CI 0.69–0.73) when including the variables from a previous pregnancy (Fig. 2c). We noted minimal overfitting for each model. With optimism-corrected C-statistics of 0.60 (95% CI 0.58–0.62), 0.68 (95% CI 0.66–0.70), and 0.70 (95% CI 0.69–0.72), respectively.
Model fit and calibration
Visual inspection of the calibration plots in the entire cohort suggested good agreement between observed and expected events for the primary outcome, with slightly worse calibration for mortality (Figure S3a-b). Among parous women, model calibration for maternal end-organ injury or death improved from the base model to models including variables measured pre-pregnancy and in a previous pregnancy (Figure S3c-e).
We have shown that a model based on variables available pre-pregnancy and in early pregnancy can moderately discriminate women destined for a severely morbid event or death from those likely to have uncomplicated pregnancies. Predictive variables retained in the final models included demographic, obstetric, and other medical risk factors. Notably, attendance at the first trimester visit with a care provider – a measure of good prenatal care – was inversely associated with the risk of SMM. Inclusion of prior pregnancy factors, which have not been incorporated in previous predictive models for SMM, further enhanced model performance, in keeping with the importance of clinical obstetrical history. Our model displayed good calibration, indicating that a combination of routinely measured pre-pregnancy and early pregnancy factors can estimate the absolute risk of acute end-organ injury or death with reasonable accuracy. Using this model effectively increased the probability of identifying a very high-risk woman with this outcome by 40%, and reduced the probability in someone considered very low-risk by 20% , but was less useful in classifying women in intermediate risk categories. This suggests that additional clinical, laboratory, or paraclinical factors are needed to accurately predict morbidity in all women, and further, that a certain proportion of these events are truly sudden and unpredictable.
Strengths and limitations
The models in this study relied on information that is routinely known at the time of the first antenatal visit, using variables that were temporally remote from when most maternal morbid events arise – largely around the time of birth . Moreover, our source population comprised all pregnancies from gestational week 20. However, our datasets had few routinely collected clinical measures, such as blood pressure and haemoglobin or glucose concentrations, or first-trimester screening biomarkers. In the prediction of preterm preeclampsia, for example, a model that contained a combination of clinical and paraclinical variables (including placental biomarkers) performed better than with either set of variables in isolation . BMI was incomplete in our dataset, as is common in most administrative data sources. However, the proportion in any given BMI category and those with missing values was not appreciably different among women with and without the outcome. Furthermore, there were few substantial differences in other baseline characteristics between those with missing vs. non-missing BMI (Table S8). Thus, while the contribution of BMI to the outcome may not have been well represented in our models, this unlikely changed the overall model performance. In addition, clinical practice around identification and management of SMM has evolved over time. It is plausible that the strength of different risk factors for SMM may have changed across the study period as well (e.g., use of lower-risk IVF strategies). However, we used a constant definition for the study outcomes, and any changes in clinical practice patterns would not affect the internal validity of our models.
Prediction models are often used to estimate an individual’s absolute risk of a serious adverse event that might be mitigated with the use of a particular therapy, while avoiding subjecting individuals at low predicted risk to potential harmful effects of such therapy . In obstetrics, serious adverse events are rare, with limited options for targeted prevention. We acknowledge, therefore, the limitations afforded by the C-statistic to discriminate between individuals with and without a rare adverse event, in which a high false positive rate might be justified . The LRs add clinical meaning to the model and serve as a foundation for what might be considered reasonable predictability of rare but catastrophic obstetric events. The high LR of the model for women with very high predicted risk despite the rarity of the outcome in this group speaks to the potential utility of the model as a screening tool.
The models in this study relied on information that is routinely known at the time of the first antenatal visit, and that is temporally remote from when most morbid events arise – around the time of birth . Our main model shows the potential utility of harnessing data in early pregnancy to predict a variety of later adverse maternal outcomes. Consistent with previous research on postnatal mortality , our model for all-cause mortality showed substance use, alcohol use, and psychiatric conditions to be significant predictors of death up to 365 days postpartum.
SMM rates have stagnated within Western nations, yet evidence-based strategies to reduce their burden are lacking . Despite the possibility for early identification and prevention of some forms of SMM, current practice guidelines do not incorporate recommendations for prediction of severe morbidity, and use narrow crude definitions to identify such events . Further refinement of clinical prediction models and the eventual development of a clinical risk calculator may help to inform early triage of women for enhanced surveillance or referral to subspecialty care or shared-care antenatal pathways – decisions that at present rely principally on clinical judgment. In developing and refining our study’s model in external cohorts, investigators should consider adding first-trimester placental biomarkers and other maternal biomarkers alongside routinely measured clinical variables, such as blood pressure and weight. The incorporation of such variables may facilitate prediction of the whole of severe morbidity as well as cause-specific outcomes, and better inform individualized and targeted prevention .
In conclusion, a model developed using pre-pregnancy and early pregnancy predictors available within administrative datasets had moderate prediction of maternal acute end-organ injury or death, and as such shows significant promise in the early clinical prediction of SMM. The addition of factors from a prior pregnancy among parous women slightly improved the model performance. Enhancement of these models, using direct clinical measures, and by external validation or using machine learning, is needed.
Availability of data and materials
The dataset from this study is held securely in coded form at ICES. While data sharing agreements prohibit ICES from making the dataset publicly available, access may be granted to those who meet pre-specified criteria for confidential access, available at www.ices.on.ca/DAS. The full dataset creation plan and underlying analytic code are available from the authors upon request, understanding that the computer programs may rely upon coding templates or macros that are unique to ICES and are therefore either inaccessible or may require modification.
Ray JG, Park AL, Dzakpasu S, et al. Prevalence of SMM and factors associated with maternal mortality in Ontario, Canada. JAMA Netw Open. 2018;1(7):e184571.
Lindquist A, Knight M, Kurinczuk JJ. Variation in SMM according to socioeconomic position: a UK national case-control study. BMJ Open. 2013;3(6):e002742.
Dayan N, Fell DB, Guo Y, et al. SMM in women with high BMI in IVF and unassisted singleton pregnancies. Hum Reprod. 2018;33(8):1548–56.
D'Alton ME, Bonanno CA, Berkowitz RL, et al. Putting the “M” back in maternal-fetal medicine. Am J Obstet Gynecol. 2013;208(6):442–8.
van Roosmalen J, Zwart J. Severe acute maternal morbidity in high-income countries. Best Pract Res Clin Obstet Gynaecol. 2009;23(3):297–304.
Firoz T, Chou D, von Dadelszen P, et al. Measuring maternal health: focus on maternal morbidity. Bull World Health Organ. 2013;91(10):794–6.
Malhame I, Danilack VA, Raker CA, et al. Cardiovascular SMM in pregnant and postpartum women: development and internal validation of risk prediction models. BJOG. 2021;128(5):922–32.
Aoyama K, Pinto R, Ray JG, et al. Association of maternal age with SMM and mortality in Canada. JAMA Netw Open. 2019;2(8):e199875.
Bateman BT, Mhyre JM, Hernandez-Diaz S, et al. Development of a comorbidity index for use in obstetric patients. Obstet Gynecol. 2013;122(5):957–65.
Metcalfe A, Lix LM, Johnson J-A, Currie G, Lyon AW, Bernier F, et al. Validation of an obstetric comorbidity index in an external population. BJOG. 2015;122:1748–55.
Aoyama K, D'Souza R, Pinto R, et al. Risk prediction models for maternal mortality: a systematic review and meta-analysis. PLoS One. 2018;13(12):e0208563.
Menard MK. Toward achieving risk-appropriate maternity care: maternal morbidity prediction. Obstet Gynecol. 2019;134(2):213–5.
Macones GA. Understanding and reducing serious maternal morbidity: a step in the right direction. Obstet Gynecol. 2013;122(5):945–6.
Dayan N, Joseph KS, Fell DB, et al. Infertility treatment and risk of SMM: a propensity score-matched cohort study. CMAJ. 2019;191(5):E118–27.
Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350:g7594.
BORN Ontario. Legacy data elements. 2019. https://www.bornontario.ca/en/data/legacy-data-elements.aspx. Accessed 13 Sept 2019.
Dunn S, Lanes A, Sprague AE, et al. Data accuracy in the Ontario birth registry: a chart re-abstraction study. BMC Health Serv Res. 2019;19(1):1001.
Dunn S, Bottomley J, Ali A, Walker M. 2008 Niday Perinatal Database quality audit: report of a quality assurance project. Chronic Dis Inj Can. 2011;32(1):32–42.
Joseph KS, Liu S, Rouleau J, Kirby RS, Kramer MS, Sauve R, et al. SMM in Canada, 2003 to 2007: surveillance using routine hospitalization data and ICD-10CA codes. J Obstet Gynaecol Can. 2010;32(9):837–46.
Main EK, Abreo A, McNulty J, et al. Measuring SMM: validation of potential measures. Am J Obstet Gynecol. 2016;214(5):643.e641–10.
Roberts CL, Cameron CA, Bell JC, Algert CS, Morris JM. Measuring maternal morbidity in routinely collected health data: development and validation of a maternal morbidity outcome indicator. Med Care. 2008;46(8):786–94.
Grigoriadis S, Wilton AS, Kurdyak PA, et al. Perinatal suicide in Ontario, Canada: a 15-year population-based study. CMAJ. 2017;189(34):E1085–92.
Ray JG, Zipursky J, Park AL. Injury-related maternal mortality. Am J Obstet Gynecol. 2018;219(3):307–8.
Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Multivariate data analysis. Upper Saddle River: Pearson Prentice Hall; 2006.
Steyerberg EW. Clinical prediction models: A practical approach to development, validation, and updating. New York: Springer; 2009.
Austin PC. Using the standardized difference to compare the prevalence of a binary variable between two groups in observational research. Commun Stat Simul Comput. 2009;38(6):1228–34.
Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29.
Steyerberg EW. Mathematics and statistics. Clinical prediction models: a practical approach to development, validation, and updating. New York: Springer; 2009.
Steyerberg EW, Harrell FE Jr, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol. 2001;54(8):774–81.
Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361–87.
Steyerberg EW, Vickerrs AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for some traditional and novel measures. Epidemiology. 2010;21:128–38.
Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. Br Med J. 2004;329(7458):168–9.
McGee S. Simplfying likelihood ratios. J Gen Intern Med. 2002;17(8):647–50.
Akolekar R, Syngelaki A, Poon L, Wright D, Nicolaides KH. Competing risks model in early screening for preeclampsia by biophysical and biochemical markers. Fetal Diagn Ther. 2013;33(1):8–15.
Alba AC, Agoritsas T, Walsh M, et al. Discrimination and calibration of clinical prediction models: users’ guides to the medical literature. JAMA. 2017;318(14):1377–84.
Romero-Brufau S, Huddleston JM, Escobar GJ, Liebow M. Why the C-statistic is not informative to evaluate early warning scores and what metrics to use. Crit Care (London, England). 2015;19:285.
American College of O, Gynecologists, the Society for Maternal-Fetal M, Kilpatrick SK, Ecker JL. SMM: screening and review. Am J Obstet Gynecol. 2016;215(3):B17–22.
Davidson AJ, Park AL, Ray JG. Navigating SMM using big data: green, yellow, and red flags for researchers. Obstet Med. 2019;12(3):105–6.
Dr. Dayan would like to acknowledge Dr. Ugochinyere Ukah for review of the manuscript and suggestions for model refinement.
This study was supported by funding from the Canadian Institutes of Health Research (grant number 15139). The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Dr. Dayan is supported by the Fondation de Recherche du Québec – Santé (FRQS) Chercheur Boursier Clinicien award. This study was also supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). Parts of this material are based on data and information compiled and provided by MOHLTC and CIHI. The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred. This Study is based in part on data provided by Better Outcomes Registry and Network (“BORN”), part of the Children’s Hospital of Eastern Ontario. The interpretation and conclusions contained herein do not necessarily represent those of BORN Ontario. Parts or whole of this material are based on data and/or information compiled and provided by Immigration, Refugees and Citizenship Canada (IRCC) current to May 2017. However, the analyses, conclusions, opinions and statements expressed in the material are those of the author(s), and not necessarily those of IRCC.
Ethics approval and consent to participate
The use of data in this project was authorized under section 45 of Ontario’s Personal Health Information Protection Act, which does not require review by a Research Ethics Board, or individual patient consent. ICES approved the study and all analyses of raw data were conducted at ICES. No other administrative permissions were required.
Consent for publication
The authors declare they have no conflicts of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Variables used to define cohort entry and exclusion criteria, primary outcome, and variables considered for each clinical prediction model. Table S2. List of countries used to define World region of origin among immigrant women included in the study. Table S3. Adjusted odds ratio of maternal end-organ injury or death arising between 20 weeks’ gestation and up to 42 days after birth, in association with variables measured pre-pregnancy and in the index pregnancy prior to 20 weeks’ gestation. Table S4. Adjusted odds ratio of all-cause maternal death arising between birth and up to 365 days after birth, in association with variables measured pre-pregnancy and in the index pregnancy prior to 20 weeks’ gestation. Table S5. Adjusted odds ratio of maternal end-organ injury or death arising between 20 weeks’ gestation and up to 42 days after birth in association with variables measured in the index pregnancy prior to 20 weeks’ gestation. Table S6. Adjusted odds ratio of maternal end-organ injury or death arising between 20 weeks’ gestation and up to 42 days after birth in association with variables measured in the index pregnancy prior to 20 weeks’ gestation and pre-pregnancy. Table S7. Adjusted odds ratio of maternal end-organ injury or death arising between 20 weeks’ gestation and up to 42 days after birth in association with variables measured in the index pregnancy prior to 20 weeks’ gestation, pre-pregnancy, and in any previous pregnancy. Table S8. Baseline characteristics of the study population, according to missing BMI vs. non-missing BMI. Figure S1. Flow diagram of the creation of the study cohort. Figure S2. Receiver operating characteristic curve showing discrimination of the clinical prediction model for all-cause maternal death. Figure S3. a. Calibration plot of predicted (x-axis) and observed (y-axis) deciles of probability of acute end organ injury or death, entire cohort of 634,290 births. b. Calibration plot of predicted (x-axis) and observed (y-axis) deciles of probability of all-cause maternal death, entire cohort of 634,290 births. c. Calibration plot of predicted (x-axis) and observed (y-axis) deciles of probability of acute end organ injury or death, early index pregnancy factors, sub-cohort of 333,435 births among parous women. d. Calibration plot of predicted (x-axis) and observed (y-axis) deciles of probability of acute end organ injury or death, pre-pregnancy and early index pregnancy factors, sub-cohort of 333,435 births among parous women. e. Calibration plot of predicted (x-axis) and observed (y-axis) deciles of probability of acute end organ injury or death, pre-pregnancy, previous pregnancy, and early index pregnancy factors, sub-cohort of 333,435 births among parous women.
About this article
Cite this article
Dayan, N., Shapiro, G.D., Luo, J. et al. Development and internal validation of a model predicting severe maternal morbidity using pre-conception and early pregnancy variables: a population-based study in Ontario, Canada. BMC Pregnancy Childbirth 21, 679 (2021). https://doi.org/10.1186/s12884-021-04132-6
- Women’s health