Placental growth factor for the prognosis of women with preeclampsia (fullPIERS model extension): context matters

Abstract

Background

The fullPIERS risk prediction model was developed to identify which women admitted with confirmed diagnosis of preeclampsia are at highest risk of developing serious maternal complications. The model discriminates well between women who develop (vs. those who do not) adverse maternal outcomes. It has been externally validated in several populations. We assessed whether placental growth factor (PlGF), a biomarker associated with preeclampsia risk, adds incremental value to the fullPIERS model.

Methods

Using a cohort of women admitted into tertiary hospitals in well-resourced settings (the USA and Canada), between May 2010 to February 2012, we evaluated the incremental value of PlGF added to fullPIERS for prediction of adverse maternal outcomes within 48 h after admission with confirmed preeclampsia. The discriminatory performance of PlGF and the fullPIERS model were assessed in this cohort using the area under the receiver’s operating characteristic curve (AUROC) while the extended model (fullPIERS +PlGF) was assessed based on net reclassification index (NRI) and integrated discrimination improvement (IDI) performances.

Results

In a cohort of 541 women delivered shortly (< 1 week) after presentation, 8.1% experienced an adverse maternal outcome within 48 h of admission. Prediction of adverse maternal outcomes was not improved by addition of PlGF to fullPIERS (NRI: -8.7, IDI − 0.06). Discriminatory performance (AUROC) was 0.67 [95%CI: 0.59–0.75] for fullPIERS only and 0.67 [95%CI: 0.58–0.76]) for fullPIERS extended with PlGF, a performance worse than previously documented in fullPIERS external validation studies (AUROC > 0.75).

Conclusions

While fullPIERS model performance may have been affected by differences in healthcare context between this study cohort and the model development and validation cohorts, future studies are required to confirm whether PlGF adds incremental benefit to the fullPIERS model for prediction of adverse maternal outcomes in preeclampsia in settings where expectant management is practiced.

Background

Pre-eclampsia and other hypertensive disorders of pregnancy complicate approximately 10% of pregnancies and contribute considerably to maternal, fetal, and newborn morbidity and mortality, worldwide [1]. Preeclampsia, defined broadly as hypertension with symptoms or signs of end-organ compromise, can lead to severe maternal complications (e.g., eclampsia, stroke, and liver dysfunction) and/or fetal complications (e.g., stillbirth and preterm delivery) [2, 3]. Early identification of women at high risk of these complications can guide care and prevent delays in treatment in order to prevent poor outcomes [2].

The fullPIERS risk prediction model was developed to facilitate early identification of women admitted with confirmed preeclampsia who are at greatest risk of developing severe maternal complications (e.g. eclampsia and stroke, see Appendix S1 for full list of outcomes) within 48 h of admission [4]. The published model (equation presented in Table 1) showed good discriminatory performance with an area under the receiver-operating characteristic curve (AUROC) of 0.88 (95% confidence interval [CI], 0.84–0.92) upon internal validation and AUROC 0.81 (95% CI 0.75–0.86) upon external validation [5]. Thus, the model can aid in risk stratification, to allow for corticosteroid administration, transfer to higher care facilities, and plan for delivery for high-risk women.

Table 1 Original versus Extended fullPIERS Logistic Regression Equations for the prediction of adverse maternal outcomes from pre-eclampsia

Since the development of fullPIERS, new biomarkers have been introduced that could aid the identification of adverse outcomes in preeclampsia. One such biomarker is placental growth factor (PlGF), an angiogenic factor found in the maternal circulation [6, 7]. Plasma concentrations of PlGF are decreased in pregnancies complicated by preeclampsia compared with uncomplicated pregnancies [6,7,8,9,10]. Several studies have tested the diagnostic ability of PlGF for women with suspected preeclampsia [6, 11,12,13]; the PELICAN study reported 96 and 98% sensitivity and negative predictive value, respectively, using PlGF <5th percentile, to predict confirmed preeclampsia and subsequent delivery within 14 days among women presenting with suspected preeclampsia before 35 + 0 weeks of gestation [11]. Similarly, in a cluster-randomized trial, PlGF < 100 pg/ml identified women (95 and 98% sensitivity and negative predictive value, respectively), with suspected preeclampsia who delivered within 14 days with confirmed pre-eclampsia [12]; these findings were consistent in the PreEclampsia Triage by Rapid Assay (PETRA) trial with a sensitivity of 92.5% and specificity of 63.8% [14]. However, fewer studies have aimed to investigate the prognostic value of PlGF in women with confirmed preeclampsia. In women with suspected preeclampsia, high discriminatory performance or strong likelihood ratios (LRs) have been demonstrated for PlGF for combined adverse maternal and fetal outcomes (usually the need for delivery with pre-eclampsia within 7–14 days) [10, 14,15,16,17,18]; however there are limited studies reporting on solely maternal outcomes [18,19,20,21,22].

Based on these reports, we evaluated whether addition of PlGF to fullPIERS could improve prediction of adverse maternal outcomes in women with confirmed preeclampsia.

Methods

This manuscript was prepared using the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) reporting guidelines for prediction models (Table S2) [23].

Study cohort

For this study, we used the PETRA data which included women that had PlGF measurements during admission [8, 14]. The original data originally consisted of a prospective cohort of 1217 women presenting with suspected symptoms and signs of preeclampsia with measurable PlGF values, at 24 maternity units in the United States of America (USA; N = 23) and Canada (N = 1) (November 2010 to January 2012) [14]. Women from the PETRA [8, 14] cohort who had confirmed preeclampsia were included in the study.

PlGF measurement

Maternal plasma samples were collected from eligible women between 20 and 35 weeks’ gestation, during the study recruitment. Samples were tested for PlGF using the Triage PlGF Test (Quidel Inc., San Diego, USA) according to the manufacturer’s instructions. We restricted our analyses to women with PlGF measured during hospital admission for confirmed preeclampsia (or at most 14 days before admission), to reduce the potential effect of changing PlGF concentration over time in the analyses and to be consistent with the other fullPIERS variable measurements [4, 5].

Definition of pre-eclampsia and outcomes

To be consistent with the fullPIERS model development study [5], preeclampsia was defined as hypertension plus one of proteinuria, hyperuricemia, or as HELLP (Hemolysis, Elevated Liver enzyme, Low Platelet) syndrome. A composite adverse maternal outcome was defined as the occurrence of maternal death or any maternal morbidity as determined by Delphi consensus (see online [https://pre-empt.cfri.ca/monitoring/fullpiers] and Appendix S1).

Statistical analyses

The distribution of participant characteristics included in this study were compared with those of women in the original fullPIERS development cohort, using Chi-squared and Mann–Whitney U tests (p-value of < 0.05) for categorical and continuous variables, respectively. Univariable analyses were also carried out comparing characteristics of women in our study cohort with those in previous studies, according to low PlGF (< 100 pg/mL) and normal PlGF (≥100 pg/mL) [18].

Model extension with PlGF

PlGF measurements were converted to percentiles based on the reference range by gestational age interval [24]. Before inclusion in the fullPIERS model, we assessed the univariable discriminatory performance of PlGF. Using the most abnormal values of test results of the model predictors within 48 h of admission, a linear predictor variable was calculated for all the women in our study cohort using the fullPIERS model equation (1).

A logistic regression model was fitted with two variables: (i) the linear predictor and (ii) converted PlGF percentiles [7]. Hence, a new intercept and slope were estimated for the fullPIERS model as well as a regression coefficient for PlGF (Table 1). The extended model was then used to calculate the predicted probabilities of experiencing an adverse outcome for each woman and its performance was evaluated.

Prediction performance measures

The ability of PlGF to predict adverse maternal outcomes and the performance of the original fullPIERS model in the data were evaluated based on discrimination capacity. Discriminative ability was assessed using the AUROC and was interpreted using the following pre-specified criteria: non-informative (AUROC ≤0.5), poor discrimination (0.5 < AUROC ≤0.7), good discrimination (AUROC > 0.7) [25]. In addition to discrimination, the extended model performance was evaluated based on the net reclassification index (NRI) and integrated discrimination improvement (IDI) to assess the incremental value of PlGF compared with the performance of the original model [26,27,28].

NRI gives a summary of the overall improvement of sensitivity and specificity associated with addition of PlGF to fullPIERS. NRI integrates the “upward movement” (improved reclassification) and the “downward movement” (worse reclassification) for the women with adverse outcomes in the predicted risk groups with the reverse movements for the women without adverse outcomes [29, 30]. The sum of the differences between the two movements for the women with and without adverse outcomes is the NRI.

IDI is the difference in the discrimination slope between the original model and the extended model. The discrimination slope is calculated as the difference between the average predicted probabilities of the women with and without adverse maternal outcomes. This measurement also incorporates the NRI across all possible cut-offs.

All analyses were done in R version 3.5.1 (R Foundation for Statistical Computing, Vienna, Austria).

Results

The extension cohort included only a subset of women who met our inclusion criteria (N = 541), as previously described (Fig. 1).

Fig. 1

Flow showing identification of study population

Demographics

The women in the extension cohort (compared with those in the original fullPIERS development cohort) were different in a number of ways (Table 2). In general, women admitted with preeclampsia in the original fullPIERS cohort were known to be managed expectantly, whereas those admitted in the extension cohort settings (mostly in the USA) were more likely to be delivered; this was also evident in the shorter admission-to-delivery interval. On average, the women in the extension cohort were younger with lower blood pressure measurements and presenting with earlier-onset of preeclampsia than the women in the fullPIERS cohort. They were also more likely to be multiparous, smokers, and receive magnesium sulfate; have a shorter admission-to-delivery interval with early-onset preeclampsia (gestational age [GA] < 34 weeks); and have babies with lower birth weight, compared with the fullPIERS cohort. There were no meaningful differences in the proportions of multifetal pregnancies or treatment with antihypertensive medication. Adverse maternal outcomes occurred in 8.1% of women within 48 h of admission, 9.6% within 7 days of admission, and 10.5% at any time during admission.

Table 2 Maternal characteristics for the fullPIERS Development cohort vs Extension cohort, (n (%) or median (interquartile range))

PlGF was classified as low (< 100 pg/ml) in 485 women (89.6%) (Table 3). Low maternal PlGF concentrations were more likely among women with earlier onset of preeclampsia, higher blood pressure, and babies with lower birth weight. Low PlGF was present in almost all women who went on to experience adverse maternal (N = 53/57, 93.0%) at any time during admission.

Table 3 Maternal characteristics for Normal vs Low PlGF values in the Extension data, (n (%) or median (interquartile range))

Prediction of adverse maternal outcomes after adding PlGF to fullPIERS

In our analyses, the original fullPIERS model (AUROC 0.67 [95% CI 0.58–0.76]) and PlGF alone (AUROC 0.60 [95% CI 0.52–0.67]) showed poor discriminatory performance for prediction of adverse maternal outcomes within 48 h of admission in the extension cohort (Fig. 2). Addition of PlGF to fullPIERS neither altered the odds of an adverse maternal outcome (OR 0.98 [95% CI 0.99–1.00]) for every percentile increase in PlGF (Table 1), nor discriminatory performance (AUROC 0.67 [95% CI 0.59–0.75]) (Fig. 2).

Fig. 2

Prediction of adverse maternal outcomes using (i) only PlGF (0.60 (95% CI 0.52–0.68)) (ii) original fullPIERS (fp) model (0.67 (95% CI 0.58–0.76)) and (iii) the extended fullPIERS model (0.67 (95% CI 0.59–0.75)) in the extension cohort

With addition of PlGF to fullPIERS, using a threshold of ≥10% for the calculated predicted probabilities, there were eight upward movements of women with adverse maternal outcomes into the high-risk category and two fewer cases without adverse maternal outcomes in the highest-risk category compared with the classification using the original model (Table 4). The overall improvement in specificity was 11.9%, with a decrease of 18.4% in sensitivity using the extended model. Thus, the NRI was − 8.7, indicating no overall improvement in reclassification of risks. The discrimination slope for the extended model (fullPIERS plus PlGF) was 0.03, compared with 0.09 for the original model (Fig. 3). The IDI was − 0.06, also indicating no meaningful improvement in discrimination.

Table 4 Reclassification Table for Extended Model with PlGF (fullPIERS plus PlGF)

Fig. 3

Discrimination slopes for (a) Original model (without PlGF) and (b) Extended model with PlGF (IDI = − 0.06) in the extension cohort

Discussion

Principal findings

In our study comprised of women with confirmed preeclampsia from tertiary hospitals in well-resourced settings, prediction of adverse maternal outcomes was not improved by addition of PlGF to fullPIERS. Of note is that fullPIERS performed poorly in this extension cohort compared with other studies where the AUROC was > 0.75 [4, 5, 31].

Results compared to other studies

In the original fullPIERS model development cohort, the model performed well in the prediction of adverse maternal outcomes (AUROC 0.88, 95% CI 0.84–0.92). Similarly, in methodologically rigorous and robustly conducted temporal and external validation studies [4, 5], the model also performed well, (AUROC of 0.82 (95% CI 0.76–0.87) for temporal and 0.81 (95% CI 0.75–0.86) for external validation). In another study assessing the model in women with early-onset of pre-eclampsia, the discriminatory performance of the model was AUROC of 0.80 (95% 0.75–0.86). However, in this study, the model performance was lower, even upon the addition of PlGF. These findings are similar to an abstract assessing the fullPIERS model in a cohort of women admitted with pre-eclampsia in the United States, which reported a lower performance (AUROC: 0.68, 95% CI 0.60–0.76) for adverse maternal outcomes [32]. The fullPIERS was developed and externally validated in settings where expectant management was the practice. We hypothesize that this difference in model performance may be due to differences in pre-eclampsia management between the original fullPIERS setting and the other hospital settings in the USA. Further research to test this hypothesis would be valuable.

In contrast with our study, the PARROT trial reported lower incidence of adverse maternal outcomes in women with suspected preeclampsia (rather than confirmed preeclampsia as studied here), when PlGF values were revealed to the clinician, compared with the concealed group [12]. However, in the PETRA study [14] (from which our study extension cohort was derived), clinicians were masked to PlGF values and, therefore, could not have been influenced by the results. Although the PARROT trial results suggest that clinicians might positively respond to low PlGF possibly by increasing surveillance for women potentially at higher risk of adverse outcomes, it did not assess the prognostic value of PlGF for adverse maternal outcomes in women with preeclampsia.

Strengths

To our knowledge, this is the first study to assess the addition of PlGF to fullPIERS for prediction of adverse maternal outcome in women with confirmed preeclampsia. Our use of a composite outcome that warrants consistent clinical action makes the results of our study robust and clinically relevant. This evaluation of an externally-validated model provides benchmark performance estimates against which comparisons can be made for different locations and upon addition of new biomarkers.

Limitations

The main limitation of our study was having limited power due to a relatively small sample size and number of adverse outcomes. Based on our small sample size, we did not re-estimate the fullPIERS model coefficients when extending the model with PlGF as this would lead to significant overfitting of the model in the data set [33]. However, this may have been necessary if the addition of PlGF were to change the predictive coefficients of the other variables in the model. Second, we may have underestimated the relationship between PlGF and adverse outcome because some PlGF measurement results were carried forward (by no more than 14 days), rather than ideally have been measured within 48 h of admission. Third, practice varied in our study settings; expectant care (pregnancy prolongation) was less frequently practised in the extension cohort setting, so the natural history of disease was truncated by expedited delivery. In contrast, in the fullPIERS model development cohort, expectant care was the norm, resulting in a longer admission to delivery interval (mean of 11 days, compared with 5 days in the extension cohort – Table 2).

Clinical implications

Despite our study limitations outlined above, there are possible explanations for the poor predictive performance for adverse maternal outcomes observed in our study. PlGF is a marker of placental dysfunction; therefore, this angiogenic factor may be more reflective of the initiation of preeclampsia (i.e. from suspected to established pre-eclampsia), rather than the progression or disease severity after the diagnosis of preeclampsia. Thus, PlGF appears to be less useful as a prognostic marker for women with confirmed pre-eclampsia in settings where less expectant management is practised.

Of note, almost 90% of included women had low PlGF suggesting a higher risk cohort. Such a case-mix is likely to reduce prediction performance, as more population homogeneity is associated with lower discrimination [34]. However, the high proportion of low PlGF values supports in the role of this angiogenic marker as a risk marker of preeclampsia development; women with low PlGF were more likely to have higher blood pressure measurements and experience worse neonatal outcomes.

Conclusion

In our study, the addition of PlGF did not improve the performance of the fullPIERS model in predicting adverse maternal outcomes in the extension cohort. Given the poor performance of fullPIERS for prediction of adverse maternal outcomes in this study compared with previous studies, we speculate that our findings may relate to differences in case-mix and/or context between this cohort and the original fullPIERS model development cohort. These data suggest that fullPIERS may not be predictive of maternal outcome in settings where expedited delivery for preeclampsia is the standard of care. Given the paucity of relevant datasets for these analyses, future work is needed to evaluate the incremental benefit of adding PlGF or other biomarkers e.g. sFLT-1 to fullPIERS, a model based on routinely collected maternal history, physical examination, and laboratory variables, especially in healthcare settings with expectant management.