This article has Open Peer Review reports available.
Long-term health-related and economic consequences of short-term outcomes in evaluation of perinatal interventions
© Teune et al; licensee BioMed Central Ltd. 2010
Received: 20 August 2009
Accepted: 10 August 2010
Published: 10 August 2010
Many perinatal interventions are performed to improve long-term neonatal outcome. To evaluate the long-term effect of a perinatal intervention follow-up of the child after discharge from the hospital is necessary because serious sequelae from perinatal complications frequently manifest themselves only after several years. However, long-term follow-up is time-consuming, is not in the awareness of obstetricians, is expensive and falls outside the funding-period of most obstetric studies. Consequently, short-term outcomes are often reported instead of the primary long-term end-point. With this project, we will assess the current state of affairs concerning follow-up after obstetric RCTs and we will develop multivariable prediction models for different long-term health outcomes. Furthermore, we would like to encourage other researchers participating in follow-up studies after large obstetric trials (> 350 women) to inform us about their studies so that we can include their follow-up study in our systematic review. We would invite these researchers also to join our effort and to collaborate with us on the external validation of our prediction models.
A systematic review of neonatal follow-up after obstetric studies will be performed. All reviews of the Cochrane Pregnancy and Childbirth group will be assessed for reviews on interventions that aimed to improve neonatal outcome. Reviews on interventions primary looking at other aspects than neonatal outcome such as labour progress will also be included when these interventions can change the outcome of the neonate on the short or long-term. Our review will be limited to RCTs with more than 350 women. Information that will be extracted from these RCTs will address whether, how and for how long follow-up has been performed. However, in many cases long-term follow-up of the infants will not be feasible. An alternative solution to limited follow-up could be to develop prediction models to estimate long-term health outcomes of the newborn based on specific perinatal outcomes and other covariates. For the development of multivariable prediction models for several health outcomes, we will use data available from a Dutch cohort study of preterm (< 32 weeks) and/or small for gestational age infants (< 1500 g). These infants were born in The Netherlands in 1983 and followed until they reached the age of 19.
The systematic review will provide insight in the extent and methods used for follow-up assessments after obstetric RCTs in the past. The prediction models can be used by future studies to extrapolate short-term outcomes to a long-term horizon or to indicate for which neonates long-term follow-up is required, as their outcomes (either absence or presence of sequelae) cannot be adequately predicted from short-term outcomes and clinical background characteristics.
Many perinatal interventions are performed to improve long-term neonatal outcome. To evaluate the long-term effect of a perinatal intervention follow-up of the child after discharge from the hospital is necessary because serious sequelae from perinatal complications frequently manifest themselves only after several years . However, long-term follow-up is time-consuming, is not in the awareness of obstetricians, is expensive and falls outside the funding-period of most obstetric studies. Consequently, short-term outcomes are often reported instead of the primary long-term end-point.
The relevance of long-term outcomes is illustrated by the following two Cochrane reviews. In the first Cochrane review the implications of antibiotics for preterm rupture of membranes (erythromycin and/or amoxicillin-clavulanate) were evaluated. However, none of the 22 studies included in this review reported on long-term outcomes, but these data supported the routine use of antibiotics in premature rupture of membranes . Recently, the results of the 7-year follow-up of one of these included studies were published. The prescription of antibiotics for women with preterm rupture of the membranes seems to have little effect on the health of children at 7 years of age, so the findings of decreased short-term morbidity from antibiotics have not translated in long-term benefits .
A second Cochrane review where women at risk for preterm labour with intact membranes were given prophylactic antibiotics (erythromycin and/or amoxicillin-clavulanate) identified 11 studies, without any study reporting long-term follow-up . This review failed to demonstrate any overall benefit from prophylactic antibiotic treatment for preterm labour with intact membranes. Also recently, the long-term results of one of these included studies were published. The use of erythromycin by women at risk for preterm labour and with intact membranes was associated with an increase in functional impairment among their children at 7 years of age. The risk of cerebral palsy was increased by either antibiotic, although the overall risk of this condition was low . These results support the advice that antibiotics are not recommended in spontaneous preterm labour with intact membranes and that the description of antibiotics can even be harmful in these circumstances.
If long-term follow-up of the infants is not feasible, an alternative is to model the long-term consequences based on short-term neonatal outcomes. This could be realised by developing prediction models, in which the association between short-term and long-term outcomes is determined statistically, and adjusted for relevant covariates. Models are a way of representing the complexity of the real world in a more simple and comprehensible form where true data are infeasible or impracticable to obtain. Modelling should be used as a last resource when RCTs or cohort studies do not provide all required information because they did not incorporate long enough follow-up [6–8]. Modelling has real strengths. It can be inexpensive, free of ethical concerns over treatment allocation and fast: a computer model can simulate in minutes a trial lasting years. Modelling has some less obvious benefits too, as the process of constructing the model promotes systematic thought and generated insights about the nature of its components and how they interact, which may help to identify areas in which empirical research is most needed, help generate new epidemiological or clinical hypotheses, and help produce novel ideas for useful interventions. Of course, modelling has its weak points. Failings in model theory or logic, inaccuracies in model parameters, or omission of key factors can all invalidate results .
Prediction models for long-term morbidity of infants could be used to extrapolate short-term outcomes or to indicate for which neonates long-term follow-up is required, as their outcomes (either absence or presence of sequelae) cannot be predicted from short-term outcomes and clinical background characteristics. The development of such models requires a longitudinal approach, in which data surrounding pregnancy, delivery and short-term outcomes are available, as well as follow-up data on various health related outcomes. The Dutch POPS cohort is one of the few birth cohorts in which this information has been systematically assessed. Data of infants born alive with a gestational age below 32 completed weeks and/or with a birth weight of 1500 gram were collected prospectively. A total of 1338 infants were included in this cohort, constituting 94% of the eligible infants born in 1983 in the Netherlands. Follow-up assessments were done at 2, 5 and 19 years of age [10–14]. This birth cohort could provide insight in the long-term consequences of perinatal outcomes and provides us with the necessary longitudinal data for developing prediction models for long-term health outcomes.
Objectives of the project
In this project we aim to assess the current state of affairs on long-term follow-up after obstetric randomized controlled trials (RCTs). A systematic review will be performed to assess how often follow-up of infants after discharge of the hospital is carried out. Furthermore, we will develop multivariable prediction models for different long-term health outcomes which can be used by future studies to extrapolate their short-term outcomes to a long-term horizon. We would like to encourage other researchers participating in follow-up studies after large obstetric trials (> 350 women) to inform us about their studies so that we can include their follow-up study in our systematic review. Furthermore, we would invite these researchers to join our effort and to collaborate with us on the external validation of our prediction models.
We will search all reviews of the Cochrane Pregnancy and Childbirth group. Pubmed searches did not provide us with enough relevant studies on long-term outcomes, because of a lack of good MeSH terms. Therefore, we decided that we will search for these studies in the Cochrane library. We assumed that all important obstetric studies published will be included in one of the reviews of the Cochrane Pregnancy and Childbirth group. All reviews on perinatal interventions that aimed to improve neonatal outcome will be included. Reviews on interventions primary looking at other aspects than neonatal outcome such as labor progress will also be included when these interventions can change the outcome of the neonate on the short or long-term. Cochrane reviews of perinatal interventions that have no potentional health benefit for the neonate will be excluded (for example the Cochrane review "antibiotics regimens for endometritis after delivery"). All studies included in the relevant Cochrane reviews will be considered and only studies reporting RCTs with more than 350 women are included for subsequent analysis. Every included RCT will be screened for statements on long-term follow-up. RCTs without statements on long-term follow-up will be cross-checked in Web of Science to see if long-term effects are reported in subsequent publications. We assume that articles reporting on long-term effects of a specific perinatal intervention will refer to the original article reporting this RCT. Two independent reviewers will extract relevant information from these selected RCTs using a data collection sheet. The following characteristics will be extracted from included RCTs: general study characteristics (country, sample size, type of intervention, primary and secondary outcomes), whether long-term follow-up has been performed, and whether this follow-up was planned before the start of the RCT. If follow-up has been performed, we will also document duration of follow-up, follow-up rate and methods and instruments used during this follow-up. All information will be entered and analysed using SPSS 17.0 (SPSS Inc., Chicago, IL, USA).
Multivariable prediction models
For the development of prediction models for several long-term health outcomes, we will use data available from a Dutch cohort study of preterm and/or small for gestational age infants (POPS study). In this cohort all live born infants were included, that were delivered in The Netherlands between January and December 1983, either before 32 completed weeks of gestation and/or with a birth weight of less than 1500 g. In total 1338 live born infants were included, 94% of all eligible infants. Follow-up assessments were done at 2, 5 and 19 years of age [10–14]. The follow-up rate after 2 years was 97%, after 5 years 96% and after 19 years 72%. This cohort provides valuable information on long-term consequences of perinatal outcomes.
For each prediction model candidate predictors will be selected based on existing literature, combined with consulting experts in the field and the availability of these variables in the POPS cohort. Examples of these candidate predictors are social class, ethnicity, pregnancy induced hypertension, pre-eclampsia, diabetes gravidarum with diet or insulin, maternal smoking during the pregnancy, prolonged rupture of membranes, multiple pregnancy and caesarean section, gestational age at birth, birth weight, low Apgar scores, umbilical cord academia, respiratory distress syndrome (RDS), bronchopulmonary dysplasia (BPD), necrotizing enterocolitis (NEC), intraventricular haemorrhage (IVH), periventricular leukomalacia (PVL) and retinopathy of prematurity (ROP).
Prediction models for the following long-term outcomes will be developed: respiratory and neurological morbidity, visual and hearing problems, intestinal morbidity and the costs associated with these medical conditions.
We will develop multiple multivariable logistic regression models in which the association between short-term and long-term outcomes is determined statistically, and adjusted for relevant covariates. A power calculation is not possible at this stage because we have not yet defined our long-term outcomes and the potential predictors, and are unfamiliar with plausible distribution and association assumptions for the specific long-term outcomes.
We will use a multiple imputation approach to deal with missing values. Uncertainty about imputed values is reflected in differences between different imputed datasets, and incorporated in the estimated standard errors and associated p-values for each fitted model based on the pooled datasets.
For each prediction model a different selection of candidate predictors will be used. Univariable and multivariable regression analysis will be performed to estimate odds ratios (ORs), 95% confidence intervals (95% CI) and corresponding p-values for candidate predictors. As the use of too stringent p-values for variable selection is more deleterious for a model than including too many factors, all candidate predictors that showed a significance level of < 50% in univariable analyses will be entered in the multivariable logistic regression model . Furthermore, we used a stepwise backward selection procedure, using a predefined significance level of > 20% for removing predictors from the models . Discriminative capacity of the models will be evaluated by estimating the area under the receiver operating characteristic (ROC) curve. Calibration of the models will be assessed by comparing estimated probabilities with the observed proportion of respiratory morbidity. Goodness-of-fit will be tested formally with the Hosmer and Lemeshow test statistic. Data will be analysed using SPSS 17.0 (SPSS Inc., Chicago, IL, USA).
Prediction models are known to be optimised for the specific dataset in which they have been derived. Internal and external validation is required to correct for this overfit bias, also known as optimism. Internal validation will be performed by uniform shrinkage or by bootstrapping of observations within the same dataset .
Furthermore, the models will be externally validated in other datasets available from ongoing or recently completed studies. For this validation, we will consider data from two Dutch studies (PETRA and TRUFFLE). In the PETRA-study, temporizing management with or without plasma volume expansion is compared for 216 women included between 1 April 2000 and 31 May 2003 . Their infants will undergo neurological examination after 1, 5 and 10 years follow-up . The TRUFFLE study currently investigates the best timing of delivery in preterm pregnancies complicated by intrauterine growth restriction, with neurological outcome measured at 2 years of age. In addition, we want to encourage researchers participating in other follow-up studies (longitudinal registries/birth cohorts) to contact us and to collaborate with our study to further explore the external validity and international generalizability of our prediction models for long-term outcomes. One important question to be addressed is whether these prediction models are also applicable to full-term neonates.
At the moment, it is not standard policy to do follow-up of the neonate after an obstetric RCT to evaluate long-term effects of a perinatal intervention. Main reasons for this limited follow-up are as follows; long-term follow-up is time-consuming, expensive and does not fall within the funding-period of most obstetric RCTs. It is also hypothesised that obstetricians may not be aware of important long-term consequences of their interventions. However, long-term follow-up is very important, as it can change the overall verdict regarding the optimal diagnostic or treatment strategy in a pregnant woman. The systematic review will provide insight in the extent and methods used for follow-up assessments after obstetric RCTs in the past. The different prediction models can be used by future studies to extrapolate their short-term outcomes to a long-term horizon or to indicate for which neonates long-term follow-up is required, as their outcomes (either absence or presence of sequelae) cannot be predicted from short term outcomes and clinical background characteristics.
Overall, the aim of this project is to increase international awareness of the importance of and to stimulate research on long-term follow-up after obstetric RCTs when evaluating perinatal interventions. We would like to encourage other researchers participating in large follow-up studies to join our effort and to collaborate with us on the external validation of our prediction models.
Margreet Teune is a medical doctor and studied Economics. Aleid van Wassenaer is a pediatrician. Ben Willem Mol is a gynecologist and clinical epidemiologist and is specialized in diagnostic and multicenter trial design. Brent Opmeer studied psychometrics and works as a clinical epidemiologist specialized in outcomes research.
This study is financially supported by ZonMW. Grant number 80-82325-98-9010.
- National institute of Child Health and development: Follow-up care of high-risk infants. Pediatrics. 2004, 114 (5): 1377-1397.Google Scholar
- Kenyon S, Boulvain M, Neilson J: Antibiotics for preterm rupture of the membranes: a systematic review. Obstet Gynecol. 2004, 104 (5 Pt 1): 1051-1057. 10.1097/01.AOG.0000143268.36682.21.View ArticlePubMedGoogle Scholar
- Kenyon S, Pike K, Jones DR, Jones DR, Brocklehurst P, Marlow N, Salt A, Tayler DJ: Childhood outcomes after prescription of antibiotics to pregnant women with preterm rupture of the membranes: 7-year follow-up of the ORACLE I trial. Lancet. 2008, 372 (9646): 1310-1318. 10.1016/S0140-6736(08)61202-7.View ArticlePubMedGoogle Scholar
- King J, Flenady V: Prophylactic antibiotics for inhibiting preterm labour with intact membranes. Cochrane Database Syst Rev. 2002, CD000246-4Google Scholar
- Kenyon S, Pike K, Jones DR, Jones DR, Brocklehurst P, Marlow N, Salt A, Tayler DJ: Childhood outcomes after prescription of antibiotics to pregnant women with spontaneous preterm labour: 7-year follow-up of the ORACLE II trial. Lancet. 2008, 372 (9646): 1319-1327. 10.1016/S0140-6736(08)61203-9.View ArticlePubMedGoogle Scholar
- Buxton MJ, Drummond MF, Van Hout BA, Prince RL, Sheldon TA, Szucs T, Vray M: Modelling in economic evaluation: an unavoidable fact of life. Health Econ. 1997, 6 (3): 217-227. 10.1002/(SICI)1099-1050(199705)6:3<217::AID-HEC267>3.0.CO;2-W.View ArticlePubMedGoogle Scholar
- Sheldon TA: Problems of using modelling in the economic evaluation of health care. Health Econ. 1996, 5 (1): 1-11. 10.1002/(SICI)1099-1050(199601)5:1<1::AID-HEC183>3.0.CO;2-K.View ArticlePubMedGoogle Scholar
- Steyerberg EW: Clinical prediction models. 2009, SpringerView ArticleGoogle Scholar
- Royston G: Commentary: trials versus models in appraising screening programmes. BMJ. 1999, 318 (7180): 360-361. Jacobusse GW, Ens-Dokkum MH, de Groot L, Wit JM, Geven WB, Kok JH, de Kleine MJ, Kollée LA, Mulder AL, van Straaten HL, de Vries LS, van Weissenbruch MM, Verloove-Vanhorick SPPubMedGoogle Scholar
- Hille ET, Weisglas-Kuperus N, van Goudoever JB, Jacobusse GW, Ens-Dokkum MH, de Groot L, Wit JM, Geven WB, Kok JH, de Kleine MJ, Kollée LA, Mulder AL, van Straaten HL, de Vries LS, van Weissenbruch MM, Verloove-Vanhorick SP: Functional outcomes and participation in young adulthood for very preterm and very low birth weight infants: the Dutch Project on Preterm and Small for Gestational Age Infants at 19 years of age. Pediatrics. 2007, 120 (3): e587-e595. 10.1542/peds.2006-2407.View ArticlePubMedGoogle Scholar
- Kok JH, den Ouden AL, Verloove-Vanhorick SP, Brand R: Outcome of very preterm small for gestational age infants: the first nine years of life. Br J Obstet Gynaecol. 1998, 105 (2): 162-168. 10.1111/j.1471-0528.1998.tb10046.x.View ArticlePubMedGoogle Scholar
- van Zeben-van der Aa TM, Verloove-Vanhorick SP, Brand R, Ruys JH: Morbidity of very low birthweight infants at corrected age of two years in a geographically defined population. Report from Project on Preterm and Small for gestational age infants in The Netherlands. Lancet. 1989, 1 (8632): 253-255. 10.1016/S0140-6736(89)91265-8.View ArticlePubMedGoogle Scholar
- Veen S, Ens-Dokkum MH, Schreuder AM, Verloove-Vanhorick SP, Brand R, Ruys JH: Impairments, disabilities, and handicaps of very preterm and very-low-birthweight infants at five years of age. The Collaborative Project on Preterm and Small for Gestational Age Infants (POPS) in The Netherlands. Lancet. 1991, 338 (8758): 33-36. 10.1016/0140-6736(91)90015-H.View ArticlePubMedGoogle Scholar
- Vrijlandt EJ, Gerritsen J, Boezen HM, Duiverman EJ: Gender differences in respiratory symptoms in 19-year-old adults born preterm. Respir Res. 2005, 6: 117-10.1186/1465-9921-6-117.View ArticlePubMedPubMed CentralGoogle Scholar
- Steyerberg EW, Eijkemans MJ, Harrell FE, Habbema JD: Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets. Stat Med. 2000, 19 (8): 1059-1079. 10.1002/(SICI)1097-0258(20000430)19:8<1059::AID-SIM412>3.0.CO;2-0.View ArticlePubMedGoogle Scholar
- Ganzevoort W, Rep A, Bonsel GJ, Fetter WP, van Sonderen L, De Vries JI, Wolf H: A randomised controlled trial comparing two temporising management strategies, one with and one without plasma volume expansion, for severe and early onset pre-eclampsia. BJOG. 2005, 112 (10): 1358-1368. 10.1111/j.1471-0528.2005.00687.x.View ArticlePubMedGoogle Scholar
- Rep A, Ganzevoort W, van Wassenaer AG, Bonsel GJ, Wolf H, de Vries JI: One-year infant outcome in women with early-onset hypertensive disorders of pregnancy. BJOG. 2008, 115 (2): 290-298.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2393/10/42/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.