The Raine study had no evidence of significant perinatal selection bias after two decades of follow up: a longitudinal pregnancy cohort study

The Western Australian Pregnancy Cohort (Raine) Study began as a randomised controlled trial evaluating the effects of repeated ultrasound in pregnancy. The Raine Study is now a longitudinal cohort study that has followed the offspring from 18 weeks’ gestation into young adulthood to investigate the early origins of adult disease. Detailed data collection at regular cohort reviews over two decades has seen the Raine Study evolve into a powerful tool for epidemiological and genetic association studies and myriad smaller research projects encompassing a wide range of clinical, para-clinical, and basic science fields.

The Raine Study participants were recruited from the sole tertiary obstetric referral hospital in Western Australia (WA), and surrounding private clinics; WA is the geographically largest and most isolated of the Australian states. Follow up assessments of the cohort over more than two decades has been accompanied by a reduction in participation, as would be expected with any long-term cohort study. It is possible that both the recruitment from a tertiary referral hospital and loss to long-term follow up could influence the extent to which the Raine cohort is representative of the wider Western Australian population; the former potentially increasing the proportion of participants with complicated pregnancies or socioeconomic disadvantage initially recruited and the latter potentially favouring retention of socioeconomically advantaged participants.

Adequate representativeness is clearly important for descriptive studies defining the prevalence of disease within populations, for example, the Global Burden of Disease Initiative [1]. Extrapolating disease prevalence from a non-representative population sample could greatly under- or over-estimate the true population prevalence [2]. Traditional epidemiological practice dictates that cohorts being used for investigation of associations between exposures and outcomes, that is, seeking potential causal relationships, should be reasonably representative of the populations from which they are drawn.

Representative cohorts are desirable as non-representative study groups may introduce bias into observational studies of causal associations, with multiple confounding variables in aetiologically complex conditions being unequally distributed between the population sample and the wider population. For example, previous observational studies demonstrated an apparent association between low antioxidant intake and pre-eclampsia [3–5], the biological plausibility of this link being supported by evidence of oxidative stress in the pathophysiology of established pre-eclampsia [6, 7]. However, the hypothesised reduction in risk with vitamin supplementation, despite early promise [8], was not borne out in subsequent randomised controlled trials among either low-risk [9–11] or high-risk [12–15] women. Meta-analyses of such trials have confirmed similarly poor efficacy [16–19]. It is likely, therefore, that low vitamin status is a surrogate marker for one or more other contributors to pre-eclampsia risk. Statistical adjustment for confounding factors, no matter how apparently thorough, often leaves unidentified residual confounding which, as in this example, is corrected by the randomisation within a controlled trial whereby the study design ensures representativeness between the trial arms [2].

Deliberate non-representative sampling, by contrast, is of scientific value in some situations [2]. For example, twin studies allow the investigation of environmental and epigenetic associations with disease by controlling for genetic variation. “Natural experiments”, such as the Chernobyl reactor disaster, provide insight into rare environmental exposures and subsequent health outcomes. Over-sampling of individuals within minority groups of particular interest allows later subgroup analyses which remain adequately powered without having to recruit a much larger cohort as a whole or to allow for selective attrition of these participants. Achieving representativeness may come at the expense of internal validity, however, a trade-off which has the potential to limit the robustness of inferences of causation, a major objective of observational research [20].

In the context of a longitudinal cohort study, selection bias introduced due to greater loss to follow up among certain subsets of the cohort may influence the assessment of exposure-outcome associations due to evolving differences in the prevalence of exposures, outcomes, or confounding factors between the populations. In particular, if the exposure of interest is associated with the probability of ongoing study participation then the exposure-outcome association may be biased [21] and, therefore, inferences of causation may be flawed. However, previous studies have, by assessing the relative magnitude of known associations in cohorts and their source populations, demonstrated that such bias is likely to be minimal [22, 23].

For large longitudinal studies, in which smaller sub-studies are conducted, the extent of representativeness may be both helpful and a hindrance, depending on the nature of the individual study performed. In such circumstances, therefore, it is particularly important to examine and report the representativeness of the cohort such that external validity may be reliably determined for each study undertaken. Moreover, an assessment of the selection bias resulting from loss to follow up or subgroup analyses should be made.

The aim of this study was to assess perinatal exposure-outcome associations between the Raine Study cohort at multiple time points over the last 20 years and the contemporaneously born Western Australian population.

The Raine Study participant numbers at recruitment and each follow up assessment are described in Fig. 1. The similarities and differences between the Raine Study Pregnancy Cohort at birth, the contemporaneous general WA population, and subsequent the Raine Study follow up and analysis subsets are presented in Table 1 and Additional file 1: Table S1.

	Western Australia		Raine Pregnancy Cohort at birth		Raine 5-year Subset		Raine 20-year Subset		Raine Fetal Growth Subset		Raine NAFLD Subset
n	99,141		2863		2010		1213		1377		879
Mothers	Reference
Age (years)		27.7	=	27.5	+	28.4	+	28.8	+	28.2	+	28.8
Married (%)		89.4%	−	82.1%	−	85.4%	−	86.8%	−	85.7%	−	86.7%
Caucasian (%)		87.4%	+	89.6%	+	90.8%	+	90.8%	+	99.1%a	+	99.2%a
Nulliparous (%)		39.0%	+	48.1%	+	46.7%	+	47.2%	+	47.6%	+	48.4%
Pregnancies
Complications (%)		30.0%	+	38.6%	+	37.9%	+	37.4%	+	38.7%	+	46.1%
Caesareans (%)		18.9%	+	21.1%	+	21.7%	+	21.4%	=	20.2%	=	20.4%
Infants
Birth weight (g)		3344	−	3283	−	3315	−	3316	+	3389	+	3384
Birth length (cm)		49.9	−	48.8	−	49.0	−	49.0	−	49.3	−	49.3
Ponderal index (kg/m3)		26.7	+	27.9	+	28.0	+	27.9	+	28.1	+	28.1
Gestation (weeks)		39.1	=	39.0	=	39.1	=	39.2	+	39.3	+	39.3
Nursery admissions (%)		7.6%	+	9.7%	+	9.2%	=	8.5%	=	7.1%	=	7.4%
Socioeconomic status
IRSD		1022b	=	1021	+	1035	+	1046	+	1035	+	1047

^aThe Raine Growth and NAFLD Subsets excluded non-Caucasian participants by design

NAFLD: Non-alcoholic fatty liver disease

IRSD: Index of Relative Socioeconomic Disadvantage. Higher value describes greater socioeconomic advantage

^bThe IRSD for Western Australia includes all Western Australians within the Perth metropolitan area and not just the source population of mainly young metropolitan mothers

Significance defined as p < 0.003

+/=/–: Significant increase/no significant difference/significant decrease compared to general Western Australian population

Compared to the WA population of contemporaneous births, the Raine Study mothers were less likely to be married (82% vs 89%, p < 0.0001), probably reflecting the relatively lower socioeconomic status of the population referred to a public tertiary antenatal clinic as, at the time of the Raine Study recruitment, ex-nuptial birth was strongly correlated with low socioeconomic status [28] and, indeed, IRSD was lower in unmarried women compared to married women in the Raine Pregnancy Cohort (IRSD 980 vs 1020, p < 0.0001). Whilst the Raine Pregnancy Cohort IRSD was not different to the whole WA population IRSD, the whole WA population IRSD included elderly and non-metropolitan groups which suggests IRSD for the source population for the Raine Pregnancy Cohort (mainly young metropolitan mothers) was likely to be higher and the Raine Study average may be elevated by the exclusion of non-participants in the 14-year follow up assessment who were more likely to be socially disadvantaged. Raine Study participants were more likely to be Caucasian (90% vs 87%, p < 0.0001), reflecting the English language competency inclusion criterion of the original study. The Raine Study pregnancies were more complicated than the WA population (39% vs 30%, p < 0.0001), at least in part due to a higher rate of nulliparous mothers (48% vs 39%, p < 0.0001) in whom pre-eclampsia and fetal growth restriction are more common.

Raine Study infants were, on average, 61 g lighter at birth than their WA contemporaries (3283 g vs 3344 g, p < 0.0001), again reflecting the higher rate of nulliparity and pregnancy complications. The Raine Study infants were substantially shorter at birth than the WA population (48.8 cm vs 49.9 cm, p < 0.0001). This difference in birth length accounts for the observed difference in ponderal index between the Raine Study and WA (27.9 kg/m³ vs 26.7 kg/m³, p < 0.0001). Neonatal nursery admissions were more common in the Raine Study than in WA (9.7% vs 7.6%, p < 0.0001), likely due to the higher rate of pregnancy complications and caesarean section in the Raine Study and the immediate availability of nursery facilities in the tertiary hospital setting of the Raine Study.

As the Raine Study has progressed over more than two decades, there have been fewer participants completing all components of cohort reviews: 1213 participants completed the 20-year assessment representing 42% of the original cohort. In general, those who continue to engage with cohort assessments are similar to the original Raine Pregnancy Cohort on perinatal characteristics. Participants who remained engaged were more likely to have older mothers who were married at recruitment than those not participating in childhood and adult reviews. Socioeconomic indices showed greater advantage among those retained to follow up. This reflects selective attrition of those from socioeconomically disadvantaged backgrounds. The pregnancy and infant data of those with continuing participation was not significantly different to the WA contemporaneous birth population (detailed data presented in Additional file 1: Table S1.

The Raine Growth Subset and Raine NAFLD Subset were compared to the Raine Pregnancy Cohort by assessing their perinatal characteristics. Similar to those completing the five and 20-year cohort reviews, these two subsets share the same loss of socioeconomically disadvantaged participants; both had a greater proportion of older mothers (28.2 years and 28.8 years at the Growth and NAFLD subsets, respectively, vs 27.5 years at birth) who were married at recruitment (86% and 87% at the five and 20-year follow up, respectively, vs 82% at birth) than the original cohort. Again, the pregnancy and infant data of those in the subsets were not significantly different to the cohort at recruitment.

Assessing the perinatal characteristics of the Raine Pregnancy Cohort demonstrates some statistically and potentially clinically significant demographic differences between the cohort and its source population of contemporaneously born Western Australians. Moreover, there have been some further demographic changes in the cohort as it has evolved over the more than two decades of the study through non-participation in follow up and subgroup analyses. These demographic differences have the potential to influence the capacity of the Raine Study to demonstrate exposure-outcome associations and to diminish the external validity of the findings of the Raine Study. It is important, however, to recognise that statistically significant demographic non-representativeness does not necessarily result in significant selection bias, and this should be assessed independently of demographic characteristics.

One method of assessing bias in cohort studies due to non-participation is to compare exposure-outcome associations within the study population and the source population. When this method was applied to the Raine pregnancy cohort, and the five- and 20-year follow up cohorts, the exposure-outcome association comparisons were all not significantly different to the exposure-outcome odds ratios in the WA general population of contemporaneous births. These data suggest that any demographic differences that exist between the WA population and the Raine study cohorts have not introduced significant bias into the Raine study, suggesting the Raine study findings can be generalised to the WA population.

In general, any selection bias due to non-participation in follow up assessments appears not to have any substantial effect on the representativeness of the evolving Raine Study for the outcomes assessed. It must be acknowledged that this study is unable to exclude an effect of selection bias for all possible outcomes, however, the consistency of null findings suggests that cohort non-participation does not introduce systematic bias into the findings of the Raine Study and the generalizability of findings to the source population.

Of the six exposure-outcome associations tested, only one suggested that significant bias may be present when comparing the original Raine Pregnancy Cohort to the genetic analysis subsets of Growth and NAFLD. This is not entirely unexpected, as these subsets were specifically designed to be ethnically homogeneous and this, by design, has introduced non-representativeness to these subsets. This deliberate homogeneity is required for genetic analyses, as ethnic genetic heterogeneity has the capacity to cloud the often subtle associations with single genetic variants: this is an example of a study design where population representativeness is neither necessary nor helpful. This potential source of bias must, however, be taken into account when extrapolating the findings of these particular subgroup analyses to the general population, and similar caution should be applied to other analyses from deliberately non-representative subgroups.

One of the strengths of the Raine Study, when compared to contemporaneous cohort studies, is its high cohort retention over the last 20 years despite the frequency of cohort reviews and the significant time commitment required of participants at each of these reviews. This is largely due to the substantial investment in cohort management, including participant involvement and the use of emerging social media technologies in cohort engagement. There has, however, been some selective attrition of participants from socioeconomically disadvantaged perinatal backgrounds who may face greater challenges to ongoing participation and are more difficult to engage [29]. How these perinatal socioeconomic changes reflect changes in subsequent assessments of socioeconomic status should be evaluated in further research.

Cohort studies are expensive to recruit and even more expensive to regularly review for decades; the Raine pregnancy cohort cost more than AUD$1 M to recruit and more than AUD$25 M has been spent on cohort reviews. The data in this study demonstrate that this money has been well spent because even though non-participation in cohort reviews has increased over time, those who continue to participate are a non-biased group representative of the WA population. Providing that attrition rates are limited, the value of this cohort will increase as they age due to the unique nature of the Raine pregnancy cohort with its dense data collected during pregnancy and the hundreds of thousands of phenotype measures available for analyses.

This work is potentially limited to its confinement to ssessment of selection bias within zexposures and outcomes within the perinatal period. Selection bias may exist in later outcomes which have not been assessed with this approach. Available data did not permit accurate assessment of health outcomes at later life stages due to the relative infrequency of morbidity severe enough to be captured by hospital admissions. Alternative approaches to assessment of selection bias, such as multiple imputation, are likely to be of benefit to the Raine Study and this will be a target of future research.

The Raine Study cohort had, at recruitment, slightly different perinatal demographic characteristics to the general contemporaneous births WA population. As the Raine Study cohort has evolved over time there has been a shift in demographic characteristics back towards the source population which may or may not have made the cohort more representative depending on the unknown influences of unmeasured potential factors. Neither the limited perinatal differences at recruitment nor at later follow up assessments appear to have resulted in significant selection bias. Subgroup analyses, however, may be somewhat less representative, limiting the external validity of the findings of the Raine Study to broader populations, especially to non-Caucasian populations.