Funded by the Mary Crosse Fund at Birmingham Women's Hospital a systematic review project based on this protocol will be conducted.
In 1973 Dr Crosse bequeathed the legacy of her estate to the former South Birmingham Hospital Management Committee for the development of research in Maternity, Neonatal and Special Care Baby Unit.
Objectives
To determine the association and clinical impact of neonatal findings and tests (including birth weight, Apgar scores and umbilical cord pH) with morbidity and mortality in infancy, childhood and adulthood, using systematic reviews and meta-analyses.
Search Strategy
Literature will be identified using:
-
General bibliographic databases including MEDLINE (PubMED) and EMBASE (OVID)
-
Specialist electronic databases: the Cochrane Library (DARE, CCTR), MEDION
-
Contact with individual experts and those with an interest in this field to uncover grey literature
-
Hand- searching of selected specialist journals
-
Checking of reference lists of relevant review articles and papers that will be eligible for inclusion
Searches will we performed to identify the neonatal tests in question and combined with a search to identify morbidity and mortality. The comprehensive search strategy will aim to find all primary studies reporting the association of each neonatal test with any measure of childhood or adult morbidity and mortality. The search strategy for umbilical cord pH may be viewed as an additional file 1 (other searches are available for authors on request). Search terms related to the test (e.g. Umbilical cord, Hydrogen-ion concentration, Asphyxia neonatorum, umbilical artery pH, cord pH) are combined using 'and' with MESH headings (e.g. Human development, Infant mortality) and keywords (e.g. developmental delay, handicap) to encompass neonatal mortality and short and long term morbidity. The search will be restricted to human studies only. No language restrictions will be applied. All databases will be searched from inception and updated at 6 monthly intervals. A comprehensive database of the literature will be constructed (Reference Manager 11.0) to allow us to handle citations efficiently [21].
Inclusion Criteria
Studies will be selected for inclusion in the reviews using the selection criteria based on population, index test, reference standard and study design of interest.
Population
Neonates in any health care setting
Tests
neonatal tests will be prioritised on the basis of clinical relevance after consultation with experts in the field (figure 1).
Outcome measure
Any measure of infant, childhood or adult morbidity or mortality, (figure 1).
Study design
Observational studies (cohorts, case-control) allowing generation of 2 × 2 tables of the association between neonatal test and outcome measure. Case series ≤ 5 will be excluded due to the likely association with bias and imprecision.
Study selection process
Studies will be selected for inclusion in the review in a two stage process using the selection criteria detailed above. Firstly, the titles and abstracts of the citations in the Reference Manager database will be assessed by one reviewer. All papers felt to be relevant will be obtained in full text version. Two independent reviewers will then select the studies which meet predefined criteria, defined prior to commencement and individualised for each review. Disagreements will be resolved by consensus or input from a third reviewer.
Data Extraction
A data extraction form will be designed for each review; variations between reviews will mainly be on the information extracted regarding the index test. Data will be extracted on: identification of study (first author, year of publication, country of investigation, language of paper); population (health care setting, number of participating centres, level of risk assigned by author and clinical data on risk factors, inclusion period); study design (design, data collection, enrolment, completeness of follow up); index test (gestation, method of performing test, intra and inter-observer variation, cut off level); reference standard (incidence, reference standard used, cut off level, total number of individuals analysed for results); results (necessary data for construction of 2 × 2 table, all results will be collected for reported index tests at any cut-off level, any measure of statistical accuracy reported).
The data extraction will be conducted in duplicate using the pre-designed form. Disagreements between reviewers will again be resolved by consensus or arbitration. Where multiple publications are identified, only the most recent and/or complete study will be included. Data will be entered onto an Excel spreadsheet.
Study quality assessment
Study and reporting quality will be assessed by at least one reviewer for all included manuscripts. Methodologic quality is a construct defined as the confidence that the study design, conduct and analysis minimises bias[22] in the estimation of the association between test and outcome, thereby maintaining internal validity (i.e. the degree to which the results of this observation are correct for the patients being studied). Another construct is that it is a set of parameters in the design and conduct of a study that reflects the validity of the outcome, related to the external and internal validity and the statistical model used [23]. For our review these parameters will be developed adapting the QUADAS tool [24]. Elements of study design which may have a direct relationship to bias and variation in a test accuracy study will be assessed with elements of the STARD checklist [25]. We have used such tools in our previous work [26].
In the assessment of study quality, prospective recruitment of patients with a consecutive or random recruitment pattern will be considered ideal. Sufficient clinical information should be given to assign a level of risk of complications, which ideally should be stated by the authors. The quality of performance and reporting of the index test will be assessed to look at elements of the test that may introduce bias. Information regarding the reference standard including method of determination, execution and blinding will be extracted. Ideal study design will be cohort studies; case control study design has been shown to affect accuracy and where numbers of studies permit these will be excluded from meta-analysis [27]. Verification bias will be assessed using a flow diagram to assess the number of eligible individuals completing both index test and outcome measure, and those excluded from the analysis with reasons. With ideal verification studies will account for all eligible individuals, state how indeterminate results were handled, and > 90% of those undergoing the index test should progress to complete the outcome measure. Where possible an individual quality assessment will be tailored to each review, using the most important items from validated tools. The assessment of quality will be represented by a stacked bar chart.
We will use the GRADE approach to determine whether we could recommend the use of each test in a clinical context. This approach is transparent in its considerations [28]. This considers the quality of the evidence not only according to the test accuracy, but the impact of the test on patient-important outcomes and takes into account factors influencing the quality of the evidence such as the study design, potential sources of bias and the precision of the results [29].
Data description
For each test, information on individual studies will be summarised as follows:
-
Table with methodological and reporting characteristics of included studies
The table will state the number of women in each study, the incidence of each adverse outcome (based on the number of analysed cases divided by the total number of individuals at baseline).
-
Summary of quality and reporting items of the included studies
Results will be presented as 100% stacked bars, where the bars represent a quality item and the figures in the stacks represent the number of studies
-
Forest plots of odds ratios and 95% CIs
Odds ratios, analysed as (true positive/false positive)/(false negative/true negative) will be presented.
-
Table with subgroup analyses (if applicable)
-
Grade tables
For each test the tables will state the number of studies, design, limitations, test results with outcomes important to patients, the indirectness of the impact of the test result on patient-important outcomes, the precision of the data, publication bias and an assessment of the overall quality of the evidence.
Statistical Analyses
From the 2 × 2 tables, odds ratios will be calculated for each study along with their 95% confidence intervals (CIs) [30]. When 2 × 2 tables contain zero cells, 0.5 will be added to each cell to enable calculations [31]. In each review, results will be visualised using Forest plots and ROC plots; extreme values, outliers and threshold phenomena will be explored.
Results will be analysed in groups according to the index test performed and the outcome measure studied, these will be defined a priori for each review. Meta-analysis will be used when appropriate. Pooled summary estimates will be produced in the form of odds ratios, as these are often relatively constant regardless of the diagnostic threshold and are frequently used to demonstrate a causal association in epidemiological studies [32]. The range of uncertainty will be calculated using the 95% confidence intervals of the odds ratios for each test. A fixed or random effects model will be used as appropriate depending on the degree of heterogeneity present.
Heterogeneity of results between studies will be assessed graphically by inspection of forest plots and ROC plots. The X2 and inconsistency squared will be used as statistical measures of heterogeneity. Where heterogeneity is not present (X2 >0.10, p < 0.05 and I2 < 50%) the fixed effect pooling method will be used and where relevant we will consider the use of the bivariate meta-regression model [22, 33]. Where heterogeneity is present, this will be explored using meta-regression analyses. Factors considered to be important beforehand will be used for the analysis, including:
-
Variations in population, high and low risk depending antenatal or intrapartum factors
-
Study quality
-
Study design: Prospective vs. Retrospective data collection
-
Variations in the type of index test and outcome measure and the thresholds used
Analysis for the assessing the risk of publication bias will be carried out by producing funnel plots of accuracy estimates against corresponding variances [28]. When no publication bias is suspected the plots will be symmetrical and funnel shaped because smaller studies are expected to have increased variation in estimates of accuracy.
When interpreting the data we will consider the criteria proposed by Hill to establish causality [34]. The consistency of the results, the biological plausibility of the findings and the specificity and temporality of the associations demonstrated will be examined.
Data syntheses will be performed using meta-disc version 1.4, STATA version 10.0 and StatsDirect version 2.7.2.