Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Intraclass correlation coefficients in the Brazilian network for surveillance of severe maternal morbidity study

  • Samira M Haddad1,
  • Maria H Sousa2,
  • Jose G Cecatti1, 2Email author,
  • Mary A Parpinelli1,
  • Maria L Costa1,
  • Joao P Souza3 and
  • for the Brazilian Network for Surveillance of Severe Maternal Morbidity Group
BMC Pregnancy and Childbirth201212:101

DOI: 10.1186/1471-2393-12-101

Received: 20 June 2012

Accepted: 19 September 2012

Published: 21 September 2012

Abstract

Background

The purpose of the study was to evaluate intraclass correlation coefficients (ICC) of variables concerning personal characteristics, structure, outcome and process in the Brazilian Network for Surveillance of Severe Maternal Morbidity study conducted to identify severe maternal morbidity/near miss cases using the World Health Organization criteria.

Method

It was a cross-sectional, multicenter study involving 27 hospitals providing care for pregnant women in Brazil. Cluster size and the mean size of the primary sampling unit were described. Estimated prevalence rates, ICC, their respective 95% confidence intervals, the design effect and the mean cluster size were presented for each variable.

Results

Overall, 9,555 cases of severe maternal morbidity (woman admitted with potentially life-threatening conditions, near miss events or death) were included in the study. ICC ranged from < 0.001 to 0.508, with a median of 0.035. ICC was < 0.1 for approximately 75% of the variables. For process-related variables, median ICC was 0.09, with 0.021 for those related to outcome. These findings confirm data from previous studies. Homogeneity may be considered minor, thus increasing reliability of these findings.

Conclusions

These results may be used to design new cluster trials in maternal and perinatal health and to help calculate sample sizes.

Keywords

Intraclass correlation coefficient Maternal near miss Severe maternal morbidity Maternal and perinatal health

Background

Cluster studies are widely used in epidemiological research to evaluate health interventions and implement public policies. In these cases, selection units or randomization units consist of population groups (specific geographical areas) or healthcare units (hospitals) or healthcare sectors rather than individuals [1, 2].

In single-stage cluster sampling, all subjects belonging to each group are included to obtain data of interest. Unlike simple random sampling (SRS) in which each individual has an equal likelihood of being selected within the general population, data obtained from clusters may not be sufficiently representative to allow for generalization. This is due to a greater degree of homogeneity in characteristics of the population under observation, as opposed to heterogeneity found in the general population [2].

Reliability of estimates obtained from studies using cluster sampling may be analyzed by measuring inter- and intra-cluster variance. One way to perform these measurements is to calculate δ, the intraclass correlation coefficient (ICC), for variables evaluated in the study [3]. Another way to perform measurements is to calculate the design effect (DEFF).

ICC is a coefficient that measures the degree of homogeneity among elements within clusters. The ICC of a variable indicates to what extent variance of a parameter can be explained by variation between clusters [4, 5]. Its value depends on the type of variable, cluster size and prevalence of the condition [6]. Coefficients closest to zero suggest intraclass heterogeneity, indicating that the variable is randomly distributed among clusters. Similarly, values close to 1 indicate homogeneity in a sample and the variance of cluster units is greater than that of single elements [2].

Design effect (DEFF) indicates the extent to which the variance of parameter estimate is a result of study design, in this case the cluster sampling, compared to what would be obtained if sampling had been carried out by the SRS method [7]. For example, a DEFF value of 3 indicates that the variance of parameter estimation is three times greater than would be found if the study had been based on a random sample of equal size. Components involved in calculating the DEFF are ICC and mean cluster size. The larger the ICC and the larger the cluster, the higher the DEFF value [8].

The routine adoption of ICC calculation in cluster studies may improve interpretation of the results and facilitate the development of new studies in the field. These values could be used as a correction factor for the calculation of sample size in future cluster studies, thus avoiding underestimates, since, in studies in which SRS is used, the sample size required to achieve sufficient statistical power is generally smaller [4].

The Brazilian Network for Surveillance of Severe Maternal Morbidity study was developed with the objective of identifying cases of severe maternal morbidity/near miss (woman admitted with potentially life threatening conditions, near miss) or death in 27 hospitals distributed throughout Brazil. The participating centers were selected using the following criteria: the availability of each center to participate in the surveillance study, the geographical region of the country in which the hospital was situated and a requirement that the hospital in question performed at least 1,000 deliveries per year. Therefore, the participating centers constituted the primary sampling units, while the subjects at each center represented the units of analysis [9].

The objective of the present report is to evaluate the intraclass correlation coefficients of the variables associated with outcome, process, personal characteristics and structure, based on the data collected by the Brazilian Network for Surveillance of Severe Maternal Morbidity, as well as the design effects.

Methods

The Brazilian Network for Surveillance of Severe Maternal Morbidity, established in 2009, conducted a cross-sectional, multicenter study involving 27 hospitals providing care for pregnant women in Brazil. The objective of this network was to identify cases of severe maternal morbidity/near miss, using the criteria established by the World Health Organization (WHO) to characterize these conditions [10]. According to this definition, a maternal near miss is a woman who experienced a very serious complication during pregnancy and almost died but survived at least until the 42nd day of postpartum period [10].

The selection of clusters was based predominantly on the location of the hospitals in order to ensure broad coverage throughout the country, and on a minimum of 1,000 deliveries per center per year to enable the calculated sample size to be reached. The sample size calculation was based on an prevalence of 8 cases of near miss for every 1,000 deliveries [11], a maternal death ratio of 140 for every 100,000 liveborn infants, with confidence level of 95%, and considering a precision level of 8/1,000 for near miss and 8.5/1,000 for maternal death; it was added approximately 25% based on the fact that this definition of near miss had not yet been tested. In result, around 75,000 deliveries would have to be monitored to identify approximately 600 cases of near miss and 100 maternal deaths. The study was evaluated and approved locally by the respective Institutional Review Board of each participating hospital and also by the National Council for Ethics in Research.

Over the course of one year, all the pregnant women admitted to these institutions were monitored. Women found to have any of the morbidity criteria were included in the analysis. The data of interest to the study were collected from the women’s charts immediately after they were discharged from hospital [12]. After data collection was complete in June 2010, procedures were initiated to verify and correct any inconsistencies to ensure the quality of the data obtained. These procedures included daily check of all data inputted in the database, appointment of possible inconsistence to researchers, correction of errors and programmed check of all dataset at the end of study.

Data analysis

Initially, the cluster size and the mean size of the primary sampling unit (PSU) obtained from the total sample of 9,555 women were described. Estimated prevalences of each dichotomized variable, intraclass correlation coefficients (ICC), their respective 95% confidence intervals (95% CI), design effects (DEFF) and mean cluster size for each variable were calculated. The software programs used for the analysis were SPSS, version 17.0 [13] and Stata, version 7.0 [14], taking into consideration the cluster sampling plan (centers) for data analysis. Information on health facilities and live births for the whole country came from official data [15, 16].

Sampling plan used in the Brazilian network for surveillance of severe maternal morbidity study

A single-stage cluster sampling was used, with 27 primary sampling units (PSU) corresponding to the 27 participating centers (hospitals). The sampling plan did not involve stratification of the PSU or weighting of the data. The unit of analysis (subject) was the registration of each woman admitted with potentially life threatening conditions, near miss or death.

Prevalence – ratio estimator (r)[2]

r = y x = y α x α = α = 1 a β = 1 X α y αβ α = 1 a x α
(1)

in which y = α y α = α β y αβ (α for cluster and β for individual) is the total number of subjects in the sample possessing a certain category (for example “Yes”) for the variable y αβ (dichotomized); for example, y: the total number of subjects in the sample with “prenatal care at the same facility”, and x = α x α is the size of the available sample (valid) for that variable, where xα is the sample size for the cluster ‘α’.

For this study, the ratio estimator is:
r = y x = α = 1 27 y α α = 1 27 x α
(2)

Intraclass correlation coefficient – ICC (Roh)

According to Kish [2], the intraclass correlation coefficient is:

R o h = s a 2 s b 2 / b s ^ 2 , where s a 2 is the variance between clusters; s b 2 is the variance within clusters; b is the size of clusters and s ^ 2 is the estimate of S . 2 (the variance in individual level). The estimate s ^ 2 is obtained by: s ^ 2 = s a 2 + b 1 b s b 2

Stata’s equivalent computing formula [14] is:
I C C = F 1 a / n 1 + F 1 a / n
(3)

where ‘F’ is the Snedecor’s F-value from the ANOVA table and ‘a’ is the number of groups. The variance estimate for ICC is obtained by an extensive asymptotic formula, and because this it was not showed.

Design effect - DEFF [2]

D e f f = var actual r var SRS r = s a 2 / a s 2 / n
(4)

where varactual(r) is the estimated variance according to the complex design being studied and varSRS(r) is the variance in the estimator considering the design as if it were calculated using a SRS of the same size, n.

Variance estimator of r under the design being studied:
var actual r = 1 a 1 a = 1 a r a r 2 a
(5)
Variance estimator of r under SRS:
var SRS r = r 1 r n 1
(6)

where r is the ratio estimator under the SRS.

Recalculation of the sample size

Using the ICC from the main outcome as a correction factor [17]:
n * = a / ICC . Deff 1 + ICC
(7)

Where ‘a’ is the number of clusters, ‘Deff’ the design effect and ‘ICC’ the intraclass correlation coefficient of the main outcome.

Results

Of the 82,388 deliveries performed in the participating institutions during the study period, 82,144 liveborn infants and 9,555 cases of maternal morbidity were included in the study. The mean size of each cluster was 354 and the distribution of cases per center and region and of the liveborn infants per region is shown in Table 1.
Table 1

Distribution of cases and live births (LB) by cluster and according to geographical region of the country

Region

Center

nc(or xα#)

LB in the network

LB in Brazil*

Southeast n se = 2,794 (29.2%)

1

48

37,865 (46.1%)

1,130,407 (38.5%)

2

59

3

66

4

74

5

96

6

112

7

154

8

155

9

172

10

186

11

253

12

369

13

1050

Northeast n ne = 5,148 (53.9%)

1

118

33,172 (40.4%)

888,268 (30.3%)

2

210

3

263

4

281

5

294

6

465

7

566

8

920

9

945

10

1086

South n s = 939 (9.8%)

1

98

4,532 (5.5%)

371,497 (12.7%)

2

841

Midwest n mw = 609 (6.4%)

1

609

2,527 (3.1%)

222,658 (7.6%)

North n n = 65 (0.7%)

1

65

4,048 (4.9%)

321,998 (11.0%)

Total (n)

27

9555

82,144

2,934,828

Mean cluster size (na)= 354.

#Denominator of the expression of the ratio estimator.

*Data from SINASC (Brazil, 2011) [16].

The ICCs ranged from < 0.001 to 0.508, with a median of 0.035. The ICC was below 0.1 for approximately 75% of the variables. In this block of variables, the mediann ICC was 0.021, while in the other 25%, median ICC was 0.195. The ICCs for the variables related to the process are shown in Table 2, with values ranging from 0.001 to 0.508 (median 0.09), while DEFF values varied from 1.5 to 292.63 (median 20.52).
Table 2

Estimates of prevalence, intraclass correlation coefficients, their respective 95% CI, design effect, and mean cluster size for variables related to the process

Variable

P (%)

ICC

95% CI for ICC

DEFF

na

· Prenatal care at the same institution

22.1

0.103

0.033−0.174

44.72

316

· Access of women to the center (Spontaneous demand)

47.9

0.195

0.078−0.312

118.12

333

· Major type of healthcare insurance used for prenatal care (Public)

88.5

0.115

0.039−0.191

30.27

279

· Major type of healthcare insurance used for hospital admission (Public)

98.9

0.508

0.323−0.693

31.66

354

Procedure related to abortion

     

· Procedure performed: D&C

77.8

0.054

0.001−0.148

1.50

9

· Procedure performed: Oxytocin

33.3

0.339

0.136−0.542

9.39

9

· Procedure performed: Vacuum aspiration

7.0

0.001

<0.001−0.068

2.26

9

· Procedure performed: Prostaglandins

22.2

0.118

<0.001−0.245

1.91

9

· Procedure performed: Other

7.0

0.155

0.011−0.298

1.53

9

Related to PLTC criteria

     

· Any therapeutic management

76.1

0.272

0.125−0.419

225.19

354

· Transfusion of blood derivatives

16.4

0.079

0.024−0.133

37.50

354

· Central venous access

3.8

0.090

0.028−0.151

21.23

354

· Admission to ICU

22.1

0.288

0.136−0.439

199.58

354

· Prolonged hospital stay

30.0

0.098

0.032−0.164

81.88

354

· Intubation unrelated to anesthesia

3.1

0.047

0.013−0.081

12.99

354

· Return to operating room

3.3

0.021

0.005−0.038

11.75

354

· Hysterectomy/laparotomy

6.2

0.043

0.011−0.074

20.44

354

· Use of magnesium sulfate

48.3

0.363

0.192−0.534

292.63

354

· Another major surgical procedure

0.8

0.008

0.001−0.016

6.35

354

Related to near miss criteria

     

· Any management near miss criteria

6.1

0.075

0.023–0.127

20.52

354

· Continuous use of vasoactive drug

2.6

0.028

0.007–0.049

8.93

354

· Hysterectomy due to infection or hemorrhage

1.8

0.031

0.008–0.054

8.05

354

· Transfusion of ≥5 U red cells

2.6

0.053

0.015–0.091

11.42

354

· Intubation and ventilation ≥ 60 min unrelated to anesthesia

3.1

0.047

0.013–0.082

13.94

354

· Dialysis for acute renal failure

0.7

0.009

0.001−0.017

3.60

354

· Cardiopulmonary resuscitation (CPR)

1.3

0.018

0.004–0.033

8.41

354

· At least one of these conditions was already present when the woman was admitted

32.0

0.132

0.046–0.218

6.91

34

Related to delay in care

     

· Delay related to patients and/or their family members (Yes/No)

39.3

0.090

0.028−0151

27.26

310

· Search for a health service

5.3

0.174

0.067–0.281

54.03

310

· Geographic difficulty in accessing health service

2.4

0.352

0.183–0.520

113.16

310

· Refused treatment/care

5.1

0.014

0.002–0.025

7.72

310

· Prenatal care absent or inadequate

32.1

0.043

0.012–0.075

14.98

310

· Delay in care related with health professional

17.3

0.328

0.164−0.492

134.27

331

· Delay in diagnosis

5.5

0.218

0.091–0.346

72.81

331

· Delay in starting treatment

6.7

0.321

0.159–0.483

105.23

331

· Inadequate management

13.6

0.300

0.144–0.456

121.82

331

· Delay in referring or transferring the case

3.3

0.055

0.015–0.095

20.77

331

P: Prevalence of each category; ICC: Intraclass Correlation Coefficient; 95% CI: 95% confidence interval; DEFF: Design effect; na: Mean cluster size; PLTC: Potentially life-threatening condition; NM: Near miss.

Table 3 shows the variables related to outcome, with ICCs that ranged from < 0.001 to 0.375 (median 0.021) and DEFF values of 0.94 to 289 (median 6.24). ICC values were < 0.05 for approximately 74% of the variables associated with outcome, while the proportion of variables for which ICC was < 0.07 was almost 80%.
Table 3

Estimates of prevalence, intraclass correlation coefficients, their respective 95% CI, design effect and mean cluster size for variables related to outcome

Variable

P (%)

ICC

95% CI for ICC

DEFF

na

· Any hemorrhagic complication

24.8

0.165

0.062−0.268

129.54

354

· Abruptio placentae

5.7

0.016

0.003−0.029

9.55

354

· Placenta previa/acreta

2.2

0.017

0.004−0.031

6.24

354

· Complicated ectopic pregnancy

3.1

0.034

0.009−0.059

17.72

354

· Uterine rupture

0.3

0.005

<0.001−0.011

2.12

354

· Severe post abortion hemorrhage

1.3

0.015

0.003−0.028

5.92

354

· Postpartum hemorrhage

12.5

0.248

0.110−0.387

200.97

354

Atony

63.0

0.375

0.134−0.617

77.37

44

Retained placenta

11.8

0.114

0.005−0.224

12.94

44

Tears

8.8

0.130

0.008−0.251

10.04

44

Coagulopathy

4.6

0.102

0.002−0.203

13.51

44

Uterine inversion

0.6

0.035

<0.001−0.079

1.65

44

Other obstetric cause

11.2

0.370

0.130−0.610

43.77

44

· Other severe hemorrhage

1.0

0.023

0.005−0.041

6.16

354

· Any complication due to hypertension

70.2

0.183

0.072−0.294

124.72

354

· Severe preeclampsia

51.2

0.231

0.099−0.363

188.23

354

· Eclampsia

4.5

0.023

0.005−0.040

14.62

354

· Severe hypertension

18.3

0.334

0.169−0.499

289.00

354

· HELLP syndrome

6.2

0.028

0.007−0.049

15.43

354

· Acute fatty liver

0.2

0.005

0.001−0.010

1.59

354

· Any other complication

17.2

0.103

0.034−0.172

31.93

354

· Pulmonary edema

1.7

0.020

0.004−0.036

4.45

354

· Seizures

1.4

0.009

0.001−0.016

3.66

354

· Thrombocytopenia

3.9

0.022

0.005−0.039

6.77

354

· Thyrotoxic crises

0.1

0.007

0.001−0.013

1.55

354

· Shock

3.1

0.034

0.009−0.059

9.95

354

· Acute respiratory failure

4.0

0.051

0.014−0.087

19.37

354

· Acidosis

1.6

0.021

0.005−0.038

12.54

354

· Cardiopathy

1.7

0.013

0.002−0.023

5.71

354

· Stroke

0.2

0.001

0.001−0.003

1.10

354

· Coagulation defects

2.0

0.012

0.002−0.021

4.69

354

· Disseminated intravascular coagulation

0.6

0.012

0.002−0.021

2.60

354

· Thromboembolism

0.9

0.010

0.001−0.018

4.95

354

· Diabetic ketoacidosis

0.2

0.004

0.001−0.008

0.94

354

· Jaundice/liver dysfunction

1.4

0.014

0.003−0.026

2.46

354

· Meningitis

<0.1

<0.001

0.001−0.002

0.94

354

· Severe sepsis

3.5

0.064

0.019−0.109

18.09

354

Postpartum endometritis

19.5

0.131

0.011−0.252

3.95

13

Post abortion endometritis

11.0

0.043

0.001−0.115

2.13

13

Pulmonary focus

29.3

0.055

0.001−0.133

2.03

13

Urinary focus

25.0

0.013

0.001−0.067

1.08

13

Other

15.2

0.042

0.001−0.113

1.15

13

· Acute renal failure

2.2

0.027

0.006−0.047

8.68

354

· Complication possibly associated with Influenza A ‘H1N1’

2.2

0.056

0.016−0.097

20.62

354

· Any clinical maternal near miss criteria

5.5

0.037

0.010−0.065

15.44

354

· Cyanosis

1.3

0.023

0.005−0.040

12.63

354

· Gasping

0.4

0.009

0.001−0.017

4.21

354

· Respiratory rate <6 or RR>40

2.0

0.016

0.003−0.029

8.42

354

· Shock

2.6

0.031

0.008−0.054

9.24

354

· Oliguria unresponsive to fluids or diuretics

0.9

0.010

0.001−0.018

4.43

354

· Coagulation problems

1.0

0.012

0.002−0.021

4.88

354

· Loss of consciousness for 12h or more

0.7

0.005

<0.001−0.010

4.25

354

· Absence of consciousness and absence of pulse rate/heartbeat

0.7

0.006

0.001−0.011

4.96

354

· Stroke

0.3

0.002

0.001−0.005

1.16

354

· Uncontrolled seizures – total paralysis

0.2

0.001

<0.001−0.004

2.22

354

· Jaundice in presence of preeclampsia

0.3

0.001

<0.001−0.002

1.36

354

· Any laboratory near miss criteria

5.2

0.047

0.013−0.082

12.08

354

· O2 saturation < 90% for longer than 60 min

1.9

0.016

0.003−0.029

8.30

354

· PaO2/FiO2 < 200 (Yes/No)

1.1

0.015

0.003−0.028

8.37

354

· Creatinine ≥ 300 mmol/l or ≥ 3.5 mg/dl

1.0

0.007

0.001−0.013

3.03

354

· Bilirubin ≥ 100 mmol/l or ≥ 6 mg/dl

0.5

<0.001

<0.001−0.002

1.25

354

· pH < 7.1

0.8

0.009

0.001−0.016

5.32

354

· Lactate > 5

0.8

0.098

0.032−0.165

18.31

354

· Platelet < 50,000

2.1

0.025

0.006−0.044

5.32

354

· Absence of consciousness plus glucose and ketoacids in urine

0.2

0.004

<0.001−0.009

3.49

354

· Condition of woman at discharge (Medical discharge)

94.1

0.011

0.002–0.020

6.54

354

· Unsafe abortion

0.6

0.025

0.006–0.045

7.43

310

· Maternal Death

1.5

0.018

0.004−0.033

7.93

354

· Maternal Death + NM

9.5

0.077

0.023−0.130

21.09

354

P: Prevalence of each category; ICC: Intraclass Correlation Coefficient; 95% CI: 95% confidence interval; DEFF: Design effect; na: Mean cluster size; NM: Near miss.

Tables 4 and 5 show the ICC values with their respective 95% confidence intervals, prevalence rates, DEFF values and the mean size of the clusters for the variables referring to population/obstetric characteristics and structure, respectively. With the exception of some variables related to structure, particularly those associated with delays in the service or healthcare system, ICC values were very low.
Table 4

Estimates of prevalence, intraclass correlation coefficients, their respective 95% CI, design effect and the mean cluster size with respect to personal and obstetric data

Variable

P (%)

ICC

95% CI for ICC

DEFF

na

· Age (<30 years)

65.6

0.013

0.002−0.024

4.00

354

· Skin color (White)

42.5

0.285

0.127−0.443

146.55

264

· Schooling (Primary)

46.5

0.051

0.012−0.091

17.45

256

· Marital status (Has a partner)

53.2

0.176

0.066−0.286

101.96

298

· Weight (<75 kg)

52.3

0.055

0.010−0.101

21.45

167

· Height (<1.60 m)

48.6

0.054

0.007−0.101

13.51

151

· Body Mass Index (Low BMI)

15.4

0.053

0.006−0.100

18.47

146

· Number of pregnancies (One)

41.9

0.010

0.001−0.019

9.01

352

· Number of deliveries (None)

48.2

0.010

0.002−0.019

8.87

352

· Number of abortions (None)

77.6

0.005

<0.001−0.011

2.59

352

· Number of previous C-sections (None)

76.0

0.011

0.002−0.021

6.87

347

· Number of live births (None)

51.3

0.016

0.003−0.028

11.27

341

· Time since last delivery (≤2 years)

25.5

0.022

0.001−0.043

4.23

99

· Previous uterine surgery

2.0

0.067

0.017−0.117

20.52

299

· Number of prenatal visits (<6)

46.4

0.018

0.003−0.034

5.34

276

· Pregnant at admission

95.0

0.043

0.011−0.074

20.70

354

· Gestational age at admission (<37 weeks)

53.4

0.069

0.020−0.117

37.71

327

· Mode of onset of labor (no labor)

49.4

0.113

0.038−0.188

88.44

349

· Gestational age at resolution (<37 weeks)

45.7

0.055

0.015−0.095

31.28

309

· Mode of delivery (C-section before labor)

49.6

0.111

0.037−0.184

87.28

352

· Mode of onset of abortion (Spontaneous)

63.6

0.044

<0.001−0.141

1.52

8

· Total number of fetuses (None)

6.0

0.128

0.044−0.212

52.87

325

· Fetal presentation at birth (Cephalic)

91.3

0.011

0.001−0.020

4.77

273

· Some pathological or risky condition prior to this current pregnancy

48.9

0.115

0.038−0.191

47.12

305

· Chronic hypertension

17.7

0.046

0.012−0.079

15.98

305

· Obesity

24.1

0.110

0.036−0.184

65.40

305

· Low weight

0.3

0.004

<0.001−0.008

1.68

305

· Diabetes mellitus

2.5

0.023

0.005−0.041

7.84

305

· Smoking

5.7

0.065

0.018−0.111

44.05

305

· Cardiac diseases

2.9

0.025

0.006−0.045

9.23

305

· Respiratory diseases

2.8

0.018

0.004−0.033

9.67

305

· Renal diseases

1.2

0.038

0.010−0.067

11.74

305

· Sickle cell disease-thalassemia

0.8

0.005

<0.001−0.011

5.08

305

· HIV/AIDS

1.1

0.006

<0.001−0.013

3.16

305

· Thyroid disease

1.4

0.008

0.001−0.016

2.65

305

· Neurologic diseases/epilepsy

1.2

0.011

0.001−0.020

4.35

305

· Collagenosis

0.6

0.028

0.007−0.050

8.93

305

· Cancer

0.3

0.006

<0.001−0.012

2.31

305

· Other

5.2

0.037

0.009−0.065

13.99

305

· Drug addiction

1.2

0.005

<0.001−0.010

4.07

305

P: prevalence of each category; ICC: Intraclass Correlation Coefficient; 95% CI: 95% confidence interval; DEFF: Design effect; na: Mean cluster size.

Table 5

Estimates of prevalence, intraclass correlation coefficients, their respective 95% CI, design effect and mean cluster size with respect to the variables related to structure

Variable

P (%)

ICC

95% CI for ICC

DEFF

na

· Delay related to service or healthcare system

15.6

0.247

0.108−0.387

106.64

328

· Lack of medication

1.3

0.021

0.005–0.038

9.77

328

· Difficulties with municipal/hospital transport

1.3

0.031

0.007–0.055

9.27

328

· Difficulties with communication (hospital/central regulation)

8.8

0.206

0.083–0.329

99.07

328

· Lack of blood derivatives

0.6

0.014

0.002–0.025

8.17

328

· Difficulty related to monitoring (intensive care unit)

4.6

0.230

0.097–0.363

64.99

328

· Lack of trained personnel

3.1

0.063

0.018–0.107

24.25

328

· Difficulty related to access to prenatal care

1.4

0.084

0.025–0.142

20.09

328

P: Prevalence of each category; ICC: Intraclass Correlation Coefficient; 95% CI: 95% confidence interval; DEFF: Design effect; na: Mean cluster size.

Based on the fact that the sample size calculation did not take into account the ICC for the primary outcome of interest, because it was not available from other studies at the time of project development, the sample size was recalculated using the ICC for the variable near miss/deaths as the correction factor.

This was an exercise to evaluate if our sample would have been sufficient for analysis considering the cluster design. This variable was used because it is the main outcome of interest in this study and its value was 0,077. Using the formula for recalculating the sample size in such situations [17], results indicate that a sample size of 7,072 would be needed to identify the near miss/death cases and it corresponds to 74% of the total number of subjects included in the Network.

Discussion

The values of the intraclass correlation coefficients found in this study can be considered low, close to zero, for the majority of the variables. Intraclass heterogeneity was greater in the variables related to outcome in comparison with the others.

In selecting the clusters, stratification by region was not performed. Proportionally more of the centers were situated in the southeast of the country (48%), and consequently a greater number of liveborn infants were also born in this region (46%). This distribution is in conformity with the actual distribution of healthcare institutions and the proportionality of liveborn infants per region of the country. According to the Ministry of Health’s National Register of Healthcare Institutions, approximately 45% of the institutions registered are situated in the southeastern region [15] and, in 2008, 38.5% of all liveborn infants were born in this region [16]. Proportionality was also maintained in comparison with the other regions.

Although the surveillance of cases of severe maternal morbidity/near miss was prospective, the data were collected from the patients’ charts immediately following the women’s discharge from hospital. Therefore, for some variables, the number of individuals for whom data is available is less than the total number of cases, since it was impossible to recover some of the missing data. This possible loss of part of the data for some of the variables was predicted and taken into consideration in planning the study. Severely ill women may not be able to consent to participate in studies of this type because they may be unconscious, may die or may find themselves in various different situations of emotional fragility. Therefore, data collection following hospital discharge was completely anonymous and avoided the need to obtain informed consent, what allowed a larger number of research subjects to be included, factors that are important in studies of serious conditions with a relatively low prevalence rate.

Previous studies have shown that ICC values are generally higher for variables related to process compared to those for variables related to outcome, since for the same intervention (measure of process), responses may differ between the different individuals under therapeutic management (outcome measurement). Furthermore, higher healthcare levels tend to increase the degree of homogeneity [1720]. The explanation for this is that when the level of care is higher, there is a greater likelihood that management techniques will be standardized and institutional protocols will be used. In these cases, when one of the objectives is to evaluate compliance with established guidelines, homogeneity may be interpreted as a leveling of institutions with respect to certain recommendations.

This tendency is also seen in the present results, which were obtained from secondary and tertiary care hospitals, the majority of which were teaching hospitals. In this type of institution, the majority of procedures are performed in conformity with evidence-based healthcare protocols. Indeed, the mean ICC value for the variables related to process was 2.6 times higher than the mean ICC value for the variables related to outcome.

The variable with the highest ICC was “Major type of healthcare insurance used for hospital admission” with a value of 0.508. This homogeneity was expected, since only 3 of the 27 centers accepted private patients, all the other centers being exclusively public healthcare services.

Some of the variables related to process were obtained from a specific section of the study focusing on delays during patient care. In addition to obtaining data from the women’s charts, this study also involved another step in which the investigators were instructed to make a subjective analysis of the chain of healthcare services provided, based on the data available on the charts. In addition, after all the variables had been completed, the data for each subject were reanalyzed by the principal investigators and standard procedures for the classification of delays were implemented for all the cases in all the clusters. Therefore, the greater homogeneity found for these variables may be explained by this “standard correction” adopted for all the centers.

Previous studies in the primary care sector have reported ICC values < 0.05 for variables related to outcome and ICC values > 0.05 for variables related to the process [20]. In the field of maternal and perinatal healthcare, Taljaard et al. calculated ICC values based on data obtained from secondary/tertiary services [18]. The ICC of some variables analyzed in this study can be compared with ours as reported in Table 6.
Table 6

Comparisom of Intraclass Correlation Coefficients of some variables between this study and study by Taljaard  et al.[18]

Variable

Haddadet al.

Taljaardet al.

 

ICC

na

ICC

na

· HIV positive

0.006

305

0.022

865

· Chronic respiratory conditions

0.018

305

0.042

861

· Diabetes mellitus

0.023

305

0.010

862

· Hysterectomy

0.031

354

0.002

862

· Eclampsia

0.023

354

0.007

862

· Maternal Death

0.018

354

0.0003

866

ICC: Intraclass Correlation Coefficient; na: Mean cluster size.

The findings of those investigators showed that, in general, ICC values for the variables related to the process tended to be > 0.07, with values < 0.07 for the variables related to outcome. The present findings are in agreement with this observation. Furthermore, these values can probably be considered as a good parameter of variance for calculating sample size in new studies in this area, similar to the 0.08 value used by van de Ven et al.[21].

Pagel et al.[22] estimated ICC of data from five community-based cluster-randomized controlled trials, all evaluating community interventions to improve maternal and newborn health outcomes. The mean cluster size of these studies ranged from 3,934 to 27,953 people. Of 9 key perinatal indicators, only maternal death has the possibility of comparison with our calculations and its ICC ranged from 0.00 to 0.00051.

All those comparisons show what was already known. The smaller the cluster size, the higher the ICC, and the opposite occurs regarding the prevalence of the condition. Mortality events are rare in community based population and the prevalence rises with morbidity association. Pagel et al.[22] established as a limitation of their findings that the ICC estimates for rare outcomes as maternal mortality are not likely to be reliable.

Taljaard et al.[18] collected data from hospital based population, as our study, and obtained ICC similar to our findings.

This study is based on pregnant women who had at least one of the potentially life threatening conditions. So, prevalence of any factor that is associated with these conditions will be higher in the study sample than in the general obstetric population. This could potentially inflate ICC estimation as ICC generally tends to increase with higher prevalence. The ICCs of this study also depend on other factors such as the hospitals’ characteristics. Caution in applying these ICCs to other settings should be used and this could be considered as a possible limitation of the study.

The absence of previous studies using these new near miss criteria standardized by the WHO [10] made it impossible to obtain the respective ICC values for the variables of interest related to outcome. Therefore, in this study sample size was calculated based exclusively on previous prevalence rates of the condition.

Considering the mean general characteristics of women in this study, most of whom received care in public hospitals linked to universities, and that the ICC values showed sample heterogeneity, studies conducted in middle income countries with similar characteristics to Brazil probably can use the ICC values of this study as the basis for their calculations.

Conclusions

The Brazilian Network for Surveillance of Severe Maternal Morbidity conducted a pioneering, cross sectional, multicenter study on the application of the new WHO near miss criteria to identify severe cases in obstetrics. This paper reports the intraclass correlation coefficients for study variables. The results found are in agreement with those of previous studies and homogeneity of the data obtained from variables related to outcome may be considered minor (median ICC 0.021), increasing reliability of the study estimates. These values may be used to plan new cluster studies in maternal and perinatal health, mainly studies associated with severe maternal morbidity/near miss events. They may be useful for sample size calculation.

Brazilian Network for Surveillance of Severe Maternal Morbidity Group

Fernanda G. Surita, João L. Pinto e Silva, Rodolfo C. Pacagnella, Rodrigo S. Camargo, Vilma Zotareli, Carla Silveira, Lúcio T. Gurgel, Lale Say, Robert C. Pattinson, Marilza V Rudge, Iracema M Calderon, Maria V Bahamondes, Danielly S Santana, Simone P Gonçalves, Eliana M Amaral, Olímpio B Moraes Filho, Simone A Carvalho, Francisco E Feitosa, George N Chaves, Ione R Brum, Gloria C Saint’ynes, Carlos A Menezes, Patricia N Santos, Everardo M Guanabara, Elson J Almeida Jr, Joaquim L Moreira, Maria R Sousa, Frederico A Peret, Liv B Paula, Luiza E Schmaltz, Cleire Pessoni, Leila Katz, Adriana Bione, Antonio C Barbosa Lima, Edilberto A Rocha Filho, Melania M Amorim, Debora Leite, Ivelyne Radaci, Marilia G Martins, Frederico Barroso, Fernando C Oliveira Jr, Denis J Nascimento, Cláudio S Paiva, Moises D Lima, Djacyr M Freire, Roger D Rohloff, Simone M Rodrigues, Sergio M Costa, Lucia C Pfitscher, Adriana G Luz, Daniela Guimaraes, Gustavo Lobato, Marcos Nakamura-Pereira, Eduardo Cordioli, Alessandra Peterossi, Cynthia D Perez, Jose C Peraçoli, Roberto A Costa, Nelson L Maia Filho, Jacinta P Matias, Silvana M Quintana, Elaine C Moises, Fátima A Lotufo, Luiz E Carvalho, Elvira A Zanette, Carla B Andreucci, Márcia M Aquino, Maria H Ohnuma, Rosiane Mattar and Felipe F Campanharo.

Abbreviations

ANOVA: 

Analysis of variance

DEFF: 

Design effect

ICC: 

Intraclass correlation coefficient

PSU: 

Primary sampling unit

SRS: 

Simple random sampling

WHO: 

World Health Organization.

Declarations

Acknowledgements

The authors acknowledge the financial support of CNPq/DECIT (The National Research Council and the Department of Science and Technology of the Brazilian Ministry of Health), [grant number 402702/2008-5] and also the involvement of the members of its Steering Committee and all the other investigators and coordinators from the centers involved in the study. We also thank the financial support of Fapesp (Grant number 2012/18414-3) for publishing this manuscript.

Authors’ Affiliations

(1)
Department of Obstetrics & Gynecology, University of Campinas
(2)
Campinas Center for Studies in Reproductive Health (CEMICAMP)
(3)
Department of Reproductive Health and Research, World Health Organization, UNDP/UNFPA/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction

References

  1. Ukoumunne OC, Gulliford MC, Chinn S, Sterne JA, Burney PG: Methods for evaluating area-wide and organisation-based interventions in health and health care: a systematic review. Health Technol Assess. 1999, 3 (5)): iii-92-PubMedGoogle Scholar
  2. Kish L: Survey sampling. 1965, John Wiley & Sons, New YorkGoogle Scholar
  3. Thomson A, Hayes R, Cousens S: Measures of between-cluster variability in cluster randomized trials with binary outcomes. Stat Med. 2009, 28: 1739-1751. 10.1002/sim.3582.View ArticlePubMedGoogle Scholar
  4. Campbell MK, Grimshaw JM, Elbourne DR: Intracluster correlation coefficients in cluster randomized trials: empirical insights into how should they be reported. BMC Med Res Methodol. 2009, 4: 9-View ArticleGoogle Scholar
  5. McGraw KO, Wong SP: Forming inferences about some intraclass correlation coefficients. Psychol Methods. 1996, 1: 30-46.View ArticleGoogle Scholar
  6. Adams G, Gulliford MC, Ukoumunne OC, Eldridge S, Chinn S, Campbell MJ: Patterns of intra-cluster correlation from primary care research to inform study design and analysis. J Clin Epidemiol. 2004, 57: 785-794. 10.1016/j.jclinepi.2003.12.013.View ArticlePubMedGoogle Scholar
  7. US Census Bureau: Technical Paper 63RV: Current Population Survey - Design and Methodology. TP63RV. 2002, 14-18. Available at http://www.census.gov/prod/2002pubs/tp63rv.pdf. Accessed on June 22, 2011,Google Scholar
  8. Shackman G: Sample size and design effect. Presented at the Albany Chapter of the American Statistical Association. 2001, Available at http://www.albany.edu/~areilly/albany_asa/confweb01/abstract/Download/shackman.pdf. Accessed on June 22, 2011,Google Scholar
  9. Cecatti JG, Souza JP, Parpinelli MA, Haddad SM, Camargo RS, Pacagnella RC, Silveira C, Zanardi DT, Costa ML, Silva JL Pe, Passini R, Surita FG, Sousa MH, Calderon IM, Say L, Pattinson RC, Brazilian Network for Surveillance of Severe Maternal Morbidity: Brazilian network for the surveillance of maternal potentially life threatening morbidity and maternal near-miss and a multidimensional evaluation of their long term consequences. Reprod Health. 2009, 6: 15-10.1186/1742-4755-6-15.View ArticlePubMedPubMed CentralGoogle Scholar
  10. Say L, Souza JP, Pattinson RC, WHO working group on Maternal Mortality and Morbidity classifications: Maternal near miss – towards a standard tool for monitoring quality of maternal health care. Best Pract Res Clin Obstet Gynaecol. 2009, 23: 287-296. 10.1016/j.bpobgyn.2009.01.007.View ArticlePubMedGoogle Scholar
  11. Souza JP, Cecatti JG, Parpinelli MA, Serruya SJ, Amaral E: Appropriate criteria for identification of near-miss maternal morbidity in tertiary care facilities: a cross sectional study. BMC Pregnancy Childbirth. 2007, 7: 20-10.1186/1471-2393-7-20.View ArticlePubMedPubMed CentralGoogle Scholar
  12. Haddad SM, Cecatti JG, Parpinelli MA, Souza JP, Costa ML, Sousa MH, Surita FG, Pinto E, Silva JL, Pacagnella RC, Camargo RS, Bahamondes MV, Zotareli V, Gurgel LT, Say L, Pattinson RC, National Network for the Surveillance of Severe Maternal Morbidity Group: From planning to practice: building the national network for the surveillance of severe maternal morbidity. BMC Publ Health. 2011, 11: 283-10.1186/1471-2458-11-283.View ArticleGoogle Scholar
  13. SPSS for windows: Release 17.0. 2008, SPSS Inc, ChicagoGoogle Scholar
  14. StataCorp: Stata Statistical Software: Release 7.0. College Station. 2001, Stata Corporation, TXGoogle Scholar
  15. Brazilian Ministry of Health: National Registry of Health Facilities. 2011, Available at http://cnes.datasus.gov.br/Lista_Tot_Es_Estado.asp. Accessed on July 4, 2011,Google Scholar
  16. Brazilian Ministry of Health: SINASC – National Information System on Live Births. 2011, Available at http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sinasc/cnv/nvuf.def. Accessed on July 4, 2011,Google Scholar
  17. Piaggio G, Carroli G, Villar J, Pinol A, Bakketeig L, Lumbiganon P, Bergsjø P, Al-Mazrou Y, Ba'aqeel H, Belizán JM, Farnot U, Berendes H: WHO Antenatal Care Trial Research Group. Methodological considerations on the design and analysis of an equivalence stratified cluster randomization trial. Stat Med. 2001, 20: 401-416. 10.1002/1097-0258(20010215)20:3<401::AID-SIM801>3.0.CO;2-1.View ArticlePubMedGoogle Scholar
  18. Taljaard M, Donner A, Villar J, Wojdyla D, Velazco A, Bataglia V, Bataglia V, Faundes A, Langer A, Narváez A, Valladares E, Carroli G, Zavaleta N, Shah A, Campodónico L, Romero M, Reynoso S, de Pádua KS, Giordano D, Kublickas M, Acosta A, World Health Organization 2005 Global Survey on Maternal and Perinatal Health Research Group: Intracluster correlation coefficients from the WHO Global Survey on Maternal and Perinatal Health: implications for implementation research. Paediat Perinatal Epidemiol. 2005, 2008 (22): 117-125.Google Scholar
  19. Campbell SM, Braspenning J, Hutchinson A, Marshall M: Research methods used in developing and applying quality indicators in primary care. Qual Safe Health Care. 2002, 11: 358-364. 10.1136/qhc.11.4.358.View ArticleGoogle Scholar
  20. Campbell MK, Fayers PM, Grimshaw JM: Determinants of the intracluster correlation coefficient in cluster randomized trials: the case of implementation research. Clin Trials. 2005, 2: 99-107. 10.1191/1740774505cn071oa.View ArticlePubMedGoogle Scholar
  21. van de Ven J, Houterman S, Steinweg RA, Scherpbier AJ, Wijers W, Mol BW, Oei SG, TOSTI-Trial Group: Reducing errors in health care: cost-effectiveness of multidisciplinary team training in obstetric emergencies (TOSTI study); a randomised controlled trial. BMC Pregnancy Childbirth. 2010, 10: 59-PubMedPubMed CentralGoogle Scholar
  22. Pagel C, Prost A, Lewycka S, Das S, Colbourn T, Mahapatra R, Azad K, Costello A, Osrin D: Intracluster correlation coefficients and coefficients of variation for perinatal outcomes from five cluster-randomised controlled trials in low and middle-income countries: results and methodological implications. Trials. 2011, 12: 151-10.1186/1745-6215-12-151.View ArticlePubMedPubMed CentralGoogle Scholar
  23. Pre-publication history

    1. The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2393/12/101/prepub

Copyright

© Haddad et al.; licensee BioMed Central Ltd. 2012

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.