Estimating coverage of a women’s group intervention among a population of pregnant women in rural Bangladesh

Progress towards Millennium Development Goals four and five of reducing child and maternal morality requires focused attention on better access, coverage and utilisation of good quality maternal and reproductive health services and interventions [1–3]. Understanding coverage of population health interventions and services is important in monitoring their implementation and receipt and assessing whether they reach the intended target population(s). Understanding coverage also enables the assessment of whether certain target groups are being reached more effectively than others. Coverage levels and trends are, therefore, helpful to assess the performance of health systems, services and interventions. They guide policy makers in planning strategies to improve health by identifying gaps in health systems and resources that require focused attention [4–6].

Coverage indicators are quantitative in nature and typically express the proportion of a target population exposed to or receiving a health intervention. In their most simple form, coverage indicators are calculated by dividing the number of individuals receiving an intervention by the total population eligible for that intervention [7]. For example, coverage indicators of maternity care include the proportion of pregnant women who received at least two antenatal care visits, the proportion of deliveries occurring in a health facility, the proportion of deliveries with skilled attendance at birth, and the proportion of women attended at least once during the postpartum period by skilled health personnel [7]. Clear definitions and measurements of the numerator and the denominator for these proportions are therefore crucial.

Methods to estimate intervention coverage in resource- and data-poor settings are rarely described or documented and measuring coverage among pregnant women can be particularly challenging. In settings with weak or non-existent vital registration systems and where records of population numbers, uptake of services or exposure to interventions are scarce, estimating intervention coverage demands creative use of primary and secondary data sources. This paper describes four different methods, and their underlying assumptions, to estimate population coverage of a women’s group community mobilisation intervention for maternal and newborn health in rural Bangladesh. The advantages and disadvantages of each method are discussed.

The women’s group community mobilisation intervention is delivered in three districts in Bangladesh and is described in detail elsewhere [8]. Essentially, the intervention is comprised of monthly women’s group meetings whereby communities prioritise problems that threaten the health of mothers and their babies and plan, implement and evaluate strategies to overcome these problems. Local women, who were trained in participatory methods and provided with basic information about maternal and newborn health, facilitate a participatory learning and action cycle.

The target population for the intervention is pregnant women and so all ever-married women of reproductive age are invited to participate. Every woman’s group meeting, however, is open to participation by anybody in the community, including men. Open participation is crucial to the community’s acceptance of the women’s group intervention and the dissemination of key health facts. Specific strategies were employed to attract women of reproductive age, pregnant or newly married women to participate in the groups. These strategies included actively approaching pregnant and newly married women in the community, providing them with information about the nature and objectives of the women's group intervention and encouraging them to participate.

A prospective surveillance system is in place to record and monitor birth outcomes in intervention areas. Traditional Birth Attendants (TBAs) act as key informants to notify a project monitoring and evaluation (M&E) team of all births and deaths to women of reproductive age in the intervention areas. Monitors verify births and deaths, and all women who gave birth are approached to be interviewed. Structured interviews gather detailed data on birth outcomes and care before, during and after delivery and are conducted between 42 days and one year after delivery using a survey. Detailed quantitative and qualitative process evaluation information on intervention processes, context and participation, particularly of pregnant women, is also collected throughout the intervention period. The third source of primary data gathered by the intervention implementation team is an annual household census, which records the total number of women of reproductive age and children living in the intervention areas. Publicly available secondary population data sources in Bangladesh include the 2001 Bangladesh Maternal Health Services and Maternal Mortality Survey (BMMS) [9], the 2007 Bangladesh Demographic and Health Survey (BDHS) [10] and the Bangladesh Bureau of Statistics (BBS) national census data from 2001 [11].

Four methods have been developed to estimate intervention coverage among pregnant women, each using at least one of the primary and/or secondary data sources mentioned above. Confidence intervals for all estimates are calculated using the confidence interval calculator in STATA version 12. Coverage estimates that are based on secondary data and assumptions about underlying fertility levels are presented with ranges that reflect extreme values and uncertainty in these assumptions. Estimates derived from each method are compared and critically discussed in terms of these inherent assumptions, methodological strengths and weaknesses and differing data and resource demands. All estimates are for the period of October 2009 to May 2010.

A total of 45,820 eligible women (i.e. aged 15–49 years) reside in intervention areas and were indentified using the study’s household census. A total of 4046 women gave birth in the intervention areas between October 2009 and May 2010. Of the deliveries identified, a total of 3972 mothers were interviewed using the study’s M&E survey, a response rate of 98%. On average, 847 pregnant women were reported to have attended women’s group meetings each month from October 2009 to May 2010.

Applying four different methods to estimate the coverage of a women’s group community mobilisation intervention among pregnant women in rural Bangladesh, coverage estimates ranged from 30% to 34%, depending on method used. Each of the methods presented has varying data and resource demands and is associated with specific strengths and limitations, which must be considered when selecting the most appropriate methods for end-user needs.

Method 1 depends on women accurately reporting participation in the women’s group intervention, and this may be subject to reporting bias during data capture. The method also makes the assumption that all pregnancies end in a delivery that is noticed and recorded by TBAs. This is unlikely to be true as early terminations through miscarriage or abortion are likely to be missed. The implication of this is an underestimation of the total number of pregnant women in the population and thus an overestimation of programme coverage. The survey methods used to gather data for Method 1 are also particularly resource demanding and require a reasonably high degree of competency from a monitoring and evaluation team for effective and efficient data processing. Such resources and competencies are unlikely to be available in many resource-poor settings. Nevertheless, in settings where resources and competency are available for direct measurement and provided that limitations of survey methods (such as recall and reporting bias) are recognised and minimised, direct measurement of proxy measures may be more informative and reliable than entirely modelled estimates.

Method 2 is largely based on the assumption that fertility remains constant in the intervention areas and study period and therefore the number of deliveries will equal the number of conceptions. Notwithstanding seasonal variations and longer-term fertility declines, this assumption may be reasonable given that the total fertility rate has not changed substantially on a year to year basis. The total fertility rate remained fairly stable at around 3.3 children per woman for most of the 1990s but declined to 2.7 according to the 2007 BDHS report [10].

As previously described, the numerator in Method 2 may be an underestimate of the true number of pregnant women participating in the women’s group intervention because some women who are in the early stages of pregnancy may not be aware of their pregnancy or may not want to disclose their pregnancy status. However, this is dealt with to a certain extent by adjusting the denominator. Furthermore, the fact that the denominator is likely to exclude pregnancies that end early due to miscarriages or abortions is also likely to minimise the problems of non-disclosure in the numerator since early termination of pregnancies and non-disclosure are both more likely to occur in the first trimester of pregnancy. Nevertheless, the upper and lower estimates generated by Method 2 indicate sensitivity to the assumed time before disclosure of pregnancy status and so formative work to inform this may be needed. Overall, Method 2 is appealing in that it is based on a prospective births surveillance system, thus enabling real-time, prospective monitoring of intervention coverage. However, such data processes and key informant systems are not trivial in terms of financial resources and technical capacity.

Methods 3 and 4 are progressively less resource- and data-demanding, using secondary data sources to estimate the number of pregnant women in the intervention areas. Although the numerator does require a degree of primary data capture, namely a register of women participating in the intervention and their pregnancy status, the technical demands for such data capture and processing are minimal and it is reasonable to expect that any intervention or service will keep a record of uptake. Both Methods 3 and 4 use secondary, cross-sectional data to derive the coverage estimate denominator. The cross-sectional nature of these data may be considered a limitation of these methods in that derived coverage estimates provide only a snapshot view and any changes in population size or fertility since the denominator data were captured cannot be taken into consideration. This is likely to affect the precision of any estimates. Furthermore, the application of national figures at a single point in time to relatively small geographical areas may mask regional variability and temporal fluctuations in population and fertility.

Unlike Method 2, it is not so simple to allow for non-disclosure of early pregnancy in coverage estimates in Methods 3 and 4 and thus the estimates derived from these methods may mis-represent programme coverage. Nonetheless, compromises of precision for low data and resource demands are likely to be acceptable to health programme and intervention managers wishing to approximate the coverage of their intervention without the need for more complex and expensive monitoring systems and both Methods 3 and 4 are likely to satisfy this need.

All coverage estimates presented here are for the same calendar period of the women’s group intervention in Bangladesh and, overall, estimates were remarkably close irrespective of method used. Methods 2, 3 and 4 showed only 4% difference in coverage estimates. The close similarity in coverage estimates between these three methods may be explained by the fairly consistent fertility rates reported in national figures in recent years. As a proxy measure, Method 1 estimates coverage among deliveries rather than pregnant women. This method may be subject to recall and reporting biases in recently-delivered women’s self-reports of participation in women’s yet the similarity of this estimate to the estimates derived from the other methods is notable. None of the estimates presented say anything about the equity of intervention coverage or biases in participation in the intervention, although Method 1 has the potential to do so as the survey method enables the collection of background and socio-economic data.

In the absence of complete and timely population data, choice of coverage estimation method must be based on the strengths and limitations of available methods, capacity and resources for measurement and the ultimate end user needs. Each of the methods presented and discussed here is likely to provide a useful understanding of intervention coverage at a single point in time and Methods 1 and 2 may also provide more reliable estimates of coverage trends. No single method is likely to satisfy all user needs and capabilities and the use of multiple data sources and methodologies are likely to facilitate better understanding of coverage estimates and may stimulate critical review of processes and data where estimates diverge.