This article has Open Peer Review reports available.
An option for measuring maternal mortality in developing countries: a survey using community informants
- Siti Nurul Qomariyah1Email author,
- David Braunholtz†2,
- Endang L Achadi†1,
- Karen H Witten†2,
- Eko Setyo Pambudi†1,
- Trisari Anggondowati†1,
- Kamaluddin Latief†1 and
- Wendy J Graham†2
© Qomariyah et al; licensee BioMed Central Ltd. 2010
Received: 20 February 2010
Accepted: 17 November 2010
Published: 17 November 2010
The maternal mortality ratio (MMR) remains high in most developing countries. Local, recent estimates of MMR are needed to motivate policymakers and evaluate interventions. But, estimating MMR, in the absence of vital registration systems, is difficult. This paper describes an efficient approach using village informant networks to capture maternal death cases (Maternal Deaths from Informants/Maternal Death Follow on Review or MADE-IN/MADE-FOR) developed to address this gap, and examines its validity and efficiency.
MADE-IN used two village informant networks - heads of neighbourhood units (RTs) and health volunteers (Kaders). Informants were invited to attend separate network meetings - through the village head (for the RT) and through health centre for the kaders. Attached to the letter was a form with written instructions requesting informants list deaths of women of reproductive age (WRA) in the village during the previous two years. At a 'listing meeting' the informants' understanding on the form was checked, informants could correct their forms, and then collectively agreed a consolidated list. MADE-FOR consisted of visits relatives of likely pregnancy related deaths (PRDs) identified from MADE-IN, to confirm the PRD status and gather information about the cause of death. Capture-recapture (CRC) analysis enabled estimation of coverage rates of the two networks, and of total PRDs.
The RT network identified a higher proportion of PRDs than the kaders (estimated 0.85 vs. 0.71), but the latter was easier and cheaper to access. Assigned PRD status amongst identified WRA deaths was more accurate for the kader network, and seemingly for more recent deaths, and for deaths from rural areas. Assuming information on live births from an existing source to calculate the MMR, MADE-IN/MADE-FOR cost only $0.1 (US) per women-year risk of exposure, substantially cheaper than alternatives.
This study shows that reliable local, recent estimates of MMR can be obtained relatively cheaply using two independent informant networks to identify cases. Neither network captured all PRDs, but capture-recapture analysis allowed self-calibration. However, it requires careful avoidance of false-positives, and matching of cases identified by both networks, which was achieved by the home visit.
The maternal mortality ratio (MMR) remains high in most developing countries . Local, recent estimates of MMR are needed to motivate policymakers to prioritise maternal health, and to evaluate interventions. But estimating MMR, in the absence of vital registration systems, is difficult. Large surveys are required for even moderately precise estimates of MMRs . Consequently, such surveys are not undertaken frequently, only provide national level figures, and use long 'reference' periods of 5 or more years; an example for this is the Demographic Health Survey (DHS) used by the Indonesian Government to estimate national MMR [3, 4]. What policy makers and program managers need is a relatively cheap and reliable method to provide sub-national-level estimates of recent maternal death rates. This paper describes an approach developed to address this gap.
In Serang and Pandeglang districts, Banten Province, as in all of Indonesia, the administrative system includes community level volunteers (RTs) responsible typically for 10-40 house-holds. Distinct from this are health volunteers (Kaders) responsible for the integrated health posts with coverage of approximately 100 households. Since these systems are comprehensive, functional, and extend down to a level at which it would be reasonable to expect good knowledge of deaths and maybe circumstances of death, there is an obvious possibility of using them as a basis for 'capturing' maternal deaths.
An informant-based method for identifying maternal deaths - Maternal Deaths from Informants/Maternal Death Follow on Review (MADE-IN/MADE-FOR) - was developed and tested by Immpact (Initiative for Maternal Mortality Programme Assessment) in the two districts. It formed a key part of a larger study looking at the effects of the Indonesia village midwife programme . This method was developed as an alternative to allow measurement of maternal mortality down to community level, along with analysis on the cause of death. This paper is written by some of the researchers who involved in the development, as well as pilot and implementation of the method.
MADE-IN uses RT Heads and Kaders as informants who provide information about deaths to women in their communities. Once probable maternal deaths are identified by these informants, the MADE-FOR survey follows up with visits to the families to confirm details and circumstances of the death. This paper describes the method, and examines its validity and efficiency.
The study was conducted in all 708 villages in Serang and Pandeglang Districts. In Serang, 38 sub-district health centres (HCs) provide services for almost 1.9 million people, similarly 34 HCs in Pandeglang, with about 1.1 million populations . Banten Province ANC coverage is about 86% but skilled attendance at birth is only about 52% .
Maternal death definition
In this study, as for instance in DHS studies, the MMR is estimated using information on the number of Pregnancy Related Deaths (PRD) . A PRD is defined as the death of a woman while pregnant or within 42 days of termination of pregnancy, irrespective of the cause of death. This category of case is introduced to facilitate identification of maternal death in circumstances in which attribution of cause might be inadequate. From verbal autopsies, we found that the proportion of women classified as PRD who died from maternal causes in the study area was very high (i.e. 97.5%) meaning that choice of PRD or maternal death definition is not of major importance.
The pilot and data collection process ran from January to June 2006.
The pilot phases were aimed at examining the advantages and disadvantages of various possible informant networks, and of different modes of accessing the informants' knowledge. A further feasibility study was conducted to assess the logistics and costs of a full-scale survey, using the most efficient administrative and data capture approaches identified from the pilot.
Kaders were chosen as forming the best network for identifying maternal deaths: they are established in the community and registered both in HCs and sub-district offices, making them easier to contact; their attendance rate was high (on average 90% compared to 52% for RTs), they were able to understand and fill-out the forms with a low error rate (5% compared to 9% for RTs); and are typically one third as many kader as RTs in a village, which reduced costs of data capture. However, because population density is much greater in urban areas and knowledge of neighbours less, it was felt many PRDs might be missed by kaders, and so it was decided to also include the RT network in urban areas. To enable capture-recapture estimates, RTs were also used in one randomly-chosen village per sub-district in rural areas.
THE 'MADE-IN' PART OF DATA CAPTURE
The key activity was the 'listing-meeting' of village informants, to list details of deaths of women of reproductive age (WRA) in the village. Note that in villages where both kader and RT networks were used, separate listing meetings were held for each network, allowing for independency between the networks. Prior to the meetings, information, instructions, and forms were distributed to the village informants, for them to list information about WRA deaths during the previous two years. The most important items requested on the form were timing of death in relation to any pregnancy, age of woman, date of death, residence of woman, and name and address of a relative. Informants were asked to bring the completed forms to the listing meeting, where the study and forms were explained verbally, questions answered, and an opportunity given for informants to correct their forms in the light of improved understanding. The meeting then discussed all deaths listed by any of the informants, and collectively agreed a 'consolidated' list of WRA deaths, and of likely PRDs. To strive for total coverage of deaths, the MADE-IN process also included a visit to village informants who did not attend the meeting.
The 'MADE-FOR' part of data capture
Sources for information on number of live births
Many sources can be used to provide estimates of live births by district. In Indonesia they include Demographic and Health Surveys, National Socioeconomic Surveys or censuses, and District Basic Health Research surveys. However, to enable useful comparisons between PRDs and all women with recent pregnancies, a stratified cluster random sample survey was conducted by Immpact, not only to estimate live births but also to describe characteristics and health status, among other variables. With some help with the sampling, such a survey is well within the capability of a team undertaking MADE-IN MADE-FOR. Villages were stratified as urban, rural or rural-remote. A total of 125 villages were sampled with probability proportional to size and with replacement. Within each sampled village, all household members were listed to identify all women with a live birth or stillbirth between April 2004 and March 2006. Eight births per village were then randomly selected, and information was collected from the families on the same background variables as for maternal deaths.
Ethical consideration was one of several issues ensured in this study. It includes permission, confidentiality, and research benefit for the community. Before the study conducted, ethical approval was sought and then issued by the Ethical Committee of Faculty of Public Health University of Indonesia. Permission was also obtained from related district health offices. At community level, permissions were obtained from all sub-districts and villages in the two districts. Before an interview was conducted, written consent was asked from each respondent after data collectors explained the objective, benefit, and process of the interview. All individual data was treated in a strictest confidentiality and no information on name was disclosed, even to the village or sub-district authority. Following the study, researcher provided feed back to various parties from various levels to make sure that public will achieve benefit from the study conducted. Feed back has been continuously given through written dissemination (publication) or seminar, at local, national, and international level.
Limitation of the study
The main limitation of the method deals with recall of the event. This method relies on informants' memories and knowledge to record deaths, thus the accuracy of the method depends on the length of recall period and size of the area covers by informants. This recall issue also related with the memory of the family in describing the circumstances of the death. Further detail about limitations of the method is available at http://www.immpact-international.org/toolkit/module4/mimf/index.html.
Estimating total maternal deaths and risk
In villages where both kader and RT networks were used, for each confirmed eligible PRD we know whether the kader, or RT, or both networks identified the case. This allowed us to use 'capture-recapture' techniques  to estimate both the total PRDs in these villages, and the coverage of each network (i.e. the proportion of the total identified by each network). A simple formula which can be used to estimate the total cases is T = (N1*N2)/M. N is number of cases captured by method 1 or method 2 (in this case by RTs or by kaders). M is the total cases captured by both methods. The estimated coverage of the kader network, and actual number of PRDs the kader identified, were then used to estimate the total deaths in the villages where only kaders were used.
It should be noted that although we used inter-VAm to interpret cause of death using information from the MADE-FOR, to calculate the MMR we used PRD definition for the numerator. Reasons for this include the experimental nature of InterVA-M, the very low proportion of non-maternal cases, and also ensure full comparability with the national level estimate derived from the Demographic and Health Survey
The name 'capture re-capture' derives from wildlife applications, where a sample of animals from a target population is captured, marked and released. A second sample is captured at some later time. The number of animals captured each time, and both times, is noted . In public health applications, it is usually individuals who are 'captured' on different databases, and a key stage is matching i.e. identifying the individuals who appear on more than one database.
The set of 'individuals or 'events' to be estimated is fixed. This is often problematic when estimating total population in animal studies (since they reproduce, migrate, and die - so the individuals, and indeed the total number, changes between capture & recapture), but not in this study since there clearly is a fixed set of PRDs in the defined period and area which are captured, and recaptured by the two networks.
Individuals captured by both databases can be matched. This is often a major problem for public health applications, but here this was largely solved via the follow-up visits, where it quickly becomes clear if two apparently different deaths reported by the networks are in fact the same woman.
Capture in the second sample is independent of capture in the first. In this study, this is not entirely the case, since despite attempts to keep the network listing processes entirely separate, there were inevitably occasions for 'contamination'. For example, it was not uncommon for RTs to be married to a kader. However, by holding separate meetings, and limiting time available for 'crosschecking, it is believed contamination was very limited. In fact, checks were made on completed forms (looking for similar order, spellings etc) and no evidence of information-sharing came to light.
Within each occasion or database, the probability of capture does not differ between individuals. This assumption is problematic, since there are certain types of deaths which tend not to be reported by either network, for example deaths in early pregnancy.
The direction of bias due to assumptions 3 and 4 not being completely met in this study is, in both cases, downwards - so the resulting estimate will tend to be lower than the true number of deaths.
To estimate live-births (to allow an estimate of MMR), fertility-rates from the survey mentioned above, undertaken in the same districts, were applied to the census population estimates. Data processing was undertaken using Epi-info , with estimation undertaken in WinBUGS  to allow description of uncertainty around estimates.
Analysis of cost
The number of PRDs identified per "survey effort" (data-collector-weeks) is one measure used to compare MADE-IN/MADE-FOR with other methods for identifying maternal deaths. Another measure used is "cost per woman-year of exposure." Exposure is calculated from the number of WRA in the area, multiplied by the number of years of exposure. Costs were divided between MADE-IN or MADE-FOR, either as they fall clearly to one (such as informants' transportation fee, which is a MADE-IN expenditure), or, for more general expenditure (wages, accommodation for the survey team), based on the number of data-collector-days spent on MADE-IN and MADE-FOR, which was generally similar. Information on the cost of population survey, of which the estimate of live births is used, is illustrated as unit cost for capturing a live birth.
Probability of each informant network capturing deaths
The probability of the Kader and RT networks in capturing deaths
In area with kaders and RTs meeting
Number of cases captured by RT
Number of cases captured by Kaders
Number of match cases (captured by both)
Total death cases captured by the two networks
Simple estimate of total number of PRDs
86 × 103/73
Simple estimate of probability of kaders capturing cases
Simple estimate of probability of RTs capturing cases
Number of cases missed by the networks
In area with kaders only meetings
Number of cases captured by Kaders
Estimate number of PRDs
In the whole area
Mean estimated number of PRDs (with 95% CI) - allowing for uncertainty in the simple estimates above.
(560 to 711)
Overall, we estimated that the RT network was more likely to capture PRDs than the kader network. This does not, however, mean that individual RTs are necessarily 'better' than individual kaders in identifying PRDs, since within the same area the number of RTs involved is much higher than the number of kaders.
The accuracy of reported eligible PRD status of identified WRA deaths
The accuracy of assigned PR status of identified WRA deaths (MADE-IN process), by informant, year of death, and district*
Efficiency of the method
Total cost of MADE-IN and MADE-FOR
MADE-IN and MADE-FOR costs, Serang and Pandeglang Districts 2004-2006
Salary for field workers
Supplies and service
# of WRA
Women-year risk of exposure
cost per women-year risk of exposure ($)
The total cost for the population survey was $ 130,435. This provided an estimate of the number of live births, but also information on locations and other potential 'risk factor' variables for pregnancy and delivery outcome (e.g. age, parity, socio-economic status, prenatal care status, utilization of health provider for pre-natal and intra-partum care). This information was used together with the information on PRDs, to estimate relative risks. We believe that in most settings where MADE-IN/MADE-FOR could be used, information on live births will be available from other existing sources, so that a population survey is an optional, but extremely valuable, extra not strictly required for estimating MMR.
Added value from the MADE-FOR step
Comparison of MMR estimates in the two districts and total cost of data collection
Cost per death ($)
MADE-IN with CRC
690 - 780
MADE-IN/MADE-FOR with CRC
377 - 499
1. IDHS (2002/2003)
2. IDHS (2007/2008)
3. WHO (2005)
While MADE-FOR increased costs by about 50%, without this step bias due to unrecognised false positives, and missed matches, is so large as to make the estimates of PRDs and MMR, almost worthless. The size of this bias is unlikely to be stable across time or region, as it depends on the operational characteristics of the informant networks, so any 'adjustment factor' to convert a MADE-IN only estimate to an unbiased one, will be subject to great uncertainty. As well as verifying the status of PRDs, MADE-FOR allows collection of information about the circumstances of death, and characteristics of the deceased. This enables assessment of risk factors, and causes of death, which are invaluable for service planning.
Currently, Indonesia relies primarily on a national MMR estimate generated from the IDHS in 2008. The design and size of the IDHS sample meant only 62 PRDs, over a 5 year 'window', were found, and hence a very imprecise estimates even at national level. Given the diversity of Indonesia in terms of health services and health profile, this national average is likely to conceal more than it reveals, and because it is averaged over the last five years it will not capture recent changes. The lack of precise, local, and current estimates of maternal mortality in Indonesia is a major problem when trying to assess local needs, or the impact of programmes to reduce maternal risks - which was one motivation for developing MADE-IN/MADE-FOR. But it is also a major problem when it comes to assessing the validity of any measurement method, including MADE-IN/MADE-FOR. We have assessed the reliability, feasibility and efficiency of MADE-IN/MADE-FOR using internal consistency checks, and assurance of high quality data capture processes.
The key characteristic of MADE-IN/MADE-FOR is the use of existing networks of village informants to report vital events retrospectively. In combination with existing information on population and fertility, this allows a precise, local estimate of MMR to be obtained quickly from a 'one-off' survey. A companion 'population survey' allows investigation of risk factors, but increases costs. There is clearly the potential to develop the process into a sustainable ongoing system, which might result in higher quality data since events would be more recent - for instance verbal autopsy interviews would always be conducted within a reasonably short period of the deaths. Many countries in the region have also used villagers in health programs including in recording and reporting systems. A study in Cambodia, for example, involved Village Health Volunteers in a community-based surveillance system , and found that it could successfully fill the gaps of the current health facility-based disease surveillance system. A similar method has also been used in India .
The use of two informant networks in the MADE-IN/MADE-FOR is an important innovation, as it allows capture re-capture estimation. This makes the survey 'self-calibrating' in that it can estimate its own coverage, in a situation where it is clear that no single method captures all deaths. A study in Cambodia which tried to compare two surveys methods (a community based survey and a household survey) for estimating maternal and perinatal mortality found detection failures in both surveys, as high as 30-40% . A study in the USA found only 62% of maternal deaths were identified through death records .
The huge advantage of MADE-IN/MADE-FOR over household surveys like DHS, is that costs are reduced by a factor of 10-100. For the DHS, the level of investment required to ensure a high-quality survey, such as long periods of training, extensive pilot testing, separate household listing teams prior to the survey, and maintaining data quality tables during fieldwork, along with considerable technical assistance needed, can result in costs easily exceeding US$150 per household interview , implying $12 or more per woman-year of exposure (calculated from mean number of WRA in a household from the Indonesia DHS which is 2.4 and a 5 year 'window'), while MADE-IN/MADE-FOR only cost $0.1 per woman-year of exposure. It should, however, be noted that the DHS collects much more information than MADE-IN/MADE-FOR.
This two step method is very similar to the rather loosely defined RAMOS (reproductive age mortality study) method which is often regarded as the gold standard for estimating maternal mortality in developing countries, if conducted properly [17–19]. Both methods start by identifying deaths to WRA together with information on time of death related to pregnancy status. The difference is that the first step of RAMOS uses all available methods - existing records (e.g. vital registration, health facilities reports, burial records), and any appropriate types of informants, to gather data on deaths to WRA. MADE-IN/MADE-FOR in contrast, restricted itself to using Kader and RT informants only - to keep costs down. The second step of RAMOS, investigating the cause of death to WRA, is very similar to MADE-FOR, except the latter only involves visits to probable PRDs. The MADE-FOR step is crucial in verifying PRDs, as otherwise false-positives and undetected matching cases severely inflate the capture-recapture MMR estimate.
This study shows that reliable local, recent estimates of maternal mortality can be obtained quickly and relatively cheaply using two independent informant networks to retrospectively identify cases. Capture-recapture estimation allows self-calibration, but requires avoidance of false-positives, and that matches are identified, which in this study was achieved by means of the visit to interview a relative of the dead women. The visit was also used to investigate cause of death and risk factors.
A similar approach might be used prospectively, although this is organisationally a very different task. Such a system might even be used as basis for a reliable vital registration system. Ultimately, Indonesia and all other countries in the world must develop reliable civil registration and health management information systems to provide the necessary data for monitoring and evaluation of health programs including reproductive health.
This work was conducted as part of an international research program - Immpact http://www.immpact-international.org funded by the Bill & Melinda Gates Foundation, the Department for International Development, the European Commission and USAID. The funders have no responsibility for the information provided or views expressed in this paper. The views expressed herein are solely those of the authors.
The authors acknowledge the support from the Banten Province Health Office, Serang and Pandeglang Districts Health Office, Serang and Pandeglang Districts Hospitals and the data collectors too numerous to mention all by name here.
- WHO, UNICEF, UNFPA, the World Bank: Maternal mortality in 2005. Estimates Developed by WHO, UNICEF, UNFPA and the World Bank. Geneva. 2007Google Scholar
- Hill K, El Arifeen Shams, Koenig Michael, Al-Sabir Ahmed, Jamil Kanta, Raggers Han: How should we measure maternal mortality in the developing world? A comparison of household deaths and sibling history approaches. Bull World Health Organ. 2006, 84 (3): 2006-Google Scholar
- Statistics Indonesia (BPS) and Macro International: Indonesia demographic and health survey 2007. 2008, Calverton, Maryland, USAGoogle Scholar
- Central Bureau of Statistics/CBS and ORC Macro: Indonesia 2002/2003 Demographic and Health Survey. 2003, BKKBN, Depkes RI dan ORC Macro. Jakarta, Indonesia, Calverton, Maryland, USAGoogle Scholar
- Shiffman J: Generating political will for safe motherhood in Indonesia. Soc Sci Med. 2003, 56: 1197-207. 10.1016/S0277-9536(02)00119-3.View ArticlePubMedGoogle Scholar
- Regional Planning Board of Banten Province and Bureau of Statistics of Banten Province: Banten in figures 2005. Banten, Indonesia. 2006Google Scholar
- International statistical classification of diseases and related health problems, 10th Revision. Instruction Manual. Geneva: World Health Organization, 2: [http://www.who.int/classifications/icd/ICD-10_2nd_ed_volume2.pdf]2
- Fottrell E, Byass P, Ouedraogo TW, Tamini C, Gbangou A, Sombie I, Hogberg U, Witten KH, Bhattacharya S, Desta T, Deganus S, Tornui J, Fitzmaurice AE, Meda N, Graham WJ: Revealing the burden of maternal mortality: a probabilistic model for determining pregnancy-related causes of death from verbal autopsies. Pop Health Metrics. 2007, 5: 1-10.1186/1478-7954-5-1.View ArticleGoogle Scholar
- Laska , Eugene M: The use of capture-recapture methods in public health. Bulletin of the World Health Organization. 2002, 80 (11): September 2, 2009, [http://www.scielosp.org/pdf/bwho/v80n11/80n11a02.pdf]Google Scholar
- WHO: Epi info Software. 2009, Assessed on 18 October, 2009, [http://www.who.int/chp/steps/resources/EpiInfo/en/index.html]Google Scholar
- Lunn DJ, Thomas A, Best N, Spiegelhalter D: WinBUGS -- a Bayesian modelling framework: concepts, structure, and extensibility. Statistics and Computing. 2000, 10: 325-337. 10.1023/A:1008929526011.View ArticleGoogle Scholar
- Sophal Oum, Chandramohan Daniel, Cairncross Sandy: Community-based surveillance: a pilot study from rural Cambodia. Tropical Medicine and International Health. 2005, 10 (7): 689-697. 10.1111/j.1365-3156.2005.01445.x.View ArticleGoogle Scholar
- Barnett S, Nair N, Tripathy P, Borghi J, Rath S, Costello A: A prospective key informant surveillance system to measure maternal mortality - findings from indigenous populations in Jharkhand and Orissa, India. BMC Pregnancy& Childbirth. 2008, 8: 6-View ArticleGoogle Scholar
- Chandy Hoey YV, Heng , Samol Ha, Husum H: Comparing two surveys for estimating, maternal and perinatal mortality in rural Cambodia. Women and Birth. 2007, 21: 9-12. 10.1016/j.wombi.2007.10.003.View ArticleGoogle Scholar
- Horon Isabelle L: Underreporting of Maternal Deaths on Death Certificates and the Magnitude of the Problem of Maternal Mortality. American Journal of Public Health. 2005, 95 (3):Google Scholar
- HMN: Issues in Health Information - Household and Facility Surveys. Assessed on June 12, 2007, [http://www.who.int/healthmetrics/library/issue_5_05apr.doc]
- Hill K, Arifeen SE, Koenig M, Al-Sabir A, Jamil K, Raggers H: How should we measure maternal mortality in the developing words? A comparison of household deaths and sibling history approaches. Bulletin of the World Health Organization. 2006, 84 (3): 27 July 2007, [http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2627303/pdf/16583075.pdf]Google Scholar
- Stanton C, Abderrahim N, Hill K: DHS maternal mortality indicators: an assessment of data quality and implications for data use. DHS Analytical Report No. 4. 1997Google Scholar
- Atrash HK, Alexander S, Berg CJ: Maternal mortality in developed countries: not just concern of the past. Obstet Gynecol. 1995, 86: 700-705. 10.1016/0029-7844(95)00200-B.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2393/10/74/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.