Caesarean section in four South East Asian countries: reasons for, rates, associated care practices and health outcomes
BMC Pregnancy and Childbirth volume 9, Article number: 17 (2009)
Caesarean section is a commonly performed operation on women that is globally increasing in prevalence each year. There is a large variation in the rates of caesarean, both in high and low income countries, as well as between different institutions within these countries. This audit aimed to report rates and reasons for caesarean and associated clinical care practices amongst nine hospitals in the four South East Asian countries participating in the South East Asia-Optimising Reproductive and Child Health in Developing countries (SEA-ORCHID) project.
Data on caesarean rates, care practices and health outcomes were collected from the medical records of the 9550 women and their 9665 infants admitted to the nine participating hospitals across South East Asia between January and December 2005.
Overall 27% of women had a caesarean section, with rates varying from 19% to 35% between countries and 12% to 39% between hospitals within countries. The most common indications for caesarean were previous caesarean (7.0%), cephalopelvic disproportion (6.3%), malpresentation (4.7%) and fetal distress (3.3%). Neonatal resuscitation rates ranged from 7% to 60% between countries. Prophylactic antibiotics were almost universally given but variations in timing occurred between countries and between hospitals within countries.
Rates and reasons for caesarean section and associated clinical care practices and health outcomes varied widely between the four South East Asian countries.
Caesarean section is a commonly performed operation on women that is globally increasing in prevalence each year [1–5]. There is a large variation in the rates of caesarean, both in high and low income countries, as well as between different institutions within these countries [3, 4].
In the past, recommended caesarean rates have been calculated using various methods and concepts, the most common of which is based on the number of births in a hospital. The most widely recommended upper limit rate of caesarean section was 15 percent as advocated by the World Health Organization (WHO) . This was based on caesarean rates of countries with the lowest maternal and neonatal mortality rate at the time of the recommendation, and took into account both developed and developing countries [4, 6]. Since then the World Health Organization has published a revision in 1994, stating that acceptable caesarean section rates should range between 5 and 15 percent .
Caesarean section in developing countries is associated with significant increases in maternal morbidity [4, 8] particularly following elective caesarean section  and caesarean section without medical indications . Increases in infant morbidity and mortality are associated with caesareans in developing countries [3, 4, 10]. However, in low income countries, very low caesarean rates (less than 1%) have been associated with higher maternal and infant mortality linked to the inability to perform a caesarean section when needed [4, 11].
Interventions aimed at reducing maternal and perinatal morbidity and mortality associated with caesarean have included auditing of the rates, indications for and associated health outcomes [12, 13], while interventions to reduce high caesarean rates and inappropriate caesarean practices have involved the use of best evidence such as in the WHO Reproductive Health Library  and mandatory second opinion for non-emergency caesarean section .
The SEA-ORCHID (South East Asia – Optimising Reproductive and Child Health in Developing countries) project  across four South East Asian countries found the average rate of caesarean section to be 27% . We therefore conducted this review of caesarean section practice in hospitals within the countries participating in SEA-ORCHID to assess information on the rates, reasons for and care practices associated with caesarean. We also looked at the pregnancy background of women and health outcomes for women who had a caesarean and their babies.
Nine hospitals participating in the SEA-ORCHID project representing different types of hospitals across four countries in South East Asia (Indonesia, Malaysia, The Philippines and Thailand) were audited, with support from three sites in Australia . The SEA-ORCHID project settings and methods have been published elsewhere .
Seven of the nine hospitals were tertiary (university and regional) referral institutions with regional referrals of women with a high risk pregnancy and two were provincial or district institutions. The hospital delivery care models included a multidisciplinary approach with midwives (including nurses with midwifery qualifications) or obstetric specialists. Caesarean section facilities and obstetric specialists were available, and doctors and/or midwives (including nurses with midwifery qualifications) conducted normal vaginal births in all hospitals.
Approval for the project was given by the local ethics committee of each hospital and by the ethics committee of the University of Sydney, the administering institution in Australia
As part of the SEA-ORCHID project baseline data collection, between January and December 2005, we previously reviewed the medical records of 9550 women and their 9665 infants (including 111 twins and two sets of triplets) admitted to the labour wards at the nine participating hospitals. Data were collected on a consecutive basis at five of the participating hospitals until a total of at least 1000 women's medical records had been reviewed. Cases were sampled using a variety of ratios at the four largest hospitals. This method was used to ensure data were collected for a minimum of three months from each hospital and over similar time periods.
For the current audit, medical records were reviewed by trained staff using pre-established and piloted data extraction forms. Information about women who gave birth by caesarean section and their babies was collected.
Main indications for caesarean section were collected by trained staff who selected a single main reason for caesarean section from a predetermined list (Table 1)
Other information collected from the medical record regarding maternal and perinatal care practice around caesarean section included prophylactic antibiotic use and estimated blood loss for women having a caesarean section (Table 2) and use of antibiotics during postnatal care for women (Table 3).
Health outcomes for infants born by caesarean section were collected and included gestational age at birth, birth weight, need for resuscitation, low Apgar scores at 1 and 5 minutes, stillbirth, babies born alive who later died and total death rates (Table 4).
Trained fieldworkers used a secure web-based database to manually enter the data. The online form allowed validation checks to be performed to detect discrepancies and missing data and thus ensured transcription errors were minimized.
Data analysis was performed using STATA software Version 8.0 . Descriptive analysis was performed between hospitals within countries as well as across countries. For categorical data, frequencies were used to describe maternal characteristics, maternal and perinatal care practices and birth outcomes. For continuous data, means and standard deviations (SDs) were used.
The SEA-ORCHID project was approved by the local ethics committees of each hospital and by the ethics committee of the University of Sydney, the administering institution in Australia.
Of the 9550 women, 2592 (27%) women and their 2645 (27%) babies were born by caesarean. Actual rates varied from 12% to 39% between hospitals and from 19% to 35% between countries (Table 1).
Rates and indications for caesarean section (Table 1)
The most common indications for caesarean were malpresentation, previous caesarean section, cephalopelvic disproportion, and fetal distress. In Indonesia and Malaysia, the most common indication was malpresentation with rates of 5.5% and 5.0% respectively. In The Philippines, caesarean in a previous pregnancy was the most common indication for a caesarean for mothers who gave birth again (10.1%), while cephalopelvic disproportion was the most frequent indication in Thailand (11.4%).
Common pregnancy complications such as preeclampsia and antepartum haemorrhage were not often given as indications for caesarean. Although maternal request for a caesarean was relatively frequent in one of the tertiary hospitals in Indonesia (3.7%), this was not an indication in Malaysia, Thailand and The Philippines.
Prophylactic antibiotic use for mothers who gave birth by caesarean section (Table 2)
Prophylactic antibiotics were almost universally given across all four countries in South East Asia, with only one tertiary hospital in The Philippines reporting a slightly lower rate of 91%. There was variation in the timing of prophylactic antibiotics, both between countries and between hospitals within countries. In Indonesia, prophylactic antibiotics were universally given post-operatively. In one Malaysian hospital they were always given pre-operatively, while in the other they were given post-operatively 95% of the time. In one hospital in The Philippines, mothers were given prophylactic antibiotics pre-operatively almost universally, while in the other hospital 41% of mothers received antibiotics pre-operatively and 43% post-operatively, with the remainder given intra-operatively after umbilical cord clamping. In Thailand almost 90% of women were given prophylactic antibiotics intra-operatively after umbilical cord clamping, with the next most common time of administration being pre-operatively.
Cephalosporin was the most common class of prophylactic antibiotics used across all hospitals with a rate of 73%. Ampicillin was the next most commonly used antibiotic in Malaysia and Thailand, while 'other' antibiotics were the next most common in Indonesia and The Philippines. The frequency of dose for prophylactic antibiotics varied both between countries and between hospitals in countries. Mothers in Indonesia received multiple doses of prophylactic antibiotics while mothers in Malaysia received either a single dose or multiple doses depending on their births or the hospital. In The Philippines and Thailand rates ranged from 1% to 93% for single doses of prophylactic antibiotics and from 7% to 99% for multiple doses.
Caesarean section and blood loss (Table 2)
In Indonesia, Malaysia, and Thailand, the majority of women were reported to have a less than 500 ml estimated blood loss, while in the Philippines 79% were estimated to have a greater than 500 ml blood loss. The reported postpartum haemorrhage rate > 500 ml for Indonesia was only 4%. Malaysia reported the highest rate for postpartum maternal transfusion (9%).
Postnatal care after caesarean section (Table 3)
Mothers were often given prophylactic antibiotics postnatally with rates varying between countries and between hospitals within countries. All mothers in Indonesia were given prophylactic antibiotics postnatally. Rates varied widely between hospitals in Malaysia (3% and 94%), The Philippines (38% and 98%) and Thailand (15% to 97%). The main reason for giving antibiotics postnatally to women was prophylaxis and this was commonly practiced in Indonesia (100%), Thailand (90%) and Malaysia (90%), although it was less common in The Philippines (36%) where wound infection was the main reason reported (52%) for postnatal antibiotic administration.
Birth and infant health outcomes (Table 4)
The mean gestational age at birth of babies born by caesarean across the hospitals was similar (range 37.7 (SD 2.2) to 39.2 (SD 1.4) weeks). The preterm birth rate (< 37 weeks gestation) varied widely from 3% to 18% between hospitals although similar from 10% to 13% between countries. Overall, 16% of the babies born by caesarean were of low birth weight (< 2500 g), with rates ranging from 8% to 28% between hospitals. The mean birth weight of babies ranged from 2.76 kg (SD 0.63) to 3.13 kg (SD 0.57) between hospitals and 2.8 kg (SD 0.60) to 3.06 kg (SD 0.60) between countries.
There were wide variations in the use of neonatal resuscitation at caesarean. Babies born by caesarean in Thailand received resuscitation in 60% of cases, however actual rates varied widely between hospitals, ranging from 21% in the regional hospital to 97% in the provincial hospital. In Malaysia, only 7% of babies born by caesarean received resuscitation. Rates of babies with Apgar scores < 7 at 5 minutes were higher in the two Indonesian hospitals compared with hospitals in the other three South East Asian countries (5% and 12% compared with range 1% to 4%).
The rates for caesarean section where the baby was stillborn ranged between 0% to 1% between countries and 0% to 2% between hospitals within countries. The rates for babies born alive by caesarean who then died were reported as 0% overall for Indonesia, Malaysia and The Philippines, with these three countries recording a rate of 1% in one hospital each, while Thailand recorded a rate of 1% in all hospitals as well as overall.
Caesarean section rates in South East Asian countries
Actual caesarean rates in developing countries, including South East Asia, are largely unknown because of a lack of reliable data. Our results showed the overall caesarean rates, for all hospitals and all countries in the audit, to be 27%, higher than the WHO recommended rates of between 5 and 15 percent . This may be attributable to the fact that most of the hospitals audited were referral centres, meaning a higher proportion of women with complications from other lower category hospitals would have been sent to these hospitals. Some of the variation in caesarean section rates between hospitals may be related to differing maternal characteristics. Regardless, the caesarean rates in the South East Asian countries and hospitals audited are higher than the nationally representative data available .
Main indications for caesarean section
Women in the four South East Asian countries audited were more likely to have a caesarean if they or their infant experienced malpresentation, previous caesarean section, cephalopelvic disproportion or fetal distress. The National Collaborating Centre for Women's and Children's Health (NCCWCH) with The Royal College of Obstetricians and Gynaecologists (RCOG)  guidelines list malpresentation, cephalopelvic disproportion and fetal distress as main indicators for caesarean section, consistent with the indications in our population. Trained staff coded the main reasons for caesarean section, therefore we consider misclassification unlikely to account for variation between institutions.
Previous caesarean section as an indication for caesarean section is not a recommendation of the NCCWCH/RCOG UK guidelines . The high incidence of this as an indicator for caesarean section in the four South East Asian countries could be due to women or providers choosing this option after a previous complicated birth; a scenario more common in developing countries . In addition, women may not be fully informed and educated about vaginal birth and associated pain and management . High caesarean rates may be attributed to limited knowledge and training of health professionals in developing countries causing limited implementation of recommendations such as vaginal birth after caesarean (VBAC). It is known that maternal morbidity particularly increases following elective caesarean section  and caesarean section without medical indications  in developing countries. It is known that VBAC is an option provided the details of the previous caesarean are available and there is close monitoring during labour with the ability to proceed to an emergency caesarean if needed [22, 23].
Other NCCWCH/RCOG-recommended indicators for caesarean section including multiple pregnancy, mother to child transmission of disease, maternal request, placenta praevia and preterm or small for gestational age  were reported as minimal indicators for caesarean section in the South East Asian hospitals audited.
Use of antibiotic prophylactic
The use of prophylactic antibiotics is recommended to reduce endometritis and wound infection after elective or non-elective caesarean section . This knowledge has been applied in all hospitals of the four South East Asian countries audited, where prophylactic antibiotics were almost always given.
Evidence suggests that prophylactic antibiotics should be administered pre-operatively to result in the lowest risk of surgical wound infection . The variation in timing of antibiotic prophylactic administration between hospitals, and the lack of consistent timing in relation to the type of institution may suggest that some individual hospitals have developed standardized policies for use of antibiotic prophylactics, while some individual health professionals may practice in line with their own preferences at other hospitals.
First generation cephalosporin and ampicillin have been found to be equally effective agents for antibiotic prophylaxis for women who underwent a caesarean  and this recommendation was followed in nearly 90% of all cases reviewed in the four South East Asian countries.
Multiple doses of prophylactic antibiotics have been found no more effective than a single dose  and are more expensive. It is therefore of interest that our review found that all mothers who underwent caesarean in Indonesia, in one hospital in Malaysia and in one hospital in Thailand, were given multiple doses. Current evidence does not support this expensive practice.
Advantages and limitations of the study
There is lack of completeness of reporting for deaths and infection within South East Asian hospitals. The reporting may be improved for certain outcomes that may be considered as important quality control or assurance indicators for health care. Many of the SEA-ORCHID data indicators may be recommended for such purpose. The health indicators used were clearly defined and dedicated data staff were trained in their collection. The variation in care practices seen such as for use of resuscitation of the newborn at caesarean section, are likely to reflect different hospital policies as well as differences in casemix.
Adherence to best practice recommendations
Within nine hospitals in four South East Asian countries, our audit has shown varying and non-structured uptake of evidence-based clinical guidelines and recommendations in relation to caesarean section. This may be due to lack of availability and access to medical journals and reviews and therefore limited dissemination of evidence-based guidelines and recommendations. Availability of access and enablers and barriers to uptake of evidence based guidelines need to be examined at individual institutions. The SEA-ORCHID study plans to conduct a survey of evidence-based practice knowledge and clinical change among maternal and infant health practitioners in South East Asia to explore this issue . It would be of benefit for each institution to develop policies regarding caesarean, and particularly the timing and dosing of administration of prophylactic antibiotics so as to encourage standardized practice and to reinforce that access to knowledge and information is important.
The baseline rates of caesarean section, associated clinical practices and outcomes varied considerably in nine hospitals of four South East Asian countries comprising Thailand, The Philippines, Malaysia and Indonesia. The most common indications for caesarean delivery were malpresentation, previous caesarean section, cephalopelvic disproportion, and fetal distress. Maternal request remained rare. Giving prophylactic antibiotics was nearly universal, with variations in the timing of administration, (either pre-operatively, after cord clamping, or post-operatively), and variation also in the class and number of doses of antibiotics given. Blood loss during caesarean was commonly estimated to be less then 500 ml. A few women received blood transfusions. Postnatal care for mothers varied widely between hospitals within countries and also across countries.
Onsrud L, Onsrud M: Increasing use of caesarean section, even in developing countries. Tidsskr Nor Laegeforen. 1996, 116 (1): 67-71.
National Collaborating Centre for Women's and Children's Health: Caesarean section: Clinical Guideline. 2004, London: RCOG Press
Villar J, Valladares E, Wojdyla D, Zavaleta N, Carroli G, Velazco A, Shah A, Campodonico L, Bataglia V, Faundes A, Langer A, Narvaez A, Donner A, Romero M, Reynoso S, Simonia de Padua K, Giordano D, Kublickas M, Acosta A, for the WHO 2005 global survey on maternal and perinatal health research group: Caesarean delivery rates and pregnancy outcomes: the 2005 WHO global survey on maternal and perinatal health in Latin America. Lancet. 2006, 367: 1819-1829. 10.1016/S0140-6736(06)68704-7.
Althabe F, Sosa C, Belizán J, Gibbons L, Jacquerioz F, Bergel E: Caesarean section rates and maternal and neonatal mortality in low-, medium- and high-income countries: an ecological study. Birth. 2006, 33: 270-277. 10.1111/j.1523-536X.2006.00118.x.
Stanton C, Holtz S: Levels and trends in caesarean birth in the developing world. Stud Fam Plann. 2006, 37: 41-48. 10.1111/j.1728-4465.2006.00082.x.
World Health Organization: Appropriate technology for birth. Lancet. 1985, 2: 436-437.
World Health Organization: Indicators to monitor maternal health goals. Report of a Technical Working Group. 1994, Geneva: WHO
Hofmeyr G, Say L, Gulmezoglu A: WHO systematic review of maternal mortality and morbidity: the prevalence of uterine rupture. BJOG. 2005, 112: 1221-1228. 10.1111/j.1471-0528.2005.00725.x.
Oladapo O, Lamina M, Sule-Odu A: Maternal morbidity and mortality associated with elective caesarean delivery at a university hospital in Nigeria. Aust N Z J Obst Gynaecol. 2007, 47 (2): 110-114. 10.1111/j.1479-828X.2007.00695.x.
MacDorman M, Declercq E, Menacker F, Malloy M: Infant and neonatal mortality for primary caesarean and vaginal births to women with "no indicated risk". United States, 1998–2001 birth cohorts. Birth. 2006, 33: 175-182. 10.1111/j.1523-536X.2006.00102.x.
Ronsmans C, Holtz S, Stanton C: Socioeconomic differentials in caesarean rates in developing countries: a retrospective analysis. Lancet. 2006, 368: 1516-1523. 10.1016/S0140-6736(06)69639-6.
Holtz S, Stanton C: Assessing the quality of caesarean birth data in the Demographic and Health Surveys. Stud Fam Plann. 2007, 38: 47-54. 10.1111/j.1728-4465.2007.00115.x.
Huskins WC, Ba-Thike K, Festin MR, Limpongsanurak S, Lumbiganon P, Peedicayil A, Purwar M, Shenoy S, Goldmann DA, Tolosa JE, Global Network for Perinatal and Reproductive Health: An international survey of practice variation in the use of antibiotic prophylaxis in cesarean section. Int J Gynaecol Obstet. 2001, 73 (2): 141-5. 10.1016/S0020-7292(01)00365-4.
Gulmezoglu A, Langer A, Piaggio G, Lumbiganon P, Cillar J, Grimshawd J: Cluster randomised trial of an active, multifaceted educational intervention based on the WHO Reproductive Health Library to improve obstetric practices. BJOG. 2007, 114: 16-23. 10.1111/j.1471-0528.2006.01091.x.
Althabe F, Belizán J, Villar J, Alexander S, Bergel E, Ramos S, Romero M, Donner A, Lindmark G, Langer A, Farnot U, Cecatti JG, Carroli G, Kestler E, Latin American Caesarean Section Study Group: Mandatory second opinion to reduce rates of unnecessary caesarean sections in Latin America: a cluster randomised controlled trial. Lancet. 2004, 363: 1934-1940. 10.1016/S0140-6736(04)16406-4.
Henderson-Smart DJ, Lumbiganon P, Festin MR, Ho JJ, Mohammad H, McDonald SJ, Green S, Crowther CA, for the SEA-ORCHID Study Group: Optimising reproductive and child health outcomes by building evidence-based research and practice in South East Asia (SEA-ORCHID): study protocol. BMC Med Res Methodol. 2007, 7: 43-10.1186/1471-2288-7-43.
The SEA-ORCHID Study Group: Use of Evidence-Based Practices in Pregnancy and Childbirth: South East Asia Optimising Reproductive and Child Health in Developing Countries Project. PLoS ONE. 2008, 3 (7): e2646-10.1371/journal.pone.0002646.
StataCorp: Stata Statistical Software: Release 8. 2003, College Station. TX: Stata Corp LP
Betrán AP, Merialdi M, Lauer JA, Bing-Shun W, Thomas J, Van Look P, Wagner M: Rates of caesarean section: analysis of global, regional and national estimates. Paediatr Perinat Epidemiol. 2007, 21: 98-113. 10.1111/j.1365-3016.2007.00786.x.
Kwawukume EY: Caesarean section in developing countries. Best Pract Res Clin Obstet Gynaecol. 2001, 15 (1): B165-178. 10.1053/beog.2000.0155.
Tangcharoensathien V, Lertiendumrong J, Hanvaoravongchai A, Bennett S: Caesarean deliveries in Thailand: cause for concern. Regional Health Forum, WHO South-East Asia Region. 2007, 6 (2): [http://www.searo.who.int/EN/Section1243/Section1310/Section1343/Section1344/Section1356_5321.htm]
Royal College of Obstetricians and Gynaecologists (RCOG): Birth after previous caesarean birth. 2007, UK: Guidelines and Audit Committee RCOG, [http://www.rcog.org.uk/womens-health/clinical-guidance/birth-after-previous-caesarean-birth-green-top-45]
American College of Obstetricians and Gynecologists (ACOG): Vaginal birth after previous caesarean delivery. 2004, Washington (DC): American College of Obstetricians and Gynecologists (ACOG), [http://www.guideline.gov/summary/summary.aspx?doc_id=6374&nbr=4043&ss=15]
Smaill F, Hofmeyr G: Antibiotic prophylaxis for caesarean section. Cochrane Database Syst Rev. 2002, CD000933-3
Hopkins L, Smaill F: Antibiotic prophylaxis regimens and drugs for caesarean section. Cochrane Database Syst Rev. 1999, CD001136-2
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2393/9/17/prepub
The SEA-ORCHID study is jointly funded by an International Collaborative Research Grant from the National Health and Medical Research Council of Australia (No. 307703) and Wellcome Trust, United Kingdom (071672/Z/03/Z). All authors were funded individually by their respective university/institution for the preparation of the project proposal.
The following persons and institutions participated in the SEA-ORCHID Study Group.
P. Lumbiganon, MR. Festin, JJ. Ho, M Hakimi, DJ. Henderson-Smart, S. Green and CA. Crowther
SJ. McDonald, M. Laopaiboon
M. Murano, N. Narash
Data collection and entry at participating hospitals:
Country Investigator: M. Hakimi
Dr Sardjito Hospital, Yogyakarta, Indonesia
Data collection: Supiyah.
Data entry: L. Amanah, D. Lukitasari and D. Astuti
Sleman District Hospital, Yogyakarta, Indonesia
Data collection: L. Amanah
Data entry: L. Amanah, D. Lukitasari and D. Astuti
Country Investigator: JJ. Ho
Ipoh Hospital, Perak, Malaysia
Data collection: SS. Cham
Data entry: SS. Cham
Universiti Sains Malaysia, Kota Bharu, Malaysia
Data collection: ET. Aw
Data entry: SS. Cham
Country Investigator: P. Lumbiganon
Kalasin General Hospital, Kalasin, Thailand
Data collection: B. Khianman, A. Pharprapa, J. Nachaipet
Data entry: N. Narash and R. Poombankor
Khon Kaen Regional Hospital, Khon Kaen, Thailand
Data collection: B. Saenrien, N. Srisutthikamol, S. Panikom, C. Khunudom, S. Thipawat, S. Choonhapran, S. Nuanbuddee, P. Jarudphan, A. Hempira
Data entry: N. Narash and R. Poombankor
Srinagarind Khon Kaen Univeristy Hospital, Khon Kaen, Thailand
Data collection: P. Tharnprisan, P. Sarapon, O. Ponpun
Data entry: N. Narash and R. Poombankor
Country Investigator: MR. Festin
Jose Fabella Hospital, Manila, Philippines
Data collection: C. Ubaldo-Anzures and LN. Canete
Data entry: J. Magsipoc and JL. Festin
Philippines General Hospital (University of the Philippines), Manila, Philippines
Data collection: E. Torralba and LN. Canete
Data entry: J. Magsipoc and JL. Festin
Country Investigators: D. Henderson-Smart, S. Green, CA. Crowther
Project Coordinator: SJ. McDonald.
The authors declare that they have no competing interests.
MF, DHS and CC contributed to the design of the study. ML and PP cleaned and analysed the data and all authors contributed to the interpretation of data. ME prepared the first draft of the paper. All authors commented on each draft of the paper.
About this article
Cite this article
Festin, M.R., Laopaiboon, M., Pattanittum, P. et al. Caesarean section in four South East Asian countries: reasons for, rates, associated care practices and health outcomes. BMC Pregnancy Childbirth 9, 17 (2009). https://doi.org/10.1186/1471-2393-9-17