Agreement of antenatal care indicators from self-reported questionnaire and the antenatal care card of women in the 2015 Pelotas birth cohort, Rio Grande do Sul, Brazil

Background Studies of healthcare service use during the pregnancy-postpartum cycle often rely on self-reported data. The reliability of self-reported information is often questioned as administrative data or medical records, such as antenatal care cards, are usually preferred. In this study, we measured the agreement of antenatal care indicators from self-reported information and antenatal care cards of pregnant women in the 2015 Pelotas Birth Cohort, Brazil. Methods In a sample of 3923 mothers, indicator agreement strengths were estimated from Kappa and prevalence-and-bias-adjusted Kappa (PABAK) coefficients. Maternal characteristics associated with indicator agreements were assessed with heterogeneity chi-squared tests. Results The self-reported questionnaire and the antenatal care card showed a moderate to high agreement in 10 of 21 (48%) antenatal care indicators that assessed care service use, clinical examination and diseases during pregnancy. Counseling indicators performed poorly. Self-reported information presented a higher frequency data and a higher sensitivity but slightly lower specificity when compared to the antenatal card. Factors associated with higher agreement between both data sources included lower maternal age, higher level of education, primiparous status, and being a recipient of health care in the public sector. Conclusions Self-reported questionnaire and antenatal care cards provided substantially different information on indicator performance. Reliance on only one source of data to assess antenatal care quality may be questionable for some indicators. From a public health perspective, it is recommended that antenatal care programs use multiple data sources to estimate quality and effectiveness of health promotion and disease prevention in pregnant women and their offspring.


Background
Antenatal Care (ANC) comprises a number of activities and procedures aimed at preserving the health of pregnant women while ensuring delivery of preventive measures, early detection of complications and the adequate management of pre-existing maternal disease states [1]. The reduction of maternal mortality remains a priority under "Goal 3: Ensure healthy lives and promote wellbeing for all at all ages" in the new Sustainable Development Goals (SDGs) agenda through 2030 [2]. In this context, information on maternal and child health is fundamental to ensure antenatal care quality and reduction of maternal and perinatal mortality. In Brazil, a pregnancy card records health information and details of antenatal and postpartum care visits. The card is a tool included in the Prenatal and Birth Humanization Program, part of the Ministry of Health medical record systems, and its use is mandatory to ensure that healthcare for pregnant women meets the national minimum quality standards. The card includes information on ANC visits, diagnostic findings and patient behaviors and links the different stages of the health care process. The systematic collection of data during each consultation can therefore be used to provide optimal health services at any point of care.
An alternative method for collection of data on health care received during the pregnancy-postpartum cycle is the use of individual questionnaires. The in-person interview or questionnaire is frequently used in epidemiology research because it provides a comprehensive and standardized assessment of the topic of interest. It is a relatively inexpensive tool that enables collection of data from a large population sample.
Medical records are considered the gold standard and are generally the preferred data source over interviews/ questionnaires. However, evidence indicates that reliability of any individual source of data is imperfect, possibly inaccurate, and thus the combination of sources may improve reliability and completeness of information [1,[3][4][5][6]. The objective of this study was to evaluate for the first time the agreement between self-reported questionnaire information and data recorded in the ANC cards from women in the 2015 city of Pelotas Birth Cohort study, Rio Grande do Sul, Brazil.

Study settings
The data is representative of births in the municipality of Pelotas in 2015. However, this birth cohort is not representative of the total number of births in Brazil for that period. In 2015, 5598 newborn babies were identified, of whom 4329 were children of mothers who were residents in the urban area of Pelotas (cohort study targeted population) and were invited to participate. The refusal rate was 1.3% in the perinatal period.

Sampling
This was a cross-sectional study based on the mothers of children eligible for the 2015 Pelotas Birth Cohort who were interviewed during pregnancy and/or at birth. The total number of eligible pregnant mothers was 4329 (in the case of multiple pregnancies only one record was kept for each mother). Fifty-nine records corresponded to multiple births, totaling 4270 mothers in the birth cohort. Of these mothers 4172 had attended ANC but 249 did not have an antenatal card available at the time of interview, resulting in a final sample of 3923 mothers included in this study (Additional file 1: Figure S1).
The questionnaires used during prenatal and perinatal visits of the 2015 Pelotas Birth Cohort are available at http://epidemio-ufpel.org.br/site/content/coorte_2015/ questionarios.php. More detailed information on the cohort methods and follow-up visits is provided elsewhere [7].

Variables
The information was collected through interviews during pregnancy and/or the perinatal period. During antenatal care, two antenatal visits were performed, one before gestational week 16 and the second between gestational weeks 16 and 24. The third follow-up was at delivery. One questionnaire was used at weeks 16, 24 and at delivery to collect sociodemographic characteristics including maternal age, education, marital status, skin color, family income, pre-pregnancy health, and if the mother was seen by the same health professional during ANC and type of ANC provider. The perinatal questionnaire asked for reproductive health history variablesdelivery characteristics, number of pregnancies, and if the mother was seen by the same health professional during ANC and type of ANC provider. ANC data were also collected at delivery and documented service utilization, details of clinical examination, occurrence of disease during pregnancy, prophylactic interventions (vaccination and supplements), and educational interventions (counseling). Additional file 2: Table S1 shows the questionnaire contents and data collection tools.
At the end of the interview the ANC card was photographed and information on ANC routines extracted and double entered into a database (Epi Info 6.04, Centers for Disease Control and Prevention, Atlanta, USA) by two trained research assistants. A consistency analysis with a frequency verification was performed using Stata 12® (Statacorp, College Station, TX, USA).
If there was no record of a given procedure on the card, it was considered as not performed. The agreement analysis was carried out only when the card was available with at least one record present. The questionnaire items and antenatal card variables used in this study are described in Additional file 3: Table S2.

Statistical analysis
The agreement between the card and the questionnaire for each variable of interest was estimated using Kappa coefficient (κ) with corresponding 95% confidence intervals (95% CI). Kappa and Prevalence-Adjusted-and-Bias-Adjusted Kappa (PABAK) [8] values were interpreted using Landis and Koch categorization [9] into almost perfect (>0.80), substantial (0.61-0.80), moderate (0.41-0.60), fair (0.21-0.40), slight (0.00-0.20) and poor (<0.00). Sensitivity and specificity are reported with 95% CI and were calculated assuming the ANC card to be the gold standard or assuming the questionnaire to be the gold standard.
The association of maternal characteristics with an agreement between the card and the questionnaire was evaluated by building a score ranging from 1 to 10 using those ANC indicators with adjusted Kappa coefficients in the categories of almost perfect to moderate (ten indicators as shown in Table 3). A value of 1 was assigned to each indicator agreement (1 = yes/yes or no/no). The score was dichotomized into low (≤7 points as zero) and high agreement (≥ 8 points as one) to conduct bivariate analysis using the score as the dependent variable. Heterogeneity chi-squared tests were used to measure the difference between low and high agreement categories. The analyses were performed in Stata 15® (Statacorp, College Station, TX, USA).

Sample characteristics
Within the cohort of 3923 women 82.6% were between 20 and 39 years of age, 14.4% were under 20 years of age, 71.9% were white, 33.7% had up to 8 years of schooling and 86.4% were living with a partner at the time of interview. With regard to socioeconomic data, 59.7% were between the first and third quintiles of family income. In relation to obstetrical data, 50.4% of the women were primiparous, 64.8% had undergone c-section delivery, 52.8% were seen by the same health professional during ANC, and 45.0% received ANC in the public health sector. When comparing our sample with mothers who did not attend or receive ANC and lacked an antenatal card, a statistically significant difference for maternal education, marital status, family income, and parity was found (Table 1). Table 2 shows the distribution of data from ANC cards and from the self-reported questionnaire.

Agreement between ANC card indicators and questionnaire information
From the 21 indicators in the ANC card we observed that agreement strength with self-reported questionnaire information was poor in 1 indicator (K < 0.00), slight for 16 (K between 0.00-0.20), fair for 2 (K between 0.21-0.40), and moderate for 2 (K between 0.41-0.60) ( Table 3). The agreement strength category obtained with the PABAK was higher than that obtained with the Kappa coefficient in 9 (43%) indicatorsantenatal care number of visits ≥6, reported date of last menstrual period, weight, symphysisfundal height, blood pressure, syphilis test, hypertension during pregnancy, diabetes during pregnancy, and sexually transmitted diseases (Table 3). Agreement strength categories for urinary tract infection during pregnancy, breast exam, hepatitis B vaccination, and counseling on physical activities such as walking were moderate, slight, fair, and poor respectively, and were equally categorized by PABAK and Kappa values. Agreement strength categories were lower with PABAK relative to those obtained with Kappa for gynecological exam, cervical cancer screening test, counseling on risks of alcohol use during pregnancy and counseling on risks of smoking during pregnancy (Table 3).

Sensitivity and specificity
Self-reported questionnaire with the ANC card as the gold standard The sensitivity of the self-reported questionnaire was high (>90%) for 28.6% of the indicators evaluated, all of them in the components of service use and clinical examination. Good sensitivity (≥80 and < 90%) was present in 23.8% and low sensitivity (<80%) in 47.6% of the indicators. Specificity was low (<80%) for all indicators except for diabetes and sexually transmitted diseases that had >90% specificity (Table 4.).
Antenatal card with the self-reported questionnaire as the gold standard The sensitivity of antenatal card was high (>90%) for 19.0% of the indicators evaluated, all of them in the clinical examination component. The number of visits showed good sensitivity (81.1%) and the rest presented low sensitivity. Specificity was high for 42.8% of the indicators, mainly those related to diseases during pregnancy and counselling. Good specificity (≥80 and < 90%) was present in 19.0% of the indicators (Table 4).
Maternal factors associated with an agreement between data sources Table 5 presents the maternal characteristics associated with the concordance between the ANC card and self-report questionnaire. There was a significantly higher agreement among pregnant women with younger age, more years of maternal schooling, primiparous status and those using a public sector health care provider.

Main findings
To our knowledge, this is one of the few studies evaluating the agreement between self-reported questionnaire from pregnant women and the antenatal card that recorded procedures and interventions during ANC [10,11]. The assessment of agreement between sources of information on ANC assistance is an issue that has aroused the interest of the global scientific community. Our key findings indicate firstly, that assessment of ANC from self-reported questionnaire has moderate to high agreement to ANC card indicators of service utilization, clinical examination and diseases during pregnancy whereas indicators of counseling performed poorly. Secondly, self-reported information presented higher data frequency and higher sensitivity but slightly lower specificity when compared to For some variables, the number of subjects does not add up to the total of the subjects included due to missing information the ANC card. Thirdly, factors associated with higher agreement between both data sources included young maternal age, more maternal schooling years, being a primiparous mother, and health care received in the public sector.

Moderate to high agreement on indicators of service utilization, clinical examination and diseases during pregnancy. Poor agreement on indicators of counseling
The moderate to high agreement in these indicators suggests that assessment of ANC through selfreported questionnaire or the antenatal card could be equivalent. However, caution is advised when interpreting these results, especially for indicators related to diseases during pregnancy. As has been shown in other studies [12], the performance of antenatal clinical exams such as measurements of weight, symphysis-fundal height, and blood pressure reached almost perfect agreement [10,12]. These findings may be related to the ability of the patient to identify the reason for the ANC procedure/action recorded by the health professional during the ANC visit. For example, self-reporting weight may be more accurate than self-reporting a cervical cancer screening test.
Patients are more likely to understand that weight is being measured when asked to step on a scale but less likely to discern whether a cervical cancer screening test is the reason for a gynecological examination [13][14][15]. Indeed we found a very low agreement on reporting of gynecological exam, cervical cancer screening test and breast exam ( Table 3). Given that the frequency of these procedures was lower in the ANC cards relative to self-reports, the low agreements may also stem from poor record-keeping by health professionals or that these tests, that are mandatory at the national level, were not carried out, as reported in other studies [10][11][12][13][14][16][17][18][19][20][21][22][23]. This would raise serious concerns regarding the quality of healthcare that women and their offspring receive.      Efforts to improve the quality of care for pregnant women in Brazil include a National Oral Health Policy [24] which mandates a dental consultation when initiating ANC. Our study found a fair agreement. However, the results should be analyzed with caution as 70.9% (1002/1412) of the pregnant women who reported having consulted with a dentist during pregnancy also participated in the oral health sub-study of this cohort [7], therefore in the general population agreement could be lower than reported.
For gestational diabetes, agreement was classified as almost perfect, a finding confirmed by other studies [25,26] where agreement was often higher than those reported for other chronic diseases [27,28]. Studies evaluating selfreported medical information have shown that patients can provide reasonably good reports on their illnesses [29,30]. Recent publications on urinary tract infection have shown moderate agreement between self-reporting and medical records with a low sensitivity [31,32], similar to the findings obtained by our study.
Some conditions such as diabetes, hypertension, anemia, and urinary tract infection may not have been recorded on the ANC cards, but women still refer to having experienced them. Similar findings have been reported in other studies [10,33,34]. One possible reason for over reporting of gestational diabetes is that the diagnosis requires a positive result on two different glucose tests. Thus, women with a positive first test but negative on the second may have reported gestational diabetes. Sexually transmitted diseases may have been underreported possibly due to stigmatization. Counselling indicators were more frequently selfreported than reported in the antenatal card. Reports of counselling pertaining to alcohol use and smoking during pregnancy had the lowest Kappa values and the highest difference between sources, perhaps due to a low probability of reporting of these habits to healthcare providers during antenatal care. Previous studies have shown that counseling and referrals (e.g., cervical cancer screening tests) are underreported in medical records compared to self-reported records, indicating that health professionals do not consistently register these interventions [35,36].
. Self-reported information presented higher data frequency and higher sensitivity but slightly lower specificity when compared to the ANC card The use of questionnaires in epidemiologic studies is considered a valid tool with many advantages for research. However, the quality of the information obtained by selfreporting is dependent on the type of disease [37,38], the characteristics of the participants [30,39], the design of the questionnaire, and the method used to administer it [40,41]. In addition, information obtained by selfreporting or by review of medical records may not be consistent. Several studies have shown that the agreement between the two sources of information is dependent on the type of variable collected [42][43][44].
When self-report was compared to the card as gold standard, we found that specificity and sensitivity were > 80% for 9.5% (2/21) and 52% (11/51) of ANC indicators respectively. The high sensitivity in this comparison may be related to the use of self-administered questionnaires as sources of information on exposures and outcomes, which may be more complete and with fewer omissions in responses by participants.
When we compared the antenatal card with self-report as the gold standard, we found that the specificity and sensitivity were > 80% for 60% (13/21) and 25% (5/21) of ANC indicators respectively. This high specificity suggests that data recorded on the cards corresponded to the most relevant aspects identified during medical consultation. The low sensitivity suggests that registration of other ANC procedures/activities was incomplete. One reason may be that data recorded on antenatal cards do not accurately reflect all events occuring during a medical visit [45,46]. Furthermore the content of antenatal cards is not standardized for data collection in a private versus public setting or designed for data collection by health professionals with responsibility for specific aspects of ANC.
Despite these limitations, data collected in the ANC card are preferable for making decisions on potentially significant clinical interventions during antenatal and perinatal stages [47].
. Maternal characteristics associated with the agreement between the ANC card and the self-reported questionnaire Sociodemographic characteristics may contribute towards the strength of agreement between data sources and determine ANC quality [48]. Our findings show that higher maternal age, lower educational status, pregnant women with ≥2 children and type of healthcare provider associate with a lower probability of high agreement between the antenatal card and the self-reported questionnaire. We were unable to find other studies on factors associated with ANC indicator agreements for the data sources used in our study. However, there is evidence that the demand for health services and ANC quality are typically related to the sociodemographic characteristics of pregnant women [49].

Limitations and implications
Due to the intrinsic limitations of the data sources analyzed herewith, judgements about their respective validity should be made with caution.
The quality of medical record documentation may be affected by omissions in reporting, for example tests may have been performed elsewhere and not transcribed onto the antenatal card or incomplete documentation due to time constraints for record-keeping by health professionals. In addition to problems with recordingkeeping, accuracy of medical record extraction may be compromised by bias, fatigue or distraction of the systematic reviewer. These findings suggest that a purely quality-based assessment on medical records data could fail to find information on performed procedures and interventions that the pregnant woman might have supplied. This can result in the loss of important information on some ANC indicators.
The quality of an evaluation that depends solely on self-reported information can be influenced by recall bias, by subjectivity or misunderstanding of the questions pertaining to diagnoses, procedures and interventions received during ANC. In addition, the accuracy of a self-report may be affected by culturally influenced factors such as the importance of events, awareness and knowledge of health conditions [50].

Conclusions
Our study shows that data from questionnaires or the ANC cards provide substantially different information on indicator performance. From a public health perspective this raises questions with regard to reliance on indicators derived from a single data source. It may be more prudent to assess the quality of ANC programs using multiple data sources to determine quality and effectiveness of health promotion and disease prevention programs in pregnant women and their offspring. Researchers should explore alternative methods and data sources to obtain consistent estimates of ANC quality, various different indicators, be aware of factors that may influence the accuracy of data sources and conduct substudies to collect such data when not available. The implications of choosing a questionnaire or medical records should be carefully considered when evaluating health services for clinical practice, research, and public health. Deciding which data source to use will also depend on the outcome of interest and if the data is used for clinical decision-making, performance tracking, or public health.
Additional file 2: Table S1. Questionnaire contents and data collection tools, Pelotas 2015.
Additional file 3: Table S1. Questionnaire items and the antenatal care card variables.
Abbreviations ANC: Antenatal care; CI: Confidence interval; PABAK: Prevalence-Adjustedand-Bias-Adjusted Kappa; SDGs: Sustainable Development Goals manuscript and analyzing and interpreting the data; ADB, FW, MRD, MFS and DGB were responsible for planning the cohort in which this study was carried out. All authors participated in the critical review of the final version of this manuscript, provided feedback on drafts, and approved the final version.

Funding
This article is based on data from the study "Pelotas Birth Cohort, 2015" conducted by the Postgraduate Program in Epidemiology at Universidade Federal de Pelotas, in collaboration with the Brazilian Public Health Association (ABRASCO). The 2015 Pelotas (Brazil) Birth Cohort is funded by the Wellcome Trust (095582). Additional financial assistance was provided by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior -Brasil (CAPES) Finance Code 001. Funding agencies had no role in study design, collection, analysis, data interpretation or manuscript writing.

Availability of data and materials
The dataset used in this study is available upon request.

Ethics approval and consent to participate
The antenatal follow-up of the 2015 Pelotas (Brazil) Birth Cohort Study was approved by the Superior School of Physical Education Research Ethics Committee from the Federal University of Pelotas under the protocol 522.064. After agreeing to take part in the study, women signed a written informed consent.

Consent for publication
Not applicable.