- Research article
- Open Access
Quality of consumer-oriented websites containing information about the second trimester ultrasound examination during pregnancy
BMC Pregnancy and Childbirth volume 20, Article number: 235 (2020)
Providing information about prenatal tests is a clinical challenge and the public frequently accesses the Web to read pregnancy-related information. The overarching aim of this study was to investigate the quality of consumer-oriented websites addressing obstetric ultrasound examination in the second trimester of pregnancy.
Swedish websites were identified with Google, using 20 search strings and screening 400 hits (n = 71 included websites). Reliability and information about the examination were assessed with the DISCERN instrument, completeness was assessed according to national guidelines, and readability analyzed with the Readability Index. Popularity was determined with the ALEXA tool and search rank was determined according to Google hit lists.
The mean total DISCERN score was 29.7/80 (SD 11.4), with > 50% having low quality for 15 of the 16 questions. The mean completeness score was 6.8/24 (SD 4.5). The Readability Index ranged between 22 and 63, with a mean of 42.7 (SD 6.8), indicating difficult readability. Weak and non-significant correlations were observed between ALEXA/search rank and the investigated quality variables, except for search rank and reliability.
The quality of consumer-oriented websites addressing the second trimester ultrasound examination is low. Health professionals need to discuss this with expectant parents considering undergoing the examination. There is a need for efforts that aim to improve the poor quality of online sources in the field of prenatal examinations.
When second trimester obstetric ultrasound examinations are offered routinely, most expectant parents accept the offer to undergo it and find the decision easy to reach . Expectant parents regard the examination as a positive event in which they expect to be presented with a visual confirmation of the pregnancy and to be reassured about the health of their expected child . However, there is an identified discrepancy between expectant parents’ expectations and the medical purposes of the examination, calling attention to the importance of pre-ultrasound information . Research shows that while most expectant parents are satisfied with the pre-ultrasound information, they still experience a lack of information about certain aspects . Moreover, those who receive news about a medical condition feel ill prepared and experience psychological distress . Various methods to offer information to expectant parents have been tested, but no standard that improves knowledge more than usual care has yet emerged . Providing information about prenatal tests is a clinical challenge for health professionals who work in maternity care. These health professionals express a lack of training about prenatal diagnostics  and often devote insufficient time to inform expectant parents about the examinations , indicating that they are unable to provide sufficient information about the different alternatives.
Today, the Web is a highly popular source of information about reproductive health, including pregnancy-related information . Studies show that most turn to the Web first when having a health-related query and that very few decide to talk directly to a health care provider instead , indicating that the Web now dominates as a first-line source of information. The Web has the potential to serve as a source of tailored and multimodal information that could aid in making health-related decisions through the dissemination of information and possibilities to promote interaction between health professionals and service users . More specifically, high-quality and accessible web-based information could potentially help expectant parents decide whether or not to undergo the second trimester ultrasound examination . However, the unregulated structure of the Web, including the lack of peer review processes or other regulatory activities, entails a risk that consumers can encounter information of low quality [13,14,15]. The quality of web-based sources has been questioned repeatedly over the course of several years across different health contexts [13, 16,17,18]. The multidimensional concept of quality of web-based information encompasses many aspects [13, 19], with the most commonly assessed in research being accuracy, completeness, content, readability, design, disclosures, and use of references . Consumers who turn to the Web need to have the skills to identify not only relevant and trustworthy sources, but also to read and understand the information encountered, so that they can reach informed decisions . While some studies report quality deficits for websites about non-invasive prenatal testing [20,21,22], knowledge is limited concerning the quality of web-based information about the second trimester ultrasound examination. Investigating the quality of consumer-oriented sources may result in valuable insights as to how to discuss use of the Web when counseling expectant parents. The overarching aim of this study was to investigate the quality of consumer-oriented websites addressing obstetric ultrasound examination in the second trimester of pregnancy. Specifically, the purpose was to assess the reliability, quality of information about the examination, completeness and readability of these websites. Further, we set out to investigate potential associations between the investigated quality criteria and website popularity or search rank.
This study concerns Swedish websites. In Sweden, obstetric ultrasound examinations are offered to all pregnant women in the second trimester of pregnancy. The examination is used to calculate gestational age, localize the placenta, determine number of fetuses, screen for fetal anomalies, and estimate volume of amniotic fluid. Midwives in maternity care services provide information about the examination and schedule an examination for those who want to undergo the procedure. Specialist midwives or obstetricians perform the examination. There is a very high accessibility to the Internet and almost all Swedes use the Web as a source of information .
The methods in this study adhere to current guidelines for systematic evaluations of websites as described in the literature . We designed the searches to resemble search patterns among the public, based on the results of previous research. This involves using various types of search strings, limiting the data collection to the first web page presented when accessing the link in the hit list, and screening the first ten links of the hit list before moving on to a new search [25,26,27]. Swedish websites about the obstetric ultrasound examination in the second trimester of pregnancy were identified through searches in Google, the most used search engine on the Web . In total, we designed 20 search strings consisting of both medical terminology as well as generic terms used by the public (Table 1). To check for common search terms used by the public in Sweden, Google Trends was used. We explored search terms in the categories “obstetric ultrasonography” and “ultrasound midwives”, as well as related search terms for each of the 20 chosen search strings. This did not reveal any alternative search terms and validated our chosen search strings. The searches were performed in January 2019. The hit list in the search engine ranged from 35,900 to 32,100,000.
The first 20 hits of each search string were screened for inclusion, resulting in 400 hits screened in total. To be included, the website needed to: (1) contain information about the obstetric ultrasound examination in the second trimester of pregnancy, (2) include text-based information in Swedish, (3) provide information developed for consumers, i.e. expectant parents who search for information about the examination, and (4) be publicly accessible without the need for a password. In total, 63 (16%) hits led to irrelevant information, i.e. websites that did not contain any information about the obstetric ultrasound examination in the second trimester of pregnancy. Of the remaining relevant websites (n = 337), 139 (41%) hits were excluded because they were news articles, included information for health professionals, were written by laypersons to communicate with peers, were not written in Swedish, were inaccessible, were scientific articles and did not contain any text-based information. After correcting for duplicate hits (n = 127), 71 websites were included (Fig. 1). The final sample originated from independent information websites (n = 29), healthcare system (n = 27), government (n = 9), pharmaceutical companies/pharmacies (n = 3), charities/private organizations (n = 2), and a museum (n = 1). All included websites were saved with Webcite, an online archiving system for web-based sources.
The analysis was guided by current recommendations for systematic evaluation of web-based information , including reliability, quality of information about the examination, comprehensiveness, and readability. The last author, a specialist nurse-midwife and researcher, assessed the included websites. Statistical analyses were performed with RStudio (version 1.0.143).
Reliability and quality of information about the examination
The DISCERN instrument, which is a valid and reliable tool , was used to assess the reliability and quality of information about the examination. DISCERN is used extensively in research and is recommended in the literature as a measure of quality in web-based sources . In total, the instrument includes three subscales: reliability (eight questions), information about the examination (seven questions), and overall quality (one question). Each of the 16 questions was rated on a scale from 1 (no/low quality) to 5 (yes/high quality), resulting in a total score ranging from 16 to 80.
An instrument was developed to assess completeness, inspired by national guidelines for information about prenatal screening (Table 2) . In total, the instrument included 24 dichotomous questions. Each question rated “yes” received a score of 1, resulting in a total score ranging from 0 to 24.
The readability of the text-based material was analyzed with the automated calculation Readability Index [Läsbarhetsindex] (LIX), developed to compute readability of Swedish texts. Scores range from < 25 (easiest) to > 60 (most difficult). Scores > 40 indicate that the text is too difficult for persons with average literacy levels to understand fully .
Popularity and search rank
The ALEXA tool  was used to explore domain popularity. This tool produces a rank estimate of popularity in a specific country, such as Sweden. The rank calculation is based on unique visitors per day and page views on the site over the past month. The domain with the highest combination of average visitors per day and page views over the past month is ranked number one, meaning that high popularity results in a low rank score. Search rank was determined by the highest number of each included website, as presented in the hit list in Google. Spearman’s rank correlation test was used to determine correlations between popularity/highest search rank and the investigated quality criteria. P-values <.05 were considered statistically significant.
Reliability and information about the examination
The mean total DISCERN score was 29.7 (SD 11.4), illustrating poor quality (Table 3). In total, > 50% of the included websites had scores illustrating low quality (1–2) for 15 of the 16 questions (Fig. 2). The questions with the highest proportion of websites with low quality scores (1–2) were: “Is it clear what sources of information were used to compile the publication?” (n = 68, 96%), “Does it provide support for shared decision-making?” (n = 67, 94%), and “Does it describe how results of the examination may affect overall quality of life?” (n = 63, 89%). In contrast, the questions with the highest proportion of websites with high quality scores (4–5) were: “Is it clear that there may be more than one possible prenatal test/examination?” (n = 21, 30%), “Does it describe the benefits of the examination?” (n = 20, 28%), and “Does it describe what would happen if no examination is performed and that it is voluntary?” (n = 19, 27%). Approximately three quarters of the included websites were rated low quality (1–2) for subscale 3, illustrating low overall quality with serious or extensive shortcomings (n = 51, 72%).
The mean completeness score was 6.8 (SD 4.5) (Table 3). For 18 of the total 24 completeness quality criteria, > 50% of the websites did not adhere to the criteria (Fig. 3). The criteria with the lowest proportion of websites were contains information about termination of pregnancy (n = 0, 0%), the information can be tailored depending on preferences and needs (n = 2, 3%), and refers to information about termination of pregnancy (n = 4, 6%). In contrast, the criteria with the highest proportion of websites were: contains information about the purpose to calculate gestational age (n = 57, 80%), contains information about the purpose to screen for fetal anomalies (n = 57, 80%), and contains information about the purpose to determine number of fetuses (n = 55, 77%).
The mean LIX was 42.7 (SD 6.8) (Table 3). The majority of the websites had LIX > 39 (n = 51, 72%), indicating that the texts had readability levels too difficult for average persons to understand fully (Table 4).
Popularity and search rank
The mean ALEXA rank score was 99.9 (range 1.4–862.0), indicating a high variability with regard to popularity. Weak and non-significant correlations were observed between ALEXA rank score and investigated quality variables (Table 5). There were weak and non-significant correlations between Google search rank and the investigated quality variables, except for reliability (rs = − 0.28, P = 0.02). Please see the figures for a presentation of the mean DISCERN scores (Fig. 4), mean completeness score (Fig. 5), and mean LIX (Fig. 6), for each rank in the 20 searches.
The overarching aim of this study was to investigate the quality of websites addressing obstetric ultrasound examination in the second trimester of pregnancy. The reliability and quality of information about the examination was low, the completeness according to national guidelines was insufficient, and readability was difficult. There were no association between popularity or search rank and the investigated quality criteria.
According to previous studies, health professionals report insufficient training about prenatal tests , devote little time or effort to inform about prenatal tests [3, 8] and do not provide expectant parents with enough information about the option to terminate the pregnancy when a fetal anomaly is discovered . Indeed, research illustrates that those who receive a prenatal diagnosis of fetal anomaly experience an overwhelming amount of information following the diagnosis [32, 33], and at the same time a lack of information about induced abortion [33, 34]. The websites had difficult readability, which further complicates the situation concerning informational uptake. Limited health literacy is prevalent and there is a need for improvement with regard to the readability levels of health-related information . Difficult readability has been reported for web-based information in various other fields, including pregnancy-related topics . While the overall quality of websites may be low, current algorithms in search engines aim to produce hit lists with links to high-quality websites as the first hits. Indeed, studies on search strategies among the public show that most rely on the first links in the hit list and the most popular websites [25, 26]. Thus, high-ranking websites and popular domains will have the highest impact and spread among information consumers . However, our results indicate no association between popularity/search rank and quality, illustrating that parents who use the most common methods for web-based searches encounter websites of low quality. Combined, the results indicate that there is a need for studies that aim to help expectant parents search for and identify web-based information. Health professionals need to be mindful of the risk of low-quality websites when being consulted by expectant parents and raise this topic for discussion. There is a need for considerable and systematic undertakings that aim to improve the quality of web-based materials about prenatal tests.
There are methodological limitations that need to be considered when interpreting the findings. The sample consisted of Swedish websites addressing second trimester obstetric ultrasound examinations. Worldwide, there are regional differences regarding maternity care services and offers of prenatal tests. This needs to be considered when interpreting the results of this study, as the generalizability may be limited. We aimed to replicate the search patterns of expectant parents. Thus, we based our method on previous reports of how the public use the Web to find health-related information [25, 26]. In total, 400 hits generated by 20 different search strings in Google were screened, which is by far the most used search engine on the Web . We cannot dismiss the possibility that expectant parents who search for websites with information about the ultrasound examination would encounter other websites than those identified in this study. It is also not possible to know how often expectant parents access the included websites. That the searches resulted in duplicate hits, meaning that several search terms led to the same websites, indicates saturation with regard to screened hits and that it is likely that expectant parents who search for Swedish information about the ultrasound examination are presented with a link to the included websites in their hit lists. This study investigated quality of consumer-oriented websites because we wanted the results to portray information developed with the purpose of providing information for expectant parents. Web-based information written for health professionals and those written by laypersons with the purpose of communicating with peers (e.g. blogs or discussion boards) were excluded. Expectant parents may decide to access these other types of websites to read about the ultrasound examination, which is not reflected in the results of this study. We acknowledge that the results may have limited generalizability due to the identification and inclusion of websites, and encourage more studies in this field so that a comprehensive understanding of the quality of these sources may be achieved.
The quality of the included websites was assessed with a focus on reliability, quality of information about the examination, completeness, and readability. That we assessed four aspects of website quality strengthens the results and is in line with current recommendations in the literature . The DISCERN instrument was used to assess the reliability and the quality of information about the examination, which is a reliable and valid tool developed by experts and patient representatives. The last author, a researcher specialist nurse-midwife with previous experience of website assessments with standardized instruments, assessed the included websites. It has been shown previously that health professionals and laypersons assess website quality similarly when using the DISCERN instrument . Nevertheless, we acknowledge that possibility that expectant parents may assess website quality differently and we would like to encourage more studies within this field of research.
The quality of consumer-oriented websites addressing second trimester ultrasound examination is low regarding reliability, information about the examination, completeness and readability. Expectant parents who search for supplemental web-based information are at risk of encountering unreliable and incomplete information that has difficult readability and does not contain sufficient information about the examination. Midwives and physicians who work in maternity care need to discuss the risk of encountering low-quality web-based information about the second trimester ultrasound when consulting expectant parents who consider undergoing the examination. The high variability regarding completeness, and the fact that many topics highlighted in national guidelines were missing in many of the included websites, indicate that websites collectively do not sufficiently facilitate informed decisions whether or not to undergo the examination. Expectant parents are at risk of encountering information with difficult readability levels, which needs to be considered when consulting expectant parents, particularly those with low health literacy levels. There is a need for overarching clinical efforts and research that aim to guide expectant parents to high-quality online sources and improve the poor quality of web-based sources about prenatal examinations.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Readability Index [Läsbarhetsindex]
Crang-Svalenius E, Dykes A-K, Jörgensen C. Factors influencing informed choice of prenatal diagnosis: women’s feelings and attitudes. Fetal Diagn Ther. 1998;13:53–61.
Garcia J, Bricker L, Henderson J, Martin M-A, Mugford M, Nielson J, et al. Women’s views of pregnancy ultrasound: a systematic review. Birth. 2002;29:225–50.
Lalor JG, Devane D. Information, knowledge and expectations of the routine ultrasound scan. Midwifery. 2007;23:13–22.
Georgsson Ohman S, Waldenström U. Second-trimester routine ultrasound screening: expectations and experiences in a nationwide Swedish sample. Ultrasound Obstet Gynecol. 2008;32:15–22.
Wool C. Systematic review of the literature: parental outcomes after diagnosis of fetal anomaly. Adv Neonatal Care. 2011;11:182–92.
Green JM, Hewison J, Bekker HL, Bryant LD, Cuckle HS. Psychosocial aspects of genetic screening of pregnant women and newborns: a systematic review. Health Technol Assess. 2004;8(33):iii ix–x, 1–109.
Ternby E, Ingvoldstad C, Annerén G, Axelsson O. Midwives and information on prenatal testing with focus on Down syndrome. Prenat Diagn. 2015;35:1202–7.
Ferm Widlund K, Gunnarsson C, Nordin K, Hansson MG. Pregnant women are satisfied with the information they receive about prenatal diagnosis, but are their decisions well informed? Acta Obstet Gynecol Scand. 2009;88:1128–32.
Sayakhot P, Carolan-Olah M. Internet use by pregnant women seeking pregnancy-related information: a systematic review. BMC Pregnancy Childbirth. 2016;16:65.
Volkman JE, Luger TM, Harvey KLL, Hogan TP, Shimada SL, Amante D, et al. The National Cancer Institute’s health information National Trends Survey [HINTS]: a national cross-sectional analysis of talking to your doctor and other healthcare providers for health information. BMC Fam Pract. 2014;15:111.
Moreira MWL, Rodrigues JJPC, Korotaev V, Al-Muhtadi J, Kumar N. A comprehensive review on smart decision support systems for health care. IEEE Syst J. 2019;13:3536–45.
Eysenbach G, Jadad AR. Evidence-based patient choice and consumer health informatics in the internet age. J Med Internet Res. 2001;3:E19.
Eysenbach G, Powell J, Kuss O, Sa E-R. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA. 2002;287:2691–700.
Cline RJ, Haynes KM. Consumer health information seeking on the internet: the state of the art. Health Educ Res. 2001;16:671–92.
Venot A, Burgun A, Quantin C. Medical informatics, e-health: fundamentals and applications. Paris: Springer; 2014.
Rowlands S. Misinformation on abortion. Eur J Contracept Reprod Health Care. 2011;16:233–40.
Swartzendruber A, Steiner RJ, Newton-Levinson A. Contraceptive information on pregnancy resource center websites: a statewide content analysis. Contraception. 2018;98(2):158–62.
Eriksson C, Skinstad M, Georgsson S, Carlsson T. Quality of websites about long-acting reversible contraception: a descriptive cross-sectional study. Reprod Health. 2019;16:172.
Burkell J. Health information seals of approval: what do they signify? Inf Commun Soc. 2004;7:491–509.
Mercer MB, Agatisa PK, Farrell RM. What patients are reading about noninvasive prenatal testing: an evaluation of internet content and implications for patient-centered care. Prenat Diagn. 2014;34:986–93.
Skirton H, Goldsmith L, Jackson L, Lewis C, Chitty LS. Non-invasive prenatal testing for aneuploidy: a systematic review of internet advertising to potential users by commercial companies and private health providers. Prenat Diagn. 2015;35:1167–75.
Farrell RM, Agatisa PK, Mercer M, Coleridge MB. Online direct-to-consumer messages about non-invasive prenatal genetic testing. Reprod Biomed Soc Online. 2015;1:88–97.
The Internet Foundation In Sweden. Svenskarna och internet 2018 [Swedes and the internet 2018]. 2018. https://www.iis.se/docs/Svenskarna_och_internet_2018.pdf. Accessed 28 Dec 2018.
Rew L, Saenz A, Walker LO. A systematic method for reviewing and analysing health information on consumer-oriented websites. J Adv Nurs. 2018;74(9):2218–26.
Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ. 2002;324:573–7.
Fiksdal AS, Kumbamu A, Jadhav AS, Cocos C, Nelsen LA, Pathak J, et al. Evaluating the process of online health information searching: a qualitative approach to exploring consumer perspectives. J Med Internet Res. 2014;16:e224.
Feufel MA, Stahl SF. What do web-use skill differences imply for online health information searches? J Med Internet Res. 2012;14:e87.
McCool ME, Wahl J, Schlecht I, Apfelbacher C. Evaluating written patient information for eczema in German: comparing the reliability of two instruments, DISCERN and EQIP. PloS One. 2015;10:e0139895.
Socialstyrelsen. SOSFS 2012:20. Fosterdiagnostik och preimplantatorisk genetisk diagnostik. 2012. https://www.socialstyrelsen.se/globalassets/sharepoint-dokument/artikelkatalog/foreskrifter-och-allmanna-rad/2012-12-34.pdf. Accessed 14 Apr 2020.
Björnsson CH. Läsbarhet [readability]. Stockholm: Liber; 1968.
ALEXA. Website Traffic, Statistics and Analytics. 2019. https://www.alexa.com/siteinfo. Accessed 27 Mar 2019.
Carlsson T, Bergman G, Melander Marttala U, Wadensten B, Mattsson E. Information following a diagnosis of congenital heart defect: experiences among parents to prenatally diagnosed children. PLoS One. 2015;10:e0117995.
Carlsson T, Bergman G, Wadensten B, Mattsson E. Experiences of informational needs and received information following a prenatal diagnosis of congenital heart defect. Prenat Diagn. 2016;36:515–22.
Andersson I-M, Christensson K, Gemzell-Danielsson K. Experiences, feelings and thoughts of women undergoing second trimester medical termination of pregnancy. PLoS One. 2014;9:e115957.
Paasche-Orlow MK, Parker RM, Gazmararian JA, Nielsen-Bohlman LT, Rudd RR. The prevalence of limited health literacy. J Gen Intern Med. 2005;20:175–84.
Bantan N, Abenhaim HA. Vaginal births after caesarean: what does Google think about it? Women Birth. 2015;28:21–4.
Mueller J, Jay C, Harper S, Davies A, Vega J, Todd C. Web use for symptom appraisal of physical health conditions: a systematic review. J Med Internet Res. 2017;19:e202.
Griffiths KM, Christensen H. Website quality indicators for consumers. J Med Internet Res. 2005;7:e55.
Open access funding provided by Uppsala University.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Georgsson, S., Carlsson, T. Quality of consumer-oriented websites containing information about the second trimester ultrasound examination during pregnancy. BMC Pregnancy Childbirth 20, 235 (2020). https://doi.org/10.1186/s12884-020-02897-w
- Consumer health information
- Second pregnancy trimester
- Prenatal care
- World wide web