Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Quality assessment of evidence-based clinical practice guidelines for the management of pregnant women with sickle cell disease using the AGREE II instrument: a systematic review

BMC Pregnancy and Childbirth volume 20, Article number: 595 (2020) Cite this article

Abstract

Background

The management of pregnant women with sickle cell disease (SCD) poses a major challenge for maternal healthcare services owing to the potential for complications associated with morbidity and mortality. Trustworthy evidence-based clinical practice guidelines (CPGs) have a major impact on the positive outcomes of appropriate healthcare. The objective of this study was to critically appraise the quality of recent CPGs for SCD in pregnant women.

Methods

Clinical questions were identified and the relevant CPG and bibliographic databases were searched and screened for eligible CPGs. Each CPG was appraised by four independent appraisers using the AGREE II Instrument. Inter-rater analysis was conducted.

Results

Four eligible CPGs were appraised: American College of Obstetricians and Gynecologists (ACOG), National Heart, Lung, and Blood Institute (NHLBI), National Institute of Health and Care Excellence (NICE), and Royal College of Obstetricians and Gynaecologists (RCOG). Among them, the overall assessments of three CPGs (NICE, RCOG, NHLBI) scored greater than 70%; these findings were consistent with the high scores in the six domains of AGREE II, including:[1] scope and purpose,[2] stakeholder involvement,[3] rigor of development,[4] clarity of presentation,[5] applicability, and [6] editorial independence domains. Domain [3] scored (90%, 73%, 71%), domain [5] (90%, 46%, 47%), and domain [6] (71%, 77%, 52%) for NICE, RCOG, and NHLBI, respectively. Overall, the clinical recommendations were not significantly different between the included CPGs.

Conclusions

Three evidence-based CPGs presented superior methodological quality. NICE demonstrated the highest quality followed by RCOG and NHLBI and all three CPGs were recommended for use in practice.

Peer Review reports

Background

Sickle cell disease (SCD) is a genetic disorder that causes a vaso-occlusive phenomenon and hemolysis along with a myriad of other major complications that could be life-threatening. It is one of the commonest inherited diseases, worldwide, and is inherited as an autosomal recessive disease due to the substitution of valine for glutamic acid at the sixth amino acid of the beta-globin chain [1]. This substitution leads to the production of a hemoglobin variant that is poorly soluble when deoxygenated. The clinical features, which include a vaso-occlusive crisis, are the result of the polymerization of the deoxygenated hemoglobin S. SCD is associated with significant maternal morbidity and mortality in pregnant women. The recognized complications include maternal mortality, preeclampsia, eclampsia, venous thromboembolism, cesarean delivery, intrauterine fetal death, and fetal growth restriction [2].

The prevalence of SCD varies among countries. For example, data from the United States showed that the overall prevalence is roughly about 4.83 per 10,000 deliveries; [3] 28.5% of women with SCD develop a crisis at the time of delivery. The maternal mortality rate was reported to be 1.6 per 1000 deliveries in women with SCD compared to 0.1 per 1000 deliveries in those without SCD [3].

Information about the prevalence of SCD in Saudi Arabia is limited and different among the various provinces, with the highest prevalence reported in the Eastern province followed by the Southwest province [4].

SCD in pregnancy tends to cause higher episodes of painful crises and a higher frequency of blood transfusion [2]. Although the complications of SCD are more commonly associated with the HbSS genotype, patients with other types of SCD such as sickle-beta thalassemia (HbSC genotype) and Hemoglobin SC disease (HbSC genotype) should receive the same level of care as those with HbSS. The development of a multidisciplinary care approach and comprehensive sickle cell centers seem to be associated with a decrease in the incidence of perinatal complications [5, 6].

Clinical Practice Guidelines (CPGs) were defined by the Health and Medicine Division of the American National Academies (formerly, the Institute of Medicine [IOM]), as “statements that include recommendations intended to optimize patient care and are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options” [7]. To date, there are no national CPGs for the management of SCD in pregnant women in Saudi Arabia.

The second edition of the Appraisal of Guidelines for Research and Evaluation Instrument (AGREE II) is the gold standard for the quality assessment or critical appraisal of CPGs. It was first published in its original form in 2003 and lastly updated in 2017 by the AGREE enterprise. AGREE II is a validated quantitative tool that has been cited in more than 1013 articles and endorsed by several healthcare organizations [8, 9]. AGREE II identifies constituents that must be addressed by CPGs to improve their quality, and henceforth, ensure their expected trustworthiness, and positive impact on healthcare outcomes [9].

The objective of this study was to conduct a systematic review and critically appraise the quality of recently published CPGs for SCD in pregnancy using the AGREE II instrument as a part of the CPG adaptation program [10].

Methods

The protocol for this study is published in the International Prospective Register of Systematic Reviews (PROSPERO; https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=145443) [11].

Inclusion and exclusion criteria

Three reviewers independently reviewed the retrieved CPGs, based on the following inclusion criteria: evidence-based with clear and detailed documentation of the development methods; available in the English language; obtained from original sources (de novo developed); had a national or international scope; published, amended or updated between January 1, 2014, and December 31, 2018 (the search was repeated before the submission of the final manuscript to identify new relevant CPGs; those published over the last five years only (2014–2018) were included in this study (because the period between two updates was reported to range from two to five years, in several CPG handbooks). CPGs published by an organization or having group authorship in a CPG database or peer-reviewed journal were also included in this study. Only the most current version of each Source CPG was included [11, 12].

The exclusion criteria included CPGs that were published earlier than 2014, written in a non-English language, adapted from other Source CPG (s), presented as consensus or expert-based statements, and written by a single author [11].

Search strategy and selection of SCD in pregnancy CPGs

Literature searches of bibliographic databases (Medline/PubMed and Google Scholar), EBSCO DynaMed Plus (USA) and relevant CPG databases, such as the ECRI Institute Guidelines Trust, National Institute of Health and Care Excellence (NICE; UK), Guidelines International Network (G-I-N) International guideline library, Scottish Intercollegiate Guidelines Network (SIGN; UK), and the Australian National Health and Medical Research Council (NHMRC; Australia) were performed. Moreover, we searched databases of national and international societies specializing in fields related to the topic of SCD in pregnancy, such as the American College of Obstetricians and Gynecologists (ACOG), Royal College of Obstetricians and Gynaecologists (RCOG), Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG), Society for Maternal Fetal Medicine (SMFM), Saudi Society for Obstetrics and Gynecology (SSOG), and Arab Association of Obstetrics and Gynaecology Societies (FAGOS). The keywords used were “sickle cell disease” AND “pregnancy” OR “pregnant women” AND “guideline,” “practice guideline,” “clinical practice guideline,” “practice parameter,” “guidance,” OR “recommendations” [11].

The PubMed electronic search strategy included the following: “anemia, sickle cell“[MeSH Terms] OR (“anemia“[Title] AND “sickle“[Title] AND “cell“[Title]) OR “sickle cell anemia“[Title] OR (“sickle“[Title] AND “cell“[Title] AND “disease“[Title]) OR “sickle cell disease“[Title]) AND “pregnan“[Title] AND (“pregnancy“[MeSH Terms] OR “pregnancy“[Title]) OR (“pregnant women“[MeSH Terms] OR (“pregnant“[Title] AND “women“[Title]) OR “pregnant women“[Title]) AND (“guideline“[Publication Type] OR “guidelines as topic“[MeSH Terms] OR “guidelines“[Title]) AND (Practice Guideline [ptyp] AND (“2014/01/01“[PDAT] : “2019/12/31“[PDAT]) AND “humans“[MeSH Terms]) AND (practiceguideline [Filter]). Furthermore, the PIPOH (Patient Population, Interventions, Professionals, Outcomes, and Healthcare Setting) model was used to support the CPG eligibility process [10]. Three reviewers (YA, MA, YS) independently screened titles and abstracts of the retrieved CPGs and articles that met the inclusion criteria. The screening was re-checked by three different reviewers (GE, AA, OK). Disagreements were resolved by focus group discussions after retrieval and review of the full-text articles or full CPG documents, including links to any accessible online supplementary documents or web resources. The search was repeated before the submission of the final manuscript to retrieve any new eligible CPG.

Assessment of CPGs using the AGREE II instrument

The AGREE II instrument (www.agreetrust.org) consists of 23 items organized in 6 domains: scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence [13]. Each item is scored on a Likert scale (1–7). The AGREE II evaluation was guided by utilizing its online version, “My AGREE PLUS,” which supports the presence of a CPG “appraisal group” for each CPG that compiles and calculates the ratings of the items into domain ratings, and comments [13]. The four AGREE II appraisers in the study had the relevant clinical expertise in obstetrics and gynecology (YS, AA), internal medicine (GE, MA), and hematology (GE, MA); furthermore, an expert CPG methodologist (YA) was included. At the outset, the CPG methodologist conducted capacity building sessions for the reviewers via hands-on sessions on the concepts, evidence-based CPG standards, and use of the AGREE II instrument. Each reviewer scored his/her assigned CPGs, and the included CPGs was critically appraised by all the five reviewers. The CPG documents, including any updates, plus any relevant supplementary information or links to online webpages related to the CPG methods or CPG implementation tools were reviewed in full by all the appraisers. For each item, the AGREE appraisers were asked to record the justifications for their scores in the “Comments” section. Wide discrepancies between the assessors’ scores for items or questions (a difference of more than 3 between the scores) were resolved by asking those who had provided the outlying scores to re-assess the questions after discussions with the entire group. The standardized AGREE domain scores or ratings (%) were automatically calculated using My AGREE PLUS. A cut-off point of 70% was set for each AGREE standardized domain score or rating. After the appraisal, we focused on the scores of domains 3 and 5 to facilitate the filtration and final evaluation of the reporting quality of the included CPGs. Similar cut-off values have been reported in previous studies [14, 15]. In addition to the classification of the six AGREE II domains, the evidence-bases of the included CPGs and their references sections were screened for systematic reviews or meta-analyses, specifically for Cochrane reviews. We utilized the PRISMA statement flow diagram and checklist for reporting our review [16,17,18]. There was no patient nor public involvement in this study.

Inter-rater analysis

Inter-rater reliability assessment tests (IRR) were conducted to determine the agreement levels between the raters. Percent agreement IRR was used for every question (or item) in each standardized domain in the four CPGs included in this study to assess the level of agreement among the four raters and the percent agreement of the first overall assessment (OA1) of the AGREE II instrument. Furthermore, the Intra-class correlation coefficient (ICC) was used to measure the consistency in the ratings or capacities of the datasets that were gathered as clusters or arranged into clusters, including the second overall assessment (OA2 or “recommend this CPG for use”). ICC is one of the most prevalent IRR approaches used when the number of raters is more than two. A high ICC (or Kappa; K) value (near 1) indicated a high resemblance between standards from the same set. A low Kappa value (near zero) indicated that the standards from the same set are not alike. One-Way Random analysis of variance was used SPSS Statistics, version 21 was used to analyze the data in this study. The ICC was used due to the diversity in the numerical data obtained from the groups or clusters. It helped us in detecting the reproducibility of the results and in determining how closely the peers resembled each other with regard to certain traits or characteristics. The agreement between two ordinal scale classifications was evaluated, and weighted Kappa (Quadratic Weights) was used because the data was obtained from an ordered scale.

Linear weights were used because the difference between the first and second categories had the same importance as that between the second and third categories, and so on. The agreement was quantified using the K statistic, [19, 20] where K = 1 when there is perfect agreement between the classification systems, K = 0 when there is no agreement better than chance, and K is negative when the agreement is worse than chance. The strength of the agreement based on the K value was classified as follows: < 0.20 (poor), 0.21–0.30 (fair), 0.31–0.40 (moderate), 0.41–0.60 (good), 0.61–0.80 (very good), and 0.81–1.00 (excellent) [21].

Results

Identification of SCD in pregnancy CPGs

The results of the search in the PRISMA statement flow diagram are shown in Fig. 1 [17, 18]. The initial list of 33 records was reviewed and filtered by the assessors. Among them, 29 were excluded because they did not meet the inclusion criteria. The four recent SCDs in pregnancy CPGs that complied with our PIPOH and inclusion criteria were shown in Fig. 1. The CPGs were developed by ACOG in January 2007 (reaffirmed in 2018) [22], NICE in June 2012 (with a minor update in August 2016) [23], RCOG in August 2011 (updated in May 2018) [24], and the US Department of Health and Human Services, Public Health Service, National Institutes of Health, National Heart, Lung, and Blood Institute (NHLBI) in 2014 [25]

Fig. 1
figure 1

PRISMA flow diagram. Systematically searching and selecting the clinical practice guidelines for the management of pregnant women with sickle cell disease. From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. doi:https://doi.org/10.1371/journal.pmed1000097. For more information, visit www.prisma-statement.org

Full size image

.

Key characteristics of SCD in pregnancy CPGs

Table 1 highlights the characteristics of all the eligible CPGs; two CPGs were developed by US-based (n = 2, 50%), and two CPGs (n = 2, 50%) by UK-based organizations. The four CPGs were developed by two reference specialized professional organizations (ACOG, RCOG), and two national evidence-based healthcare improvement organizations (NICE, NHLBI). All the organizations are from high-income countries [22,23,24,25].

Table 1 Characteristics of the included CPGs
Full size table

Reporting the quality of SCD in pregnancy CPGs

The standardized domain ratings of AGREE II are summarized in Table 2, and the appraisers’ comments are presented in Table 3.

Table 2 AGREE II standardized domain scores for the included CPGs
Full size table
Table 3 Reviewers’ comments on the four CPGs organized according to the standardized domains in AGREE II 22–25*
Full size table

Domain 1: Scope and purpose

The AGREE II standardized score for domain 1 ranged from 76–93%. The scores of three CPGs were greater than 70% in domain 1 (NICE-2012, 93%; RCOG-2018, 89%; and NHLBI-2017, 88%).

Domain 2: Stakeholder involvement

The standardized scores for domain 2 ranged from 33–85%. The scores of two CPGs were greater than 70% (NICE-2016, 85% and RCOG-2018, 76%).

Domain 3: Rigor of development

The standardized scores for domain 3 ranged from 41–90%; the scores of three CPGs were greater than 70% (NICE-2016, 90%; RCOG-2018, 73%; and NHLBI-2017, 71%).

Domain 4: Clarity of presentation

The standardized scores for domain 4 ranged from 63% to 89%, and the cores of three CPGs were greater than 70% (NICE-2016, 89%; RCOG-2018, 83%; and NHLBI-2017, 83%).

Domain 5: Applicability

The standardized scores for domain 5 ranged from 24–90%. Only one CPG scored greater than 70% in this domain (NICE-2016 = 90%).

Domain 6: Editorial independence

The standardized scores for domain 6 ranged from 19–77%. The scores of two CPGs were greater than 70% (RCOG-2018, 76% and NICE-2016, 85%).

Overall assessment

The AGREE II standardized domain scores for the first overall assessment ranged from 46–83%. Three CPGs scored greater than 70% (NHLBI, NICE, RCOG), consistent with their high scores in the six domains. The calculated AGREE II domain scores are shown in Figs. 2 and 3. The radar maps illustrate the final scores, expressed as percentages, for every included CPG in each of the six domains in Fig. 2 and each of the 23 questions in Fig. 3. The higher standardized domain scores are mapped toward the periphery (closer to 100%) and lower domain scores are plotted toward the center. The graphs illustrate a visual display of the relative strengths or weaknesses of each CPG by domain, question, and OA1 when compared to the other plotted CPGs.

Fig. 2
figure 2

Using a Radar chart to map the AGREE II 23-questions, 6-domains, and the first overall assessment for eligible appraised clinical guidelines. Abbreviations: ACOG: American College of Obstetricians and Gynecologists, AGREE: Appraisal of Guidelines for Research and Evaluation, CPG: clinical practice guideline or guidance, D: AGREE II Domain, NICE: National Institute of Health and Care Excellence, NHLBI: National Institutes of Health, National Heart, Lung, and Blood Institute, Q: AGREE II Question (or Item), RCOG: Royal College of Obstetricians and Gynaecologists, and SCD: Sickle cell disease.

Full size image
Fig. 3
figure 3

Radar map of the AGREE II final standardized domain scores for eligible appraised clinical guidelines. Abbreviations: ACOG: American College of Obstetricians and Gynecologists, AGREE: Appraisal of Guidelines for Research and Evaluation, CPG: clinical practice guideline or guidance, NICE: National Institute of Health and Care Excellence, NHLBI: National Institutes of Health, National Heart, Lung, and Blood Institute, OA1: AGREE II overall assessment 1, RCOG: Royal College of Obstetricians and Gynaecologists, and SCD: Sickle cell disease.

Full size image

Recommending the CPGs for SCD in pregnancy for use in practice

The second (overall) assessment with regard to the recommendation for the use of the CPG in practice revealed a consensus between the reviewers. Two of the appraised CPGs were recommended for use without modification (RCOG and NHLBI), while the other two were recommended for use with modifications (NICE and ACOG).

The strengths and limitations of the included CPGs are summarized in Table 3, based on consensus and the comments of the CPG appraisers for each item in AGREE II. All four CPGs cited systematic reviews in their references list. The largest number of systematic reviews were cited in the RCOG CPG (n = 6), and five (83%) of them were Cochrane reviews. Overall, the different options of care in the management of SCD in pregnant women were similar in these CPGs (Table 4).

Table 4 Summary of key recommendations in the four CPGs from ACOG, NHLBI, NICE, and RCOG*
Full size table

Inter-rater analysis

The results of the IRR tests showed a high strength of agreement for every question in every domain in the four CPGs among the four raters. As well as the percent agreement of the first overall assessment (OA1) in Fig. 2. Most of the K values were between 0.50 and 1.00, denoting good to excellent agreement. Only two evaluations (Fig. 3) pertaining to question 6 in domain 2 (D2Q6) and question 8 in domain 3 (D3Q8) revealed poor strengths of agreement (K = 0.0) in the ACOG CPG. As seen in Table 5, out of the 24 questions in the ACOG CPG, one demonstrated excellent agreement (K = 1), 16 showed good agreement (K = 0.5), 5 presented with very good agreement (K = 0.6–0.8), and two with poor agreement (K = 0.00); the results of the overall assessment (1) demonstrated good agreement (K = 0.5). In the RCOG 2011 evaluation, none of the 24 questions demonstrated excellent agreement, 15 presented with good agreement (K = 0.5), and 9 with very good agreement (K = 0.6–0.8); the overall assessment (1) showed good agreement (K = 0.5). In the NICE 2012 evaluation, one out of 24 questions demonstrated excellent agreement (K = 1), 16 presented with good agreement (K = 0.5), and seven with very good agreement (K = 0.6–0.8); the overall assessment (1) showed good agreement (K = 0.5). In the NHLBI evaluation, none of the 24 questions demonstrated excellent agreement, 15 presented with good agreement (K = 0.5), and 9 with very good agreement (K = 0.6–0.8); the overall assessment (1) showed good agreement (K = 0.5; Table 5). The results of the K value among the raters for the four guidelines with regard to the second overall assessment (OA2) revealed the following: number of observed agreements, 6 (37.50% of the observations); number of agreements expected by chance, 4.0 (25.00% of the observations); K = 0.167; standard error of K = 0.138 (95% confidence interval, − 0.103–0.437); and weighted K = 0.077.

Table 5 Classification of the strength of agreement among the four raters for the four clinical practice guidelines
Full size table

Discussion

To the best of our knowledge, this is the first study to systematically evaluate the quality of recently published CPGs for SCD in pregnancy using the AGREE II instrument.

Four CPGs addressing the management of pregnant women with SCD were assessed using the AGREE II instrument. Several areas of improvement in the methodological rigor of the included CPGs were highlighted. One CPG (ACOG) had significant gaps in its rigor of development (domain 3), which is the largest and core domain, and three CPGs demonstrated areas for improvement in their applicability (Domain 5). The weights of these two domains have been emphasized in this study. The NICE CPG received the highest reviewer agreement ratings [23]. All four CPGs included in this review had commonalities and differences in their clinical recommendations (Table 4). The common factors included genetic screening (ACOG, RCOG), genetic diagnosis (ACOG, NHLBI, RCOG), counseling during pregnancy (all four CPGs), transfusion or prophylactic exchange transfusion (ACOG, NICE, RCOG), fetal surveillance (ACOG, NHLBI, RCOG), and contraception (NHLBI, RCOG).

One discrepancy was observed in the form of a lack of clearly articulated recommendations for the vaccination status before pregnancy in three CPGs (ACOG, NHLBI, NICE). Two CPGs (ACOG and RCOG) were more specific about pregnancy when compared with the two other CPGs that contained general recommendations for SCD and smaller sections that focused on pregnancy. Out of the two specific CPGs, RCOG consistently presented with higher scores in all the domains. This systematic and objective assessment of the available CPGs is beneficial in supporting the decision to adopt or adapt the CPGs in clinical practice. After reviewing the four CPGs and given the appropriate rigor, consistently high scores, and clinical relevance of RCOG, we decided to adopt all the recommendations of this CPG in our clinical practice.

The findings of the current study revealed that the CPG assessment was accurate. There was excellent/ very good inter-rater agreement between the four assessors who evaluated the four CPGs using AGREE II. The proposed approach can be considered as a model for similar systematic reviews and quality assessments of CPGs.

Furthermore, the statistical analysis in this study illustrated the practicability of the AGREE II instrument as a valuable tool for the critical appraisal of CPGs, without compromising on the quality. Conceivably, inexperienced staff or non-professional reviewers would not have reached similar agreements with regard to the clinical features or characteristics in the CPGs, which could impact the judgment related to the provision of care to pregnant women with SCD.

In the first half of 2019, more than eight systematic critical appraisals of CPGs in obstetrics and gynecology were published using the AGREE II instrument. These included high priority health topics such as induction of labor[26], planned cesarean Sect. [27], recurrent pregnancy loss[28], packed red cells versus whole blood transfusion for severe pregnancy-related anemia and obstetric bleeding[29], gestational diabetes mellitus[30,31,32], and bladder pain syndrome/interstitial cystitis [33]. These studies identified several gaps, including differences, discrepancies, lack of evidence-base, and inconsistencies in some clinical recommendations among the CPGs; in addition, a few commonalities and similarities in some recommendations with advice to improve these variabilities were observed in CPGs [26,27,28,29,30,31,32,33].

Strengths and limitations

One of the strengths of the current study is that the appraisal conducted in this review was performed by a specialized clinical team of obstetricians and gynecologists, internists, and hematologists guided by an expert CPG methodologist, which adds a layer of strength to the AGREE II assessment. The other advantages of this study include the following: (i) the use of an international, rigorously structured, and validated CPG appraisal tool: the AGREE II instrument; (ii) the appraisal of each CPG by four raters including three clinical topic experts and a CPG methodologist; (iii) a comprehensive search within several databases; and (iv) statistical assessment of the inter-rater differences.

Care providers for pregnant women with SCD must be encouraged to adopt and merge the principles of “evidence-based” and “eminence-based” healthcare in their daily practice through continuous training and education about the standards of high-quality CPGs and their appraisal tools [34,35,36,37,38]. The results of this review can be used as a basis for CPG development or adaptation for pregnant women with SCD. Furthermore, they highlight the importance of the inclusion of the AGREE II criteria during capacity building, which will aid the clinicians in identifying and adopting the CPGs for use in daily practice.

Our study also has several limitations. First, the AGREE II instrument has several updates and different versions. Some of the disadvantages of AGREE II have been addressed in the recently developed “AGREE-REX” (Recommendation EXcellence) tool that reports the clinical credibility of the CPG recommendations. AGREE-REX has been validated and shared publicly on the website [39]. The selection of 70% as a cut-off point for standard domain ratings is another potential limitation because the original AGREE II does not mandate such a cut- off; however, some studies have used this cut-off value [40]. Other limitations include the following: (i) the inclusion of English language CPGs only may have resulted in the exclusion of relevant CPGs intended for use in non-English speaking healthcare settings; (ii) this review mainly focused on CPGs for the management of pregnant women with SCD, due to its known burden and priority for maternity health, and did not evaluate other subcategories because it was out of the scope of this study; and (iii) the included CPGs belonged to two different healthcare systems (i.e. US-based and UK-based).

Implications for practice: guidance for guideline uptake

The adaptation of CPGs has been identified as a valid and feasible alternative to de novo development, which is a resource-extensive process [41]. Evidence-based practice initiatives in some countries, especially those with low- and middle-income economies, have opted to utilize CPG adaptation rather than development [10]. Several formal adaptation methodologies are now available and could be further customized to local contexts [42]. Similar reviews like ours could inform relevant CPG adaptation or development projects for the SCD in pregnant women especially for groups with little experience, in using the AGREE II instrument.

The current critical appraisal highlights the importance of performing quality assessments of CPGs by clinicians to ensure the transparency and strength of the CPG development process according to international CPG standards and to support the provision of the best practice for pregnant women with SCD. We recommend incorporating the AGREE II appraisal of CPGs in the capacity building plans for obstetricians, gynecologists, and hematologists.

Conclusions

The methodological qualities of three evidence-based CPGs were superior to that of the expert consensus. NICE followed by the RCOG and NHLBI SCD CPGs presented with the highest qualities and were recommended for use in practice. We recommend using the AGREE II criteria and the methodologies utilized in the RCOG and NICE CPGs as models.

Availability of data and materials

All data analyzed during this study are included in this published article.

Abbreviations

ACOG:

American College of Obstetricians and Gynecologists

AGREE:

Appraisal of Guidelines for Research and Evaluation

CPG:

clinical practice guideline or guidance

NICE:

National Institute of Health and Care Excellence

NHLBI:

National Institutes of Health, National Heart, Lung, and Blood Institute

RCOG:

Royal College of Obstetricians and Gynaecologists

SCD:

Sickle cell disease

References

  1. Wastnedge E, Waters D, Patel S, et al. The global burden of sickle cell disease in children under five years of age: a systematic review and meta-analysis. J Glob Health. 2018;8(2):021103. doi:https://doi.org/10.7189/jogh.08.021103. Epub 2008 Jun 4. PubMed PMID: 18533123.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Villers MS, Jamison MG, De Castro LM, James AH. Morbidity associated with sickle cell disease in pregnancy. Am J Obstet Gynecol. 2008;199(2):125.e1-5. doi: https://doi.org/10.1016/j.ajog.2008.04.016.

    Article  Google Scholar 

  3. Alayed N, Kezouh A, Oddy L, Abenhaim HA. Sickle cell disease and pregnancy outcomes: population-based study on 8.8 million births. J Perinat Med. 2014 Jul;42:487–92.

  4. Al Mulhim K. Pregnancy in sickle cell disease in the Al Hassa Region of Saudi Arabia. Ann Saudi Med. 2000;20(5–6):471-3.

    Google Scholar 

  5. Rahimy MC, Gangbo A, Adjou R, Dequenon C, Goussanou S, Alihonou E. Effect of active prenatal management on pregnancy outcome in sickle cell disease in an African setting. Blood. 2000;96:1685–9.

    Article  CAS  PubMed  Google Scholar 

  6. Lottenberg R, Hassell KL. An evidence-based approach to the treatment of adults with SCD. Hematology. Am Soc Hematol Educ Program. 2005; 58–65.

  7. Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines; Graham R, Mancher M, Miller Wolman D, et al., editors. Clinical Practice Guidelines We Can Trust. Washington (DC): National Academies Press (US); 2011. Available from: https://www.ncbi.nlm.nih.gov/books/NBK209539/ doi: https://doi.org/10.17226/13058.

  8. Brouwers MC, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ: Canadian Medical Association Journal. 2010;182(18):E839–42. doi:https://doi.org/10.1503/cmaj.090449.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Siering U, Eikermann M, Hausner E, Hoffman-Eber W, Neugebauer EA. Appraisal Tools for Clinical Practice Guidelines: a systematic review. PLoS ONE 2013. 8(12); e82915 doi: https://doi.org/10.137/journal.pone.0082915.

  10. Amer YS, Wahabi HA, Abou Elkheir MM, Bawazeer GA, Iqbal SM, Titi MA, et al. Adapting evidence-based clinical practice guidelines at university teaching hospitals: A model for the Eastern Mediterranean Region. J Eval Clin Pract. 2019;25:550–60. https://doi.org/10.1111/jep.12927.

    Article  PubMed  Google Scholar 

  11. Amer Y, Sabr Y, ElGohary G, Altaqy A, Khojah O, Alzahrani M. A systematic critical appraisal of evidence-based clinical practice guidelines for the management of sickle cell disease in pregnancy using the Appraisal of Guidelines for REsearch and Evaluation II (AGREE II) instrument. PROSPERO 2019 CRD42019145443 Available from: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42019145443.

  12. Vernooij RW, Sanabria AJ, Solà I, Alonso-Coello P, Martínez García L. Guidance for updating clinical practice guidelines: a systematic review of methodological handbooks. Implement Sci. 2014;9:3. https://doi.org/10.1186/1748-5908-9-3.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Login - AGREE Enterprise website. Agreetrust.org. [cited 14 August 2019]. Available from: https://www.agreetrust.org/my-agree/.

  14. Altokhais TI, Al-Obaid OA, Kattan AE, Amer YS, CPG Collaborative Groups. Assessment of implementability of an adapted clinical practice guideline for surgical antimicrobial prophylaxis at a tertiary care university hospital. J Eval Clin Pract. 2017;23:156–64. https://doi.org/10.1111/jep.12658.

    Article  PubMed  Google Scholar 

  15. Eady E, Layton A, Sprakel J, Arents B, Fedorowicz Z, van Zuuren E. AGREE II assessments of recent acne treatment guidelines: how well do they reveal trustworthiness as defined by the U.S. Institute of Medicine criteria? Br J Dermatol. 2017;177(6):1716–25. Doi: https://doi.org/10.1111/bjd.15777431.

    Article  CAS  PubMed  Google Scholar 

  16. Institute of Medicine. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press; 2011. https://doi.org/10.17226/13059.

    Book  Google Scholar 

  17. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009;6(7):e1000097. doi:https://doi.org/10.1371/journal.pmed1000097.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Liberati A, Altman D, Tetzlaff J, Mulrow C, Gøtzsche P, Ioannidis J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62(10):e1–34.

    Article  PubMed  Google Scholar 

  19. Cohen J. A Coefficient of Agreement for Nominal Scales. Educ Psychol Measur. 1960;20(1):37–46. https://doi.org/10.1177/001316446002000104.

    Article  Google Scholar 

  20. Fleiss JL, Shrout PE. Approximate interval estimation for a certain intraclass correlation coefficient. Psychometrika. 1978;43(2):259–62. doi:https://doi.org/10.1007/bf02293867.

    Article  Google Scholar 

  21. Bland JM, Altman DG. Statistics Notes: Measurement error and correlation coefficients. Bmj. 1996;313(7048):41–2. doi:https://doi.org/10.1136/bmj.313.7048.41.

  22. ACOG Practice Bulletin No. Hemoglobinopathies in Pregnancy. Obstetrics Gynecology. 2007;78(1):229–38. 109.

  23. Nice.org.uk. (2012). Sickle cell disease: managing acute painful episodes in hospital | Guidance | NICE. [online] Available at: https://www.nice.org.uk/guidance/cg143 [Accessed 30 Oct. 2019].

  24. Royal College of Obstetricians & Gynaecologists (RCOG). (2011). Sickle Cell Disease in Pregnancy, Management of (Green-top Guideline No. 61). [online] Available at: https://www.rcog.org.uk/en/guidelines-research-services/guidelines/gtg61/ [Accessed 30 Oct. 2019].

  25. Evidence-Based Management of Sickle Cell Disease: Expert Panel Report. 2014. (2014). Expert Panel Report (EPR). [online] Maryland: National Heart, Lung, and Blood Institute (NHLBI), p.iii-142. Available at: https://www.nhlbi.nih.gov/health-topics/evidence-based-management-sickle-cell-disease [Accessed 30 Oct. 2019].

  26. Coates D, Homer C, Wilson A, Deady L, Mason E, Foureur M, Henry A. Induction of labour indications and timing: A systematic analysis of clinical guidelines. Women and Birth. 2019;33(3):219.

    Article  PubMed  Google Scholar 

  27. Coates D, Homer C, Wilson A, Deady L, Mason E, Foureur M, Henry A. Indications for, and timing of, planned caesarean section: A systematic analysis of clinical guidelines. Women and Birth. 2019;33(1):22.

    Article  PubMed  Google Scholar 

  28. Youssef A, Vermeulen N, Lashley EL, Goddijn M, van der Hoorn ML. Comparison and appraisal of (inter) national recurrent pregnancy loss guidelines. Reproductive biomedicine online. 2019 Apr 25.

  29. Kohli N, Bhaumik S, Jagadesh S, Sales RK, Bates I. Packed red cells versus whole blood transfusion for severe paediatric anaemia, pregnancy-related anaemia and obstetric bleeding: an analysis of clinical practice guidelines from sub‐Saharan Africa and evidence underpinning recommendations. Tropical Med Int Health. 2019 Jan;24(1):11–22.

  30. Li-zhen L, Yun X, Xiao-Dong Z, Shu-bin H, Zi-lian W, Sandra DA, Bin L. Evaluation of guidelines on the screening and diagnosis of gestational diabetes mellitus: systematic review. BMJ open. 2019 May 1;9(5):e023014.

  31. Daley B, Hitman G, Fenton N, McLachlan S. Assessment of the methodological quality of local clinical practice guidelines on the identification and management of gestational diabetes. BMJ open. 2019 Jun 1;9(6):e027285.

  32. Zhang M, Zhou Y, Zhong J, Wang K, Ding Y, Li L. Current guidelines on the management of gestational diabetes mellitus: a content analysis and appraisal. BMC Pregnancy Childbirth. 2019 Dec;19(1):200.

  33. Pape J, Falconi G, Lourenco TR, Doumouchtsis SK, Betschart C. Variations in bladder pain syndrome/interstitial cystitis (IC) definitions, pathogenesis, diagnostics and treatment: a systematic review and evaluation of national and international guidelines. International urogynecology journal. 2019 May 9:1–1.

  34. Bhandari M, Zlowodzki M, Cole PA. From eminence-based practice to evidence-based practice: a paradigm shift. Minn Med. 2004;87(4):51–4.

    PubMed  Google Scholar 

  35. Ioannidis JPA. Evidence-based medicine has been hijacked: a report to David Sackett. J Clin Epidemiol. 2016;73:82–6. DOI:https://doi.org/10.1016/j.jclinepi.2016.02.012.

    Article  PubMed  Google Scholar 

  36. Greenhalgh T, Howick J, Maskrey N. Evidence based medicine: a movement in crisis? BMJ. 2014;348:g3725. https://doi.org/10.1136/bmj.g3725.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Isaacs D, Fitzgerald D. Seven alternatives to evidence based medicine. BMJ,1999, 319.7225: 1618. 505.

    Google Scholar 

  38. Grilli R, Magrini N, Penna A, Mura G, Liberati A. Practice guidelines developed by specialty societies: the need for a critical appraisal. The Lancet. 2000 Jan;8(9198):103–6. 355.

  39. Brouwers M, Florez I, McNair S, Vella E, Yao X. Clinical Practice Guidelines: Tools to Support High Quality Patient Care. Semin Nucl Med. 2019;49(2):145–52.

    Article  PubMed  Google Scholar 

  40. Anwer MA, Al-Fahed OB, Arif SI, Amer YS, Titi MA, Al-Rukban MO. Quality assessment of recent evidence-based clinical practice guidelines for management of type 2 diabetes mellitus in adults using the AGREE II instrument. J Eval Clin Pract. 2017;1–7. https://doi.org/10.1111/jep.12785.

  41. Amer YS, Elzalabany MM, Omar TI, Ibrahim AG, Dowidar NL. The ‘Adapted ADAPTE’: an approach to improve utilization of the ADAPTE guideline adaptation resource toolkit in the Alexandria Center for Evidence-Based Clinical Practice Guidelines. Journal of evaluation in clinical practice. 2015 Dec;21(6):1095–106. https://doi.org/10.1111/jep.12479.

  42. Wang Z, Norris SL, Bero L. The advantages and limitations of guideline adaptation frameworks. Implementation Sci. 2018;13:72. https://doi.org/10.1186/s13012-018-0763-4.

    Article  Google Scholar 

Download references

Acknowledgements

This study was supported by King Saud University, Deanship of Scientific Research, Research Chair for Evidence-Based Health Care and Knowledge Translation, Riyadh, Saudi Arabia. Furthermore, we would like to extend our thanks to the Researchers Support & Services Unit at King Saud University for technical support and the King Saud University Medical City for logistics and resources.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Clinical Practice Guidelines Unit, Quality Management Department, King Saud University Medical City, King Saud University, Riyadh, Saudi Arabia

    Yasser S. Amer

  2. Department of Pediatrics, King Khalid University Hospital, King Saud University Medical City, King Saud University, Riyadh, Saudi Arabia

    Yasser S. Amer

  3. Research Chair for Evidence-Based Health Care and Knowledge Translation, Deanship of Scientific Research, King Saud University, Riyadh, Saudi Arabia

    Yasser S. Amer

  4. Alexandria Center for Evidence-Based Clinical Practice Guidelines, Alexandria University, Alexandria, Egypt

    Yasser S. Amer

  5. Department of Obstetrics and Gynecology, College of Medicine, King Saud University, Riyadh, Saudi Arabia

    Yasser Sabr & Amer M. Altaki

  6. Prince Sattam Bin Abdul Aziz Research Chair for Epidemiology and Public Health, College of Medicine, King Saud University, Riyadh, Saudi Arabia

    Yasser Sabr

  7. University Oncology Center, King Saud University Medical City, College of Medicine, King Saud University, Riyadh, Saudi Arabia

    Ghada M. ElGohary

  8. Internal Medicine and Clinical Hematology, Faculty of Medicine, Ain Shams University, Cairo, Egypt

    Ghada M. ElGohary

  9. Pathology Department, College of Medicine, King Saud University, Riyadh, Saudi Arabia

    Osamah T. Khojah

  10. Morbidity and Mortality Unit, King Saud University Medical City, King Saud University, Riyadh, Saudi Arabia

    Ahmed El-Malky

  11. Public Health and Community Medicine Department, Theodor Bilharz Research Institute (TBRI), Academy of Scientific Research, Cairo, Egypt

    Ahmed El-Malky

  12. Department of Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia

    Musa F. Alzahrani

Authors
  1. Yasser S. Amer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yasser Sabr
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ghada M. ElGohary
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Amer M. Altaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Osamah T. Khojah
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ahmed El-Malky
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Musa F. Alzahrani
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have made substantial contributions and gave final approval of the conceptions, drafting, and final version of this manuscript. YSA conceptualized and designed the study. YSA, YSS, GME, AMA, and MFA contributed to the data collection, critical appraisal of guidelines. OTK helped in the data collection of guidelines. YSA, YSS, GME, AME, MFA have written the first draft of the manuscript. YSA and AME analyzed and interpreted the data. YSA, YSS and MFA supervised the procedures in the study and reviewed the drafts and final version of this manuscript. All authors have seen and approved the final version of the manuscript.

Author information

The authors are expert researchers in the relevant fields of Evidence-based healthcare and clinical practice guidelines (YSA), Hematology and Hemoglobinopathies (GME, OTK, and MFA), Obstetrics and gynecology and maternal healthcare (YSS, and AMA), and Public health and biostatistics (AME).

Corresponding author

Correspondence to Yasser S. Amer.

Ethics declarations

Ethical approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Amer, Y.S., Sabr, Y., ElGohary, G.M. et al. Quality assessment of evidence-based clinical practice guidelines for the management of pregnant women with sickle cell disease using the AGREE II instrument: a systematic review. BMC Pregnancy Childbirth 20, 595 (2020). https://doi.org/10.1186/s12884-020-03241-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12884-020-03241-y

Keywords

BMC Pregnancy and Childbirth

ISSN: 1471-2393

Contact us