This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on ISI (246)  • Contact me when this article is cited  Related Content  • Related articles  • Similar articles in JAMA  Topic Collections  • Evidence-Based Medicine  • Alert me on articles by topic

The Methodological Quality of Clinical Practice Guidelines in the Peer-Reviewed Medical Literature

Terrence M. Shaneyfelt, MD, MPH; Michael F. Mayo-Smith, MD, MPH; Johann Rothwangl, MD, FACG

JAMA. 1999;281:1900-1905.

Context  Practice guidelines play an important role in medicine. Methodological principles have been formulated to guide their development.

Objective  To determine whether practice guidelines in peer-reviewed medical literature adhered to established methodological standards for practice guidelines.

Design  Structured review of guidelines published from 1985 through June 1997 identified by a MEDLINE search.

Main Outcome Measures  Mean number of standards met based on a 25-item instrument and frequency of adherence.

Results  We evaluated 279 guidelines, published from 1985 through June 1997, produced by 69 different developers. Mean overall adherence to standards by each guideline was 43.1% (10.77/25). Mean (SD) adherence to methodological standards on guideline development and format was 51.1% (25.3%); on identification and summary of evidence, 33.6% (29.9%); and on the formulation of recommendations, 46% (45%). Mean adherence to standards by each guideline improved from 36.9% (9.2/25) in 1985 to 50.4% (12.6/25) in 1997 (P<.001). However, there was little improvement over time in adherence to standards on identification and summary of evidence from 34.6% prior to 1990 to 36.1% after 1995 (P=.11). There was no difference in the mean number of standards satisfied by guidelines produced by subspecialty medical societies, general medical societies, or government agencies (P=.55). Guideline length was positively correlated with adherence to methodological standards (P=.001).

Conclusion  Guidelines published in the peer-reviewed medical literature during the past decade do not adhere well to established methodological standards. While all areas of guideline development need improvement, greatest improvement is needed in the identification, evaluation, and synthesis of the scientific evidence.

Author Affiliations: Division of General Internal Medicine, Department of Medicine, Beth Israel Deaconess Medical Center, and Department of Medicine, Harvard Medical School, Boston, Mass (Drs Shaneyfelt and Mayo-Smith); and the Ambulatory Care Service, Veterans Affairs Medical Center, Manchester, NH (Drs Shaneyfelt, Mayo-Smith, and Rothwangl). Dr Shaneyfelt is now with the Division of General Internal Medicine, University of Alabama at Birmingham.

RELATED ARTICLES

The Trials and Tribulations of Clinical Practice Guidelines
Deborah Cook and Mita Giacomini
JAMA. 1999;281(20):1950-1951.
EXTRACT | FULL TEXT  

May 26, 1999
JAMA. 1999;281(20):1959-1960.
EXTRACT | FULL TEXT  

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

The management of Clostridium difficile infection: using small-scale audit to indicate the knowledge of nursing and medical staff in an acute hospital setting
Madeo et al.
British Journal of Infection Control 2008;9:12-17.
ABSTRACT  

Neuropathic pain: Redefinition and a grading system for clinical and research purposes
Treede et al.
Neurology 2008;70:1630-1635.
ABSTRACT | FULL TEXT  

Orthopaedic thromboprophylaxis: LIMITATIONS OF CURRENT GUIDELINES
Warwick et al.
J Bone Joint Surg Br 2008;90-B:127-132.
ABSTRACT | FULL TEXT  

Who has time for family medicine?
Pimlott
cfp 2008;54:14-16.
FULL TEXT  

Risk Stratification and Comorbidity
Ferraris et al.
Card Surg Adult 2008;3:199-246.
FULL TEXT  

Out-of-hours primary care: development of indicators for prescribing and referring
Giesen et al.
Int J Qual Health Care 2007;19:289-295.
ABSTRACT | FULL TEXT  

Differences Among International Pharyngitis Guidelines: Not Just Academic
Matthys et al.
Ann Fam Med 2007;5:436-443.
ABSTRACT | FULL TEXT  

Medical malpractice and anesthesiology: literature review and role of the expert witness: [Fautes medicales et anesthesiologie : revue de la litterature et role du temoin expert]
Crosby
Canadian J. Anesthesia 2007;54:227-241.
ABSTRACT | FULL TEXT  

Effectiveness of Drug-Eluting Stents in Patients With Bare-Metal In-Stent Restenosis: Meta-Analysis of Randomized Trials
Dibra et al.
J Am Coll Cardiol 2007;49:616-623.
ABSTRACT | FULL TEXT  

Subfertility guidelines in Europe: the quantity and quality of intrauterine insemination guidelines
Haagen et al.
Hum Reprod 2006;21:2103-2109.
ABSTRACT | FULL TEXT  

Adaptation of clinical guidelines: literature review and proposition for a framework and procedure.
Fervers et al.
Int J Qual Health Care 2006;18:167-176.
ABSTRACT | FULL TEXT  

Clinical Practice Guidelines in nephrology--for worse or for better
Uhlig et al.
Nephrol Dial Transplant 2006;21:1145-1153.
FULL TEXT  

Translation in the health professions: converting science into action.
Sussman et al.
Eval Health Prof 2006;29:7-32.
ABSTRACT  

What the surgeon of tomorrow needs to know about evidence-based surgery.
Maier
Arch Surg 2006;141:317-323.
FULL TEXT  

Assessing and scaling the knowledge pyramid: the good-guideline guide
Davis et al.
CMAJ 2006;174:337-338.
FULL TEXT  

Evidence-based decision making: current challenges and a new feature in the Canadian Journal of Anesthesia/La prise de decision fondee sur les preuves : les defis actuels et une nouvelle chronique dans le Journal canadien d'anesthesie
Cheng and Choi
Canadian J. Anesthesia 2006;53:6-10.
FULL TEXT  

Conflict between Guideline Methodologic Quality and Recommendation Validity: A Potential Problem for Practitioners
Watine et al.
Clin. Chem. 2006;52:65-72.
ABSTRACT | FULL TEXT  

Ottawa Panel Evidence-Based Clinical Practice Guidelines for Therapeutic Exercises and Manual Therapy in the Management of Osteoarthritis
Ottawa Panel Members et al.
ptjournal 2005;85:907-971.
ABSTRACT | FULL TEXT  

Clinical Practice Guidelines and Quality of Care for Older Patients With Multiple Comorbid Diseases: Implications for Pay for Performance
Boyd et al.
JAMA 2005;294:716-724.
ABSTRACT | FULL TEXT  

Methodological Approach: American College of Chest Physicians Guidelines for the Prevention and Management of Postoperative Atrial Fibrillation After Cardiac Surgery
Fleisher et al.
Chest 2005;128:17S-23S.
ABSTRACT | FULL TEXT  

Circumcision for the prevention of urinary tract infection in boys: a systematic review of randomised trials and observational studies
Singh-Grewal et al.
Arch. Dis. Child. 2005;90:853-858.
ABSTRACT | FULL TEXT  

Evaluation and adaptation of clinical practice guidelines
Graham and Harrison
Evid. Based Nurs. 2005;8:68-72.
FULL TEXT  

The use of clinical guidelines for asthma, diabetes, and hypertension in primary health care
Lindberg et al.
Int J Qual Health Care 2005;17:217-220.
ABSTRACT | FULL TEXT  

A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit
Vlayen et al.
Int J Qual Health Care 2005;17:235-242.
ABSTRACT | FULL TEXT  

Quality of Evidence-Based Pediatric Guidelines
Boluyt et al.
Pediatrics 2005;115:1378-1391.
ABSTRACT | FULL TEXT  

The paths from research to improved health outcomes
Glasziou and Haynes
Evid. Based Nurs. 2005;8:36-38.
FULL TEXT  

Commentary: Quality Improvement and Psychiatric Fitness-for-Duty Evaluations of Physicians
Wettstein
J Am Acad Psychiatry Law 2005;33:92-94.
FULL TEXT  

The paths from research to improved health outcomes
Glasziou and Haynes
Evid. Based Med. 2005;10:4-7.
FULL TEXT  

A Comparison of Patients' and Health Care Professionals' Preferences for Symptoms During Immediate Postoperative Recovery and the Management of Postoperative Nausea and Vomiting
Lee et al.
Anesth. Analg. 2005;100:87-93.
ABSTRACT | FULL TEXT  

Improving the quality of health care: using international collaboration to inform guideline programmes by founding the Guidelines International Network (G-I-N)
Ollenschlager et al.
Qual Saf Health Care 2004;13:455-460.
ABSTRACT | FULL TEXT  

Bridging the Guideline Implementation Gap: A Systematic, Document-Centered Approach to Guideline Implementation
Shiffman et al.
J. Am. Med. Inform. Assoc. 2004;11:418-426.
ABSTRACT | FULL TEXT  

Evidence-Based Clinical Practice Guideline for the Prevention of Ventilator-Associated Pneumonia
Dodek et al.
ANN INTERN MED 2004;141:305-313.
ABSTRACT | FULL TEXT  

The Evolving Guideline
Campbell
Clin Med Res 2004;2:145-146.
FULL TEXT  

Categorizing Asthma Severity: An Overview of National Guidelines
Colice
Clin Med Res 2004;2:155-163.
ABSTRACT | FULL TEXT  

International Assessment of the Quality of Clinical Practice Guidelines in Oncology Using the Appraisal of Guidelines and Research and Evaluation Instrument
Burgers et al.
JCO 2004;22:2000-2007.
ABSTRACT | FULL TEXT  

Otitis Media With Effusion
American Academy of Family Physicians et al.
Pediatrics 2004;113:1412-1429.
ABSTRACT | FULL TEXT  

Evidence-Based Practices in Geriatric Mental Health Care
Bartels et al.
Focus 2004;2:268-281.
ABSTRACT | FULL TEXT  

Is the methodological quality of guidelines declining in the US? Comparison of the quality of US Agency for Health Care Policy and Research (AHCPR) guidelines with those published subsequently
Hasenfeld and Shekelle
Qual Saf Health Care 2003;12:428-434.
ABSTRACT | FULL TEXT  

Validity and reliability of guidelines for neck pain treatment in primary health care. A nationwide empirical analysis in Spain
Saturno et al.
Int J Qual Health Care 2003;15:487-493.
ABSTRACT | FULL TEXT  

Atoms
Bauchner
Arch. Dis. Child. 2003;88:1031-1031.
FULL TEXT  

Ethnic specific recommendations in clinical practice guidelines: a first exploratory comparison between guidelines from the USA, Canada, the UK, and the Netherlands
Manna et al.
Qual Saf Health Care 2003;12:353-358.
ABSTRACT | FULL TEXT  

Standardized Reporting of Clinical Practice Guidelines: A Proposal from the Conference on Guideline Standardization
Shiffman et al.
ANN INTERN MED 2003;139:493-498.
ABSTRACT | FULL TEXT  

Mild cognitive impairment: Directions for future research
Luis et al.
Neurology 2003;61:438-444.
ABSTRACT | FULL TEXT  

Do Physicians Not Offer Useful Services Because Of Coverage Restrictions?
Wynia et al.
Health Aff (Millwood) 2003;22:190-197.
ABSTRACT | FULL TEXT  

From evidence-based guideline methodology to quality of care standards
Nathwani
J Antimicrob Chemother 2003;51:1103-1107.
FULL TEXT  

Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project
Qual Saf Health Care 2003;12:18-23.
ABSTRACT | FULL TEXT  

Risk Stratification and Comorbidity
Ferraris and Ferraris
Card Surg Adult 2003;2:187-224.
FULL TEXT  

Assessment of the Scope and Quality of Clinical Practice Guidelines in Lung Cancer
Harpole et al.
Chest 2003;123:7S-20S.
ABSTRACT | FULL TEXT  

Evidence-Based Practices in Geriatric Mental Health Care
Bartels et al.
Psychiatr. Serv. 2002;53:1419-1431.
ABSTRACT | FULL TEXT  

AAN clinical practice guidelines: Above the fray
Franklin and Zahn
Neurology 2002;59:975-976.
FULL TEXT  

An assessment of guidelines for prevention of ischemic stroke
Hart and Bailey
Neurology 2002;59:977-982.
ABSTRACT | FULL TEXT  

Critical appraisal of the Minimal Clinical Recommendations (MCR) of the European Society for Medical Oncology (ESMO): challenges for a European framework for the development of clinical practice guidelines
Fervers et al.
Ann Oncol 2002;13:1507-1509.
FULL TEXT  

Current and future relevance of guidelines
Klein
Heart 2002;87:497-500.
FULL TEXT  

Improving the Quality of Medical Care: Building Bridges Among Professional Pride, Payer Profit, and Patient Satisfaction
Grol
JAMA 2001;286:2578-2585.
ABSTRACT | FULL TEXT  

Building Bridges to Quality
Shaneyfelt
JAMA 2001;286:2600-2601.
FULL TEXT  

Clinical Decision Support Systems for the Practice of Evidence-based Medicine
Sim et al.
J. Am. Med. Inform. Assoc. 2001;8:527-534.
ABSTRACT | FULL TEXT  

Evidence-Based Guidelines: Application to Clinical Practice
Scalzitti
ptjournal 2001;81:1622-1628.
FULL TEXT  

Quality Improvement Report: Linking guideline to regular feedback to increase appropriate requests for clinical tests: blood gas analysis in intensive care
Merlani et al.
BMJ 2001;323:620-624.
ABSTRACT | FULL TEXT  

Practice Guidelines: Useful and "Participative" Method?: Survey of Italian Physicians by Professional Setting
Formoso et al.
Arch Intern Med 2001;161:2037-2042.
ABSTRACT | FULL TEXT  

What is the quality of drug therapy clinical practice guidelines in Canada?
Graham et al.
CMAJ 2001;165:157-163.
ABSTRACT | FULL TEXT  

Proof and Policy from Medical Research Evidence
Mulrow and Lohr
Journal of Health Politics, Policy and Law 2001;26:249-266.
ABSTRACT  

The management of hypertension in Canada: a review of current guidelines, their shortcomings and implications for the future
McAlister et al.
CMAJ 2001;164:517-522.
ABSTRACT | FULL TEXT  

Critical Appraisal of Clinical Practice Guidelines Targeting Chronic Obstructive Pulmonary Disease
Lacasse et al.
Arch Intern Med 2001;161:69-74.
ABSTRACT | FULL TEXT  

Systematic Review of Clinical Prediction Rules for Neuroimaging in the Evaluation of Dementia
Gifford et al.
Arch Intern Med 2000;160:2855-2862.
ABSTRACT | FULL TEXT  

Users' Guides to the Medical Literature: XXV. Evidence-Based Medicine: Principles for Applying the Users' Guides to Patient Care
Guyatt et al.
JAMA 2000;284:1290-1296.
ABSTRACT | FULL TEXT  

GEM: A Proposal for a More Comprehensive Guideline Document Model Using XML
Shiffman et al.
J. Am. Med. Inform. Assoc. 2000;7:488-498.
ABSTRACT | FULL TEXT  

Attitudes of Israeli Family Physicians Toward Clinical Guidelines
Vinker et al.
Arch Fam Med 2000;9:835-840.
ABSTRACT | FULL TEXT  

Improving the quality and dissemination of guidelines: the quest for the Holy Grail
Bassand
Eur Heart J 2000;21:1289-1290.
 

The Overarching Challenge
Heffner
Chest 2000;118:1S-3S.
FULL TEXT  

Translating Guidelines Into Clinical Practice : Recommendations to the American College of Chest Physicians
Heffner et al.
Chest 2000;118:70S-73S.
FULL TEXT  

Diagnosing syncope in clinical practice. Implementation of a simplified diagnostic algorithm in a multicentre prospective trial--the OESIL 2 Study (Osservatorio Epidemiologico della Sincope nel Lazio)
Ammirati et al.
Eur Heart J 2000;21:935-940.
ABSTRACT  

Stroke prevention: Narrowing the evidence-practice gap
Holloway et al.
Neurology 2000;54:1899-1906.
ABSTRACT | FULL TEXT  

Milliman and Robertson---Going in the Wrong Direction
Bauchner et al.
Pediatrics 2000;105:858-859.
FULL TEXT  

Clinical Evidence: A Compendium of the Best Available Evidence for Effective Health Care
Satya-Murti
JAMA 2000;283:1352-1353.
FULL TEXT  

Are Guidelines Following Guidelines?
Rivara
AAP Grand Rounds 1999;2:15-15.
FULL TEXT  

The Trials and Tribulations of Clinical Practice Guidelines
Cook and Giacomini
JAMA 1999;281:1950-1951.
FULL TEXT