This Article  • References  • Full text PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Contact me when this article is cited  Related Content  • Similar articles in JAMA  Social Bookmarking   What's this?

A Review and Suggested Modifications of Methodological Standards

Andreas Laupacis, MD; Nandita Sekar, MD; lan G. Stiell, MD

JAMA. 1997;277(6):488-494.

Abstract
Background.
—Clinical prediction rules are decision-making tools for clinicians, containing variables from the history, physical examination, or simple diagnostic tests.

Objective.
—To review the quality of recently published clinical prediction rules and to suggest methodological standards for their development and evaluation.

Data Sources.
—Four general medical journals were manually searched for clinical prediction rules published from 1991 through 1994.

Study Selection.
—Four hundred sixty potentially eligible reports were identified, of which 30 were clinical prediction rules eligible for study. Most methodological standards could only be evaluated in 29 studies.

Data Abstraction.
—Two investigators independently evaluated the quality of each report using a standard data sheet. Disagreements were resolved by consensus.

Data Synthesis.
—The mathematical technique was used to develop the rule, and the results of the rule were described in 100% (29/29) of the reports. All the rules but 1 (97% [28/29]) were felt to be clinically sensible. The outcomes and predictive variables were clearly defined in 83% (24/29) and 59% (17/29) of the reports, respectively. Blind assessment of outcomes and predictive variables occurred in 41% (12/29) and 79% (23/29) of the reports, respectively, and the rules were prospectively validated in 79% (11/14). Reproducibility of predictive variables was assessed in only 3% (1/29) of the reports, and the effect of the rule on clinical use was prospectively measured in only 3% (1/30). Forty-one percent (12/29) of the rules were felt to be easy to use.

Conclusions.
—Although clinical prediction rules comply with some methodological criteria, for other criteria, better compliance is needed.

Author Affiliations
From the Departments of Medicine (Drs Laupacis, Sekar, and Stiell) and Epidemiology and Community Medicine (Drs Laupacis and Stiell), University of Ottawa, and the Clinical Epidemiology Unit, Loeb Medical Research Institute, Ottawa Civic Hospital (Drs Laupacis and Stiell), Ottawa, Ontario.

Footnotes
Reprints not available from the authors.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Decision trees for indication of total hip replacement on patients with osteoarthritis
Quintana et al.
Rheumatology (Oxford) 2009;48:1402-1409.
ABSTRACT | FULL TEXT  

Prediction rules in cervical spine injury
Podichetty and Morisue
BMJ 2009;339:b4139-b4139.
FULL TEXT  

Implementation of the Canadian C-Spine Rule: prospective 12 centre cluster randomised trial
Stiell et al.
BMJ 2009;339:b4146-b4146.
ABSTRACT | FULL TEXT  

Sociodemographic and Clinical Characteristics Are Not Clinically Useful Predictors of Refill Adherence in Patients With Hypertension
Steiner et al.
Circ Cardiovasc Qual Outcomes 2009;2:451-457.
ABSTRACT | FULL TEXT  

Clinical prediction rules
Falk and Fahey
BMJ 2009;339:b2899-b2899.
FULL TEXT  

A Clinical Criterion to Exclude the Hyperimmunoglobulin D Syndrome (Mild Mevalonate Kinase Deficiency) in Patients with Recurrent Fever
STEICHEN et al.
The Journal of Rheumatology 2009;36:1677-1681.
ABSTRACT | FULL TEXT  

Should a Head-Injured Child Receive a Head CT Scan? A Systematic Review of Clinical Prediction Rules
Maguire et al.
Pediatrics 2009;124:e145-e154.
ABSTRACT | FULL TEXT  

A Guide to Interpretation of Studies Investigating Subgroups of Responders to Physical Therapy Interventions
Hancock et al.
ptjournal 2009;89:698-704.
ABSTRACT | FULL TEXT  

Prognosis and prognostic research: application and impact of prognostic models in clinical practice
Moons et al.
BMJ 2009;338:b606-b606.
FULL TEXT  

Response to Letter by Prabhakaran and Lee
Arsava and Ay
Stroke 2009;40:e409-e409.
FULL TEXT  

Prognosis and prognostic research: what, why, and how?
Moons et al.
BMJ 2009;338:b375-b375.
FULL TEXT  

Clinical Prediction Rules for Physical Therapy Interventions: A Systematic Review
Beneciuk et al.
ptjournal 2009;89:114-124.
ABSTRACT | FULL TEXT  

Derivation and Validation of a Preoperative Prediction Rule for Delirium After Cardiac Surgery
Rudolph et al.
Circulation 2009;119:229-236.
ABSTRACT | FULL TEXT  

Simplification of the Revised Geneva Score for Assessing Clinical Probability of Pulmonary Embolism
Klok et al.
Arch Intern Med 2008;168:2131-2136.
ABSTRACT | FULL TEXT  

Identifying unprovoked thromboembolism patients at low risk for recurrence who can discontinue anticoagulant therapy
Rodger et al.
CMAJ 2008;179:417-426.
ABSTRACT | FULL TEXT  

Prognostic models
Rothwell
PN 2008;8:242-253.
ABSTRACT | FULL TEXT  

Comparison of 12 Risk Stratification Schemes to Predict Stroke in Patients With Nonvalvular Atrial Fibrillation
Stroke Risk in Atrial Fibrillation Working Group
Stroke 2008;39:1901-1910.
ABSTRACT | FULL TEXT  

Systematic Review: Gene Expression Profiling Assays in Early-Stage Breast Cancer
Marchionni et al.
ANN INTERN MED 2008;148:358-369.
ABSTRACT | FULL TEXT  

Severe paediatric ulcerative colitis: incidence, outcomes and optimal timing for second-line therapy
Turner et al.
Gut 2008;57:331-338.
ABSTRACT | FULL TEXT  

Diagnosis of left ventricular systolic dysfunction (LVSD): development and validation of a clinical prediction rule in primary care
Fahey et al.
Fam Pract 2007;24:628-635.
ABSTRACT | FULL TEXT  

Validation of a Decision Aid to Assist Physicians in Reducing Unnecessary Antibiotic Drug Use for Acute Cystitis
McIsaac et al.
Arch Intern Med 2007;167:2201-2206.
ABSTRACT | FULL TEXT  

Development and Internal Validation of the Male Osteoporosis Risk Estimation Score
Shepherd et al.
Ann Fam Med 2007;5:540-546.
ABSTRACT | FULL TEXT  

Risk Factors for Delirium at Discharge: Development and Validation of a Predictive Model
Inouye et al.
Arch Intern Med 2007;167:1406-1413.
ABSTRACT | FULL TEXT  

Review of the Acuity Scoring Systems for the Pediatric Intensive Care Unit and Their Use in Quality Improvement
Marcin and Pollack
J Intensive Care Med 2007;22:131-140.
ABSTRACT  

Derivation and Validation of a Simplified Predictive Index for Renal Replacement Therapy After Cardiac Surgery
Wijeysundera et al.
JAMA 2007;297:1801-1809.
ABSTRACT | FULL TEXT  

Linking Surveillance to Action: Incorporation of Real-time Regional Data into a Medical Decision Rule
Fine et al.
J. Am. Med. Inform. Assoc. 2007;14:206-211.
ABSTRACT | FULL TEXT  

Predicting foot ulcers in patients with diabetes: a systematic review and meta-analysis
Crawford et al.
QJM 2007;100:65-86.
ABSTRACT | FULL TEXT  

A Clinical Model To Estimate the Pretest Probability of Lung Cancer in Patients With Solitary Pulmonary Nodules
Gould et al.
Chest 2007;131:383-388.
ABSTRACT | FULL TEXT  

Predischarge Risk Assessment for Severe Neonatal Hyperbilirubinemia
Keren and Bhutani
NeoReviews 2007;8:e68-e76.
ABSTRACT | FULL TEXT  

Clinical Prediction Rule for Identifying Children With Cerebrospinal Fluid Pleocytosis at Very Low Risk of Bacterial Meningitis
Nigrovic et al.
JAMA 2007;297:52-60.
ABSTRACT | FULL TEXT  

Development of a Clinical Prediction Rule for Guiding Treatment of a Subgroup of Patients With Neck Pain: Use of Thoracic Spine Manipulation, Exercise, and Patient Education
Cleland et al.
ptjournal 2007;87:9-23.
ABSTRACT | FULL TEXT  

Their last 6 months: suffering and survival of end-stage dementia patients
Aminoff and Adunsky
Age Ageing 2006;35:597-601.
ABSTRACT | FULL TEXT  

Validation of a rule for termination of resuscitation in out-of-hospital cardiac arrest.
Morrison et al.
NEJM 2006;355:478-487.
ABSTRACT | FULL TEXT  

Prognosis in Soft Tissue Disorders of the Shoulder: Predicting Both Change in Disability and Level of Disability After Treatment
Kennedy et al.
ptjournal 2006;86:1013-1032.
ABSTRACT | FULL TEXT  

Guidelines for Developing and Testing Clinical Decision Rules
Shapiro
West J Nurs Res 2006;28:244-253.
ABSTRACT  

Prediction of vesicoureteral reflux after a first febrile urinary tract infection in children: validation of a clinical decision rule
Leroy et al.
Arch. Dis. Child. 2006;91:241-244.
ABSTRACT | FULL TEXT  

Translating Clinical Research into Clinical Practice: Impact of Using Prediction Rules To Make Decisions
Reilly and Evans
ANN INTERN MED 2006;144:201-209.
ABSTRACT | FULL TEXT  

Better Care for Patients with Suspected Pulmonary Embolism
Sox
ANN INTERN MED 2006;144:210-212.
FULL TEXT  

Development and Application of Clinical Prediction Rules to Improve Decision Making in Physical Therapist Practice
Childs and Cleland
ptjournal 2006;86:122-131.
FULL TEXT  

Are the Ottawa ankle rules helpful in ruling out the need for x ray examination in children?
Myers et al.
Arch. Dis. Child. 2005;90:1309-1311.
FULL TEXT  

Comparison of the Canadian CT Head Rule and the New Orleans Criteria in Patients With Minor Head Injury
Stiell et al.
JAMA 2005;294:1511-1518.
ABSTRACT | FULL TEXT  

Clinical Predictors of Major Infections After Cardiac Surgery
Fowler et al.
Circulation 2005;112:I-358-I-365.
ABSTRACT | FULL TEXT  

Evaluating Clinical Decision Rules
Shapiro
West J Nurs Res 2005;27:655-664.
ABSTRACT  

Use of a Clinical Prediction Score in Patients with Suspected Deep Venous Thrombosis: Two Steps Forward, One Step Back?
Douketis
ANN INTERN MED 2005;143:140-142.
FULL TEXT  

A clinical return-to-work rule for patients with back pain
Dionne et al.
CMAJ 2005;172:1559-1567.
ABSTRACT | FULL TEXT  

Applicability and relevance of models that predict short term outcome after intracerebral haemorrhage
Ariesen et al.
J. Neurol. Neurosurg. Psychiatry 2005;76:839-844.
ABSTRACT | FULL TEXT  

Predictors of good outcome in medium to large spontaneous supratentorial intracerebral haemorrhages
Castellanos et al.
J. Neurol. Neurosurg. Psychiatry 2005;76:691-695.
ABSTRACT | FULL TEXT  

Clinical Prediction Rules in Trauma Imaging: Who, How, and Why?
Blackmore
Radiology 2005;235:371-374.
ABSTRACT | FULL TEXT  

Use of Clinical Prediction Rules in Detecting Osteoporosis in a Population-Based Sample of Postmenopausal Women
Mauck et al.
Arch Intern Med 2005;165:530-536.
ABSTRACT | FULL TEXT  

Idiopathic Pulmonary Fibrosis: Challenges and Opportunities for the Clinician and Investigator
Swigris et al.
Chest 2005;127:275-283.
ABSTRACT | FULL TEXT  

Prediction of maintenance of sinus rhythm after electrical cardioversion of atrial fibrillation by non-deterministic modelling
Zohar et al.
Europace 2005;7:500-507.
ABSTRACT | FULL TEXT  

A Clinical Prediction Rule To Identify Patients with Low Back Pain Most Likely To Benefit from Spinal Manipulation: A Validation Study
Childs et al.
ANN INTERN MED 2004;141:920-928.
ABSTRACT | FULL TEXT  

Clinical Decision Rules as Tools for Evidence-Based Nursing
Shapiro and Driever
West J Nurs Res 2004;26:930-937.
ABSTRACT  

Is radiologic examination necessary for a 9-year-old girl with a knee injury?
Smith and Cleland
ptjournal 2004;84:1092-1100.
FULL TEXT  

The Mortality Index for Neonatal Transportation Score: A New Mortality Prediction Model for Retrieved Neonates
Broughton et al.
Pediatrics 2004;114:e424-e428.
ABSTRACT | FULL TEXT  

Accurate Clinical Prediction of Severe Acute Respiratory Syndrome: Are We There Yet?
Jernigan et al.
ANN INTERN MED 2004;141:396-398.
FULL TEXT  

Diagnosis of Lower-Extremity Deep Vein Thrombosis in Outpatients
Riddle and Wells
ptjournal 2004;84:729-735.
FULL TEXT  

Estimating Prognosis for Nursing Home Residents With Advanced Dementia
Mitchell et al.
JAMA 2004;291:2734-2740.
ABSTRACT | FULL TEXT  

Prediction of oesophageal varices with platelet count/spleen diameter ratio or platelets alone * Author's reply
Thabut et al.
Gut 2004;53:913-915.
FULL TEXT  

Association Between Evening Admissions and Higher Mortality Rates in the Pediatric Intensive Care Unit
Arias et al.
Pediatrics 2004;113:e530-e534.
ABSTRACT | FULL TEXT  

Decision Guidelines for Prophylactic Replacement of Bjork-Shiley Convexo-Concave Heart Valves: Impact on Clinical Practice
van Gorp et al.
Circulation 2004;109:2092-2096.
ABSTRACT | FULL TEXT  

The Canadian C-Spine Rule
Mower et al.
NEJM 2004;350:1467-1469.
FULL TEXT  

Prospective validation of a proposal for diagnosis and management of patients attending the emergency department for mild head injury
Fabbri et al.
J. Neurol. Neurosurg. Psychiatry 2004;75:410-416.
ABSTRACT | FULL TEXT  

Test Research versus Diagnostic Research
Moons et al.
Clin. Chem. 2004;50:473-476.
FULL TEXT  

Can We Predict Which Children With Clinically Suspected Pneumonia Will Have the Presence of Focal Infiltrates on Chest Radiographs?
Lynch et al.
Pediatrics 2004;113:e186-189.
ABSTRACT | FULL TEXT  

Identification of Individuals With Patellofemoral Pain Whose Symptoms Improved After a Combined Program of Foot Orthosis Use and Modified Activity: A Preliminary Investigation
Sutlive et al.
ptjournal 2004;84:49-61.
ABSTRACT | FULL TEXT  

The Canadian C-Spine Rule versus the NEXUS Low-Risk Criteria in Patients with Trauma
Stiell et al.
NEJM 2003;349:2510-2518.
ABSTRACT | FULL TEXT  

Predicting Mortality Among Patients Hospitalized for Heart Failure: Derivation and Validation of a Clinical Model
Lee et al.
JAMA 2003;290:2581-2587.
ABSTRACT | FULL TEXT  

Reporting of Predictive Logistic Models Should Be Based on Evidence-Based Guidelines
Hernandez et al.
Chest 2003;124:2034-2035.
FULL TEXT  

Clinical Diagnosis of Hypersensitivity Pneumonitis
Lacasse et al.
Am. J. Respir. Crit. Care Med. 2003;168:952-958.
ABSTRACT | FULL TEXT  

Platelet count/spleen diameter ratio: proposal and validation of a non-invasive parameter to predict the presence of oesophageal varices in patients with liver cirrhosis
Giannini et al.
Gut 2003;52:1200-1205.
ABSTRACT | FULL TEXT  

Validity of a Simple ST-Elevation Acute Myocardial Infarction Risk Index: Are Randomized Trial Prognostic Estimates Generalizable to Elderly Patients? * Response
Das et al.
Circulation 2003;108 :e9-e10.
FULL TEXT  

Assessment of Volume of Hemorrhage and Outcome From Pelvic Fracture
Blackmore et al.
Arch Surg 2003;138:504-509.
ABSTRACT | FULL TEXT  

A Clinical Prediction Rule to Identify Patients With Atrial Fibrillation and a Low Risk for Stroke While Taking Aspirin
van Walraven et al.
Arch Intern Med 2003;163:936-943.
ABSTRACT | FULL TEXT  

Reliability of the variables in a new set of models that predict outcome after stroke
Weir et al.
J. Neurol. Neurosurg. Psychiatry 2003;74:447-451.
ABSTRACT | FULL TEXT  

Development and validation of a simple risk score to predict the need for permanent pacing after cardiac valve surgery
Koplan et al.
J Am Coll Cardiol 2003;41:795-801.
ABSTRACT | FULL TEXT  

Accuracy of the Preoperative Assessment in Predicting Pulmonary Risk after Nonthoracic Surgery
McAlister et al.
Am. J. Respir. Crit. Care Med. 2003;167:741-744.
ABSTRACT | FULL TEXT  

Validation and Modification of Streptococcal Pharyngitis Clinical Prediction Rules
McGinn et al.
Mayo Clin Proc. 2003;78:289-293.
ABSTRACT  

Validity of a Simple ST-Elevation Acute Myocardial Infarction Risk Index: Are Randomized Trial Prognostic Estimates Generalizable to Elderly Patients?
Rathore et al.
Circulation 2003;107:811-816.
ABSTRACT | FULL TEXT  

Effect of Arthroscopic Débridement for Osteoarthritis of the Knee on Health-Related Quality of Life
Dervin et al.
JBJS 2003;85:10-19.
ABSTRACT | FULL TEXT  

A comparison of three scoring systems for mortality risk among retrieved intensive care patients
Tibby et al.
Arch. Dis. Child. 2002;87:421-425.
ABSTRACT | FULL TEXT  

Development and Validation of a Multivariable Predictive Model to Distinguish Bacterial From Aseptic Meningitis in Children in the Post-Haemophilus influenzae Era
Nigrovic et al.
Pediatrics 2002;110:712-719.
ABSTRACT | FULL TEXT  

Practice corner: using clinical prediction rules
McGinn et al.
Evid. Based Med. 2002;7:132-134.
FULL TEXT  

Validation of a clinical prediction rule to reduce preoperative type and screen procedures{dagger}{dagger}{dagger}
van Klei et al.
Br J Anaesth 2002;89:221-225.
ABSTRACT | FULL TEXT  

Diagnostic studies as multivariable, prediction research
Moons and Grobbee
J. Epidemiol. Community Health 2002;56:337-338.
FULL TEXT  

Predicting Outcome After Acute and Subacute Stroke: Development and Validation of New Prognostic Models
Counsell et al.
Stroke 2002;33:1041-1047.
ABSTRACT | FULL TEXT  

Decisions About Cervical Spine Radiography Following Trauma
Lynch et al.
JAMA 2002;287:583-584.
FULL TEXT  

Response to first drug trial predicts outcome in childhood temporal lobe epilepsy
Dlugos et al.
Neurology 2001;57:2259-2264.
ABSTRACT | FULL TEXT  

What White Blood Cell Count Should Prompt Antibiotic Treatment in a Febrile Child? Tutorial on the Importance of Disease Likelihood to the Interpretation of Diagnostic Tests
Kohn and Newman
Med Decis Making 2001;21:479-489.
ABSTRACT  

Identification of poor prognostic features among patients requiring mechanical ventilation after hematopoietic stem cell transplantation
Bach et al.
Blood 2001;98:3234-3240.
ABSTRACT | FULL TEXT  

The Canadian C-Spine Rule for Radiography in Alert and Stable Trauma Patients
Stiell et al.
JAMA 2001;286:1841-1848.
ABSTRACT | FULL TEXT  

The Early Identification of Candidates for Epilepsy Surgery
Dlugos
Arch Neurol 2001;58:1543-1546.
ABSTRACT | FULL TEXT  

Developing a Prediction Rule From Automated Clinical Databases to Identify High-Risk Patients in a Large Population With Diabetes
Selby et al.
Diabetes Care 2001;24:1547-1555.
ABSTRACT | FULL TEXT  

Predictors of good outcome after intravenous tPA for acute ischemic stroke
Demchuk et al.
Neurology 2001;57:474-480.
ABSTRACT | FULL TEXT  

Evaluation of Decision Rules for Referring Women for Bone Densitometry by Dual-Energy X-ray Absorptiometry
Cadarette et al.
JAMA 2001;286:57-63.
ABSTRACT | FULL TEXT  

Searching for Clinical Prediction Rules in MEDLINE
Ingui and Rogers
J. Am. Med. Inform. Assoc. 2001;8:391-397.
ABSTRACT | FULL TEXT