This Article  • References  • Full text PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Contact me when this article is cited  Related Content  • Similar articles in JAMA  Social Bookmarking   What's this?

Getting Better but Still Not Good

M. Carrington Reid, PhD, MD; Mark S. Lachs, MD, MPH; Alvan R. Feinstein, MD

JAMA. 1995;274(8):645-651.

Abstract
Objective.
—To determine the frequency and temporal changes in application of seven accepted methodological standards for the evaluation of diagnostic tests.

Data Sources.
—A search of the MEDLINE database yielded 1302 articles about diagnostic test studies, during a 16-year secular interval, 1978 through 1993, in four prominent general medical journals.

Study Selection.
—In the 112 eligible studies, the test was intended for clinical use, indexes of accuracy (sensitivity and specificity or likelihood ratios) were provided, and more than 10 patients were enrolled.

Data Extraction.
—Although each study was critically reviewed by one primary observer, a subset was independently evaluated for interrater consistency.

Data Synthesis.
—The percentage of studies that fulfilled criteria for each of the seven methodological standards are as follows: (1) specify spectrum of evaluated patients, 27%; (2) report test indexes for clinical subgroups, 8%; (3) avoid workup bias, 46%; (4) avoid review bias, 38%; (5) provide numerical precision for test indexes, 11%; (6) report frequency and management of indeterminate results when calculating test indexes, 22%; and (7) specify test reproducibility, 23%. Secular increases were found for six of the seven standards in ranges of use from 14% to 31% during 1978-1981 to 1990-1993. Nevertheless, only one standard, avoidance of workup bias, was fulfilled by more than 50% of studies in the most recent secular interval.

Conclusions.
—These results indicate that most diagnostic tests are still inadequately appraised. The routine demand for methodological standards could raise the quality of diagnostic test information, and the careful predissemination evaluation of diagnostic tests could eliminate useless tests before they receive widespread application.

(JAMA. 1995;274:645-651)

Author Affiliations
From the Robert Wood Johnson Clinical Scholars Program (Dr Reid) and Departments of Internal Medicine and Epidemiology (Dr Feinstein), Yale University School of Medicine, New Haven, Conn; and the Division of General Internal Medicine, The New York (NY) Hospital—Cornell University School of Medicine (Dr Lachs).

Footnotes
Reprint requests to the Robert Wood Johnson Clinical Scholars Program, Yale University School of Medicine, 333 Cedar St, New Haven, CT 06520-8025 (Dr Feinstein).

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

The diagnostic value of history and physical examination for COPD in suspected or known cases: a systematic review
Broekhuizen et al.
Fam Pract 2009;26:260-268.
ABSTRACT | FULL TEXT  

Overinterpretation of Clinical Applicability in Molecular Diagnostic Research
Lumbreras et al.
Clin. Chem. 2009;55:786-794.
ABSTRACT | FULL TEXT  

STrengthening the REporting of Genetic Association Studies (STREGA): An Extension of the STROBE Statement
Little et al.
ANN INTERN MED 2009;150:206-215.
ABSTRACT | FULL TEXT  

Role of PET in the Initial Staging of Cutaneous Malignant Melanoma: Systematic Review
Krug et al.
Radiology 2008;249:836-844.
ABSTRACT | FULL TEXT  

STARD Statement: Still Room for Improvement in the Reporting of Diagnostic Accuracy Studies
Bossuyt
Radiology 2008;248:713-714.
FULL TEXT  

Quality of Reporting of Diagnostic Accuracy Studies: No Change Since STARD Statement Publication--Before-and-after Study
Wilczynski
Radiology 2008;248:817-823.
ABSTRACT | FULL TEXT  

Performance measures of the illiterate E-chart vision-screening test used in Northern District Israeli school children
Ore et al.
J Med Screen 2008;15:65-71.
ABSTRACT | FULL TEXT  

Developing a Sensitive Search Strategy in MEDLINE to Retrieve Studies on Assessment of the Diagnostic Performance of Imaging Techniques
Astin et al.
Radiology 2008;247:365-373.
ABSTRACT | FULL TEXT  

Physical examination tests of the shoulder: a systematic review with meta-analysis of individual tests
Hegedus et al.
Br. J. Sports. Med. 2008;42:80-92.
ABSTRACT | FULL TEXT  

The evaluation of genetic tests
Zimmern and Kroese
J Public Health (Oxf) 2007;29:246-250.
ABSTRACT | FULL TEXT  

Are you reading what we are reading? The effect of who interprets medical images on estimates of diagnostic test accuracy in systematic reviews
Brealey and Westwood
Br. J. Radiol. 2007;80:674-677.
ABSTRACT | FULL TEXT  

Technology Assessment in Radiology: Putting the Evidence in Evidence-based Radiology
Hollingworth and Jarvik
Radiology 2007;244:31-38.
ABSTRACT | FULL TEXT  

Sample size in studies on diagnostic accuracy in ophthalmology: a literature survey
Bochmann et al.
Br J Ophthalmol 2007;91:898-900.
ABSTRACT | FULL TEXT  

Evidence-based Practice in Radiology: Steps 3 and 4--Appraise and Apply Systematic Reviews and Meta-Analyses
Halligan and Altman
Radiology 2007;243:13-27.
ABSTRACT | FULL TEXT  

Clinical Depression Versus Distress Among Patients With Type 2 Diabetes: Not just a question of semantics
Fisher et al.
Diabetes Care 2007;30:542-548.
ABSTRACT | FULL TEXT  

Prognostic Value of Heart Rate Increase at Onset of Exercise Testing
Leeper et al.
Circulation 2007;115:468-474.
ABSTRACT | FULL TEXT  

Developmental Dysplasia of the Hip: Quality of Reporting of Diagnostic Accuracy for US
Roposch et al.
Radiology 2006;241:854-860.
ABSTRACT | FULL TEXT  

A Gray Zone Assigned to Inconclusive Results of Quantitative Diagnostic Tests: Application to the Use of Brain Natriuretic Peptide for Diagnosis of Heart Failure in Acute Dyspneic Patients
Coste et al.
Clin. Chem. 2006;52:2229-2235.
ABSTRACT | FULL TEXT  

The Value of Quantitative Electroencephalography in Clinical Psychiatry: A Report by the Committee on Research of the American Neuropsychiatric Association
Coburn et al.
J. Neuropsychiatry Clin. Neurosi. 2006;18:460-500.
ABSTRACT | FULL TEXT  

Microscopic-Observation Drug-Susceptibility Assay for the Diagnosis of TB.
Moore et al.
NEJM 2006;355:1539-1550.
ABSTRACT | FULL TEXT  

Systematic reviews of diagnostic tests in cancer: review of methods and reporting
Mallett et al.
BMJ 2006;333:413.
ABSTRACT | FULL TEXT  

Role of Sonography in the Diagnosis of Axillary Lymph Node Metastases in Breast Cancer: A Systematic Review.
Alvarez et al.
Am. J. Roentgenol. 2006;186:1342-1348.
ABSTRACT | FULL TEXT  

Bias in Research Studies
Sica
Radiology 2006;238:780-789.
ABSTRACT | FULL TEXT  

Evidence of bias and variation in diagnostic accuracy studies.
Rutjes et al.
CMAJ 2006;174:469-476.
ABSTRACT | FULL TEXT  

Noninvasive Screening for Coronary Artery Disease With Computed Tomography Is Useful
Clouse et al.
Circulation 2006;113:125-146.
FULL TEXT  

Development and Application of Clinical Prediction Rules to Improve Decision Making in Physical Therapist Practice
Childs and Cleland
ptjournal 2006;86:122-131.
FULL TEXT  

Accuracy of Helical Computed Tomography and [18F] Fluorodeoxyglucose Positron Emission Tomography for Identifying Lymph Node Mediastinal Metastases in Potentially Resectable Non-Small-Cell Lung Cancer
Pozo-Rodriguez et al.
JCO 2005;23:8348-8356.
ABSTRACT | FULL TEXT  

Heart Rate Characteristics: Novel Physiomarkers to Predict Neonatal Infection and Death
Griffin et al.
Pediatrics 2005;116:1070-1074.
ABSTRACT | FULL TEXT  

Use of Protein:Creatinine Ratio Measurements on Random Urine Samples for Prediction of Significant Proteinuria: A Systematic Review
Price et al.
Clin. Chem. 2005;51:1577-1586.
ABSTRACT | FULL TEXT  

Role of US in Detection of Crohn Disease: Meta-Analysis
Fraquelli et al.
Radiology 2005;236:95-101.
ABSTRACT | FULL TEXT  

Association between Compliance with Methodological Standards of Diagnostic Research and Reported Test Accuracy: Meta-Analysis of Focused Assessment of US for Trauma
Stengel et al.
Radiology 2005;236:102-111.
ABSTRACT | FULL TEXT  

Challenges in Systematic Reviews of Diagnostic Technologies
Tatsioni et al.
ANN INTERN MED 2005;142:1048-1055.
ABSTRACT | FULL TEXT  

Quality of Reporting of Diagnostic Accuracy Studies
Smidt et al.
Radiology 2005;235:347-353.
ABSTRACT | FULL TEXT  

Meta-Analysis: Methods for Diagnosing Intravascular Device-Related Bloodstream Infection
Safdar et al.
ANN INTERN MED 2005;142:451-466.
ABSTRACT | FULL TEXT  

Evidence about evidence
Reeves
Br J Ophthalmol 2005;89:253-254.
FULL TEXT  

The quality of reporting of diagnostic accuracy studies published in ophthalmic journals
Siddiqui et al.
Br J Ophthalmol 2005;89:261-265.
ABSTRACT | FULL TEXT  

Clinical Evaluation of Diagnostic Tests
Weinstein et al.
Am. J. Roentgenol. 2005;184:14-19.
FULL TEXT  

Systematic Review of Computed Tomographic Angiography for Assessment of Carotid Artery Disease
Koelemay et al.
Stroke 2004;35:2306-2312.
ABSTRACT | FULL TEXT  

Caveat emptor: The treachery of work-up bias
Blackstone and Lauer
J. Thorac. Cardiovasc. Surg. 2004;128:341-344.
FULL TEXT  

Health-Care Costs and Exercise Capacity
Weiss et al.
Chest 2004;126:608-613.
ABSTRACT | FULL TEXT  

ROC Curves in Clinical Chemistry: Uses, Misuses, and Possible Solutions
Obuchowski et al.
Clin. Chem. 2004;50:1118-1125.
ABSTRACT | FULL TEXT  

From Diagnostic Accuracy to Accurate Diagnosis: Interpreting a Test Result with Confidence
Zou
Med Decis Making 2004;24:313-318.
ABSTRACT  

Effect of Age and End Point on the Prognostic Value of the Exercise Test
Yamazaki et al.
Chest 2004;125:1920-1928.
ABSTRACT | FULL TEXT  

Evidence-Based Guidelines in Laboratory Medicine: Principles and Methods
Oosterhuis et al.
Clin. Chem. 2004;50:806-818.
ABSTRACT | FULL TEXT  

Diagnostic value of high-resolution sonography in ulnar neuropathy at the elbow
Beekman et al.
Neurology 2004;62:767-773.
ABSTRACT | FULL TEXT  

The Quality of Reporting in Diagnostic Test Research: Getting Better, Still Not Optimal
Bossuyt
Clin. Chem. 2004;50:465-466.
FULL TEXT  

Test Research versus Diagnostic Research
Moons et al.
Clin. Chem. 2004;50:473-476.
FULL TEXT  

Methodology in Diagnostic Laboratory Test Research in Clinical Chemistry and Clinical Chemistry and Laboratory Medicine
Lumbreras-Lacarra et al.
Clin. Chem. 2004;50:530-536.
ABSTRACT | FULL TEXT  

Treadmill scores in elderly men
Lai et al.
J Am Coll Cardiol 2004;43:606-615.
ABSTRACT | FULL TEXT  

Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative
Bossuyt et al.
Fam Pract 2004;21:4-10.
ABSTRACT | FULL TEXT  

The Accuracy of the Ottawa Knee Rule To Rule Out Knee Fractures: A Systematic Review
Bachmann et al.
ANN INTERN MED 2004;140:121-124.
ABSTRACT | FULL TEXT  

Best Threshold for Diagnosis of Stenosis or Thrombosis within Six Months of Access Flow Measurement in Arteriovenous Fistulae
Tonelli et al.
J. Am. Soc. Nephrol. 2003;14:3264-3269.
ABSTRACT | FULL TEXT  

Retrospective analysis of evidence base for tests used in diagnosis and monitoring of disease in respiratory medicine
Borrill et al.
BMJ 2003;327:1136-1138.
ABSTRACT | FULL TEXT  

The Effect of Ethnicity on Survival in Male Veterans Referred for Electrocardiography and Treadmill Testing
Prakash et al.
Arch Intern Med 2003;163:2204-2210.
ABSTRACT | FULL TEXT  

Improving the flow of diagnostic information: The importance of STARD for authors and readers
Holloway
Neurology 2003;61:600-601.
FULL TEXT  

Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative
Bossuyt et al.
Am. J. Roentgenol. 2003;181:51-55.
ABSTRACT | FULL TEXT  

Cross sectional study of conventional cervical smear, monolayer cytology, and human papillomavirus DNA testing for cervical cancer screening
Coste et al.
BMJ 2003;326:733-733.
ABSTRACT | FULL TEXT  

A grey zone for quantitative diagnostic and screening tests
Coste and Pouchot
Int J Epidemiol 2003;32:304-313.
ABSTRACT | FULL TEXT  

Severe Liver Fibrosis or Cirrhosis: Accuracy of US for Detection--Analysis of 300 Cases
Colli et al.
Radiology 2003;227:89-94.
ABSTRACT | FULL TEXT  

Accuracy of Ottawa ankle rules to exclude fractures of the ankle and mid-foot: systematic review
Bachmann et al.
BMJ 2003;326:417-417.
ABSTRACT | FULL TEXT  

Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative
Bossuyt et al.
ANN INTERN MED 2003;138:40-44.
ABSTRACT | FULL TEXT  

The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt et al.
ANN INTERN MED 2003;138:W1-W12.
ABSTRACT | FULL TEXT  

Reporting diagnostic tests
Straus
BMJ 2003;326:3-4.
FULL TEXT  

Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative
Bossuyt et al.
BMJ 2003;326:41-44.
ABSTRACT | FULL TEXT  

Clinical efficacy of Doppler-echocardiographic indices of aortic valve stenosis:a comparative test-based analysis of outcome
Bermejo et al.
J Am Coll Cardiol 2003;41:142-151.
ABSTRACT | FULL TEXT  

A Comparison of Bayesian and Maximum Likelihood Methods to Determine the Performance of a Point of Care Test for Helicobacter pylori in the Office Setting
Delaney et al.
Med Decis Making 2003;23:21-30.
ABSTRACT  

Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative
Bossuyt et al.
Clin. Chem. 2003;49:1-6.
ABSTRACT | FULL TEXT  

The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt et al.
Clin. Chem. 2003;49:7-18.
ABSTRACT | FULL TEXT  

The STARD Initiative and the Reporting of Studies of Diagnostic Accuracy
Bruns
Clin. Chem. 2003;49:19-20.
FULL TEXT  

Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative
Bossuyt et al.
Radiology 2003;226:24-28.
ABSTRACT | FULL TEXT  

Contrast-Enhanced Magnetic Resonance Angiography of Carotid Arteries: Utility in Routine Clinical Practice
Johnston et al.
Stroke 2002;33:2834-2838.
ABSTRACT | FULL TEXT  

Spectrum Bias or Spectrum Effect? Subgroup Variation in Diagnostic Test Evaluation
Mulherin and Miller
ANN INTERN MED 2002;137:598-602.
ABSTRACT | FULL TEXT  

Evaluation of studies of assessment and screening tools, and diagnostic tests
Jull
Evid. Based Nurs. 2002;5:68-72.
FULL TEXT  

The winding road towards evidence based diagnoses
Hernandez-Aguado
J. Epidemiol. Community Health 2002;56:323-325.
FULL TEXT  

Misguided efforts and future challenges for research on "diagnostic tests"
Feinstein
J. Epidemiol. Community Health 2002;56:330-332.
ABSTRACT | FULL TEXT  

Improving standards of medical and public health research
Colditz
J. Epidemiol. Community Health 2002;56:333-334.
FULL TEXT  

Challenges in dia-prognostic research
Knottnerus
J. Epidemiol. Community Health 2002;56:340-341.
FULL TEXT  

Exercise Capacity and Mortality among Men Referred for Exercise Testing
Myers et al.
NEJM 2002;346:793-801.
ABSTRACT | FULL TEXT  

Evidence base of clinical diagnosis: Evaluation of diagnostic procedures
Knottnerus et al.
BMJ 2002;324:477-480.
FULL TEXT  

Methodological standards in radiographer plain film reading performance studies
Brealey et al.
Br. J. Radiol. 2002;75:107-113.
ABSTRACT | FULL TEXT  

Use of magnetic resonance angiography to select candidates with recently symptomatic carotid stenosis for surgery: systematic review
Westwood et al.
BMJ 2002;324:198-198.
ABSTRACT | FULL TEXT  

Autoantibodies to Extractable Nuclear Antigens: Making Detection and Interpretation More Meaningful
Phan et al.
CVI 2002;9:1-7.
FULL TEXT  

Heart rate recovery: validation and methodologic issues
Shetler et al.
J Am Coll Cardiol 2001;38:1980-1987.
ABSTRACT | FULL TEXT  

Rapid and simple point of care diagnostics for STIs
Mabey et al.
Sex. Transm. Infect. 2001;77:397-398.
FULL TEXT  

Comparison of Exercise Test Scores and Physician Estimation in Determining Disease Probability
Lipinski et al.
Arch Intern Med 2001;161:2239-2244.
ABSTRACT | FULL TEXT  

A systematic review of the staging performance of endoscopic ultrasound in gastro-oesophageal carcinoma
Kelly et al.
Gut 2001;49:534-539.
ABSTRACT | FULL TEXT  

Clinical and Exercise Test Predictors of All-Cause Mortality : Results From > 6,000 Consecutive Referred Male Patients
Prakash et al.
Chest 2001;120:1003-1013.
ABSTRACT | FULL TEXT  

Examining Diagnostic Tests: An Evidence-Based Perspective
Fritz and Wainner
ptjournal 2001;81:1546-1564.
ABSTRACT | FULL TEXT  

Endometrial Evaluation with Transvaginal US and Hysterosonography in Asymptomatic Postmenopausal Women with Breast Cancer Receiving Tamoxifen
Fong et al.
Radiology 2001;220:765-773.
ABSTRACT | FULL TEXT  

Overview of Evidence-based Medicine: Challenges for Evidence-based Laboratory Medicine
McQueen
Clin. Chem. 2001;47:1536-1546.
ABSTRACT | FULL TEXT  

Systematic reviews in health care: Systematic reviews of evaluations of diagnostic and screening tests
Deeks
BMJ 2001;323:157-162.
FULL TEXT  

Simple Treadmill Score To Diagnose Coronary Disease
Raxwal et al.
Chest 2001;119:1933-1940.
ABSTRACT | FULL TEXT