This Article  • References  • Full text PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Contact me when this article is cited  Related Content  • Similar articles in JAMA  Social Bookmarking   What's this?

A Problem in Diagnostic Radiology

Thomas K. P. Egglin, MD; Alvan R. Feinstein, MD

JAMA. 1996;276(21):1752-1755.

Abstract
Objective.
—To determine whether radiologists' interpretations of images are biased by their context and by prevalence of disease in other recently observed cases.

Methods.
—A test set of 24 right pulmonary arteriograms with a 33% prevalence of pulmonary emboli (PE) was assembled and embedded in 2 larger groups of films. Group A contained 16 additional arteriograms, all showing PE involving the right lung, so that total prevalence was 60%. Group B contained 16 additional arteriograms without PE so that total prevalence was 20%. Six radiologists were randomly assigned to see either group first and then "cross over" to review the other group after a hiatus of at least 8 weeks. The direction of changes in a 5-point rating scale for the 2 readings of each film in the test set was compared with the sign test; mean sensitivity, specificity, and areas under receiver operating characteristic (ROC) curves were compared with the paired t test.

Results.
—In the context of group A's higher disease prevalence, radiologists shifted more of their diagnoses toward higher suspicion than expected by chance (P=.03, sign test). In group A, mean sensitivity for diagnosing PE was significantly higher (75% vs 60%; P=.04), and area under the ROC curve was significantly larger (0.88 vs 0.82; P=.02)

Conclusions.
—Radiologists' diagnoses are significantly influenced by the context of interpretation, even when spectrum and verification bias are avoided. This "context bias" effect is unique to the evaluation of subjectively interpreted tests, and illustrates the difficulty of obtaining unbiased estimates of diagnostic accuracy for both new and existing technologies.

Author Affiliations
From the Departments of Diagnostic Radiology (Dr Egglin) and Internal Medicine and Epidemiology (Dr Feinstein), Yale University School of Medicine, New Haven, Conn.

Footnotes
Reprints: Thomas K. P. Egglin, MD, Department of Diagnostic Radiology, Yale University School of Medicine, 333 Cedar St, PO Box 208042, New Haven, CT 06520-8042.

Topics in Radiology section editors: Ronald G. Evens, MD, Mallinckrodt Institute of Radiology, Washington University School of Medicine, St Louis, Mo; Charles Clayman, MD, Contributing Editor, JAMA.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

The "Laboratory" Effect: Comparing Radiologists' Performance and Variability during Prospective Clinical and Laboratory Mammography Interpretations
Gur et al.
Radiology 2008;249:47-53.
ABSTRACT | FULL TEXT  

Accuracy of Diagnostic Procedures: Has It Improved Over the Past Five Decades?
Berlin
Am. J. Roentgenol. 2007;188:1173-1178.
FULL TEXT  

Bias in Research Studies
Sica
Radiology 2006;238:780-789.
ABSTRACT | FULL TEXT  

Organized Breast Screening Programs in Canada: Effect of Radiologist Reading Volumes on Outcomes
Coldman et al.
Radiology 2006;238:809-815.
ABSTRACT | FULL TEXT  

Adding in Vivo Quantitative 1H MR Spectroscopy to Improve Diagnostic Accuracy of Breast MR Imaging: Preliminary Results of Observer Performance Study at 4.0 T
Meisamy et al.
Radiology 2005;236:465-475.
ABSTRACT | FULL TEXT  

Physician Predictors of Mammographic Accuracy
Smith-Bindman et al.
JNCI J Natl Cancer Inst 2005;97:358-367.
ABSTRACT | FULL TEXT  

Detection of Lung Cancer on Radiographs: Receiver Operating Characteristic Analyses of Radiologists', Pulmonologists', and Anesthesiologists' Performance
Monnier-Cholley et al.
Radiology 2004;233:799-805.
ABSTRACT | FULL TEXT  

Radiographs in the Office: Is a Second Reading Always Needed?
Smith et al.
J Am Board Fam Med 2004;17:256-263.
ABSTRACT | FULL TEXT  

One Less Bias to Worry About [letter]
Obuchowski
Radiology 2004;232:302-302.
FULL TEXT  

Sources of Variation and Bias in Studies of Diagnostic Accuracy: A Systematic Review
Whiting et al.
ANN INTERN MED 2004;140:189-202.
ABSTRACT | FULL TEXT  

Special Topics III: Bias
Obuchowski
Radiology 2003;229:617-621.
ABSTRACT | FULL TEXT  

Evaluation of Proscriptive Health Care Policy Implementation in Screening Mammography
Beam et al.
Radiology 2003;229:534-540.
ABSTRACT | FULL TEXT  

Isotope bone scans: an assessment of their diagnostic use in polyarticular pain of uncertain origin
Whallett et al.
Ann Rheum Dis 2003;62:784-785.
FULL TEXT  

Cross sectional study of conventional cervical smear, monolayer cytology, and human papillomavirus DNA testing for cervical cancer screening
Coste et al.
BMJ 2003;326:733-733.
ABSTRACT | FULL TEXT  

Does Practice Make Perfect When Interpreting Mammography? Part II
Elmore et al.
JNCI J Natl Cancer Inst 2003;95:250-252.
FULL TEXT  

Association of Volume and Volume-Independent Factors With Accuracy in Screening Mammogram Interpretation
Beam et al.
JNCI J Natl Cancer Inst 2003;95:282-290.
ABSTRACT | FULL TEXT  

The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt et al.
ANN INTERN MED 2003;138:W1-W12.
ABSTRACT | FULL TEXT  

The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt et al.
Clin. Chem. 2003;49:7-18.
ABSTRACT | FULL TEXT  

Predicting the Presence of Acute Pulmonary Embolism: A Comparative Analysis of the Artificial Neural Network, Logistic Regression, and Threshold Models
Eng
Am. J. Roentgenol. 2002;179:869-874.
ABSTRACT | FULL TEXT  

Does Training in the Breast Imaging Reporting and Data System (BI-RADS) Improve Biopsy Recommendations or Feature Analysis Agreement with Experienced Breast Imagers at Mammography?
Berg et al.
Radiology 2002;224:871-880.
ABSTRACT | FULL TEXT  

The Effects of Proficiency and Bias on Residents' Interpretation of the Microscopic Urinalysis
Flach et al.
Med Decis Making 2002;22:318-325.
ABSTRACT  

Does Practice Make Perfect When Interpreting Mammography?
Elmore and Carney
JNCI J Natl Cancer Inst 2002;94:321-323.
FULL TEXT  

Examining Diagnostic Tests: An Evidence-Based Perspective
Fritz and Wainner
ptjournal 2001;81:1546-1564.
ABSTRACT | FULL TEXT  

Interpretation of Emergency Department Radiographs: A Comparison of Emergency Medicine Physicians with Radiologists, Residents with Faculty, and Film with Digital Display
Eng et al.
Am. J. Roentgenol. 2000;175:1233-1238.
ABSTRACT | FULL TEXT  

Alliterative Errors
Berlin
Am. J. Roentgenol. 2000;174:925-931.
FULL TEXT  

Confidence limits and the limits of confidence
Kee
QJM 2000;93:121-124.
FULL TEXT  

Cranial Computed Tomography Before Lumbar Puncture: A Prospective Clinical Evaluation
Gopal et al.
Arch Intern Med 1999;159:2681-2685.
ABSTRACT | FULL TEXT  

Journey toward Computer-aided Diagnosis: Role of Image Texture Analysis
Tourassi
Radiology 1999;213:317-320.
FULL TEXT  

Radiologists' Interpretations: The Problem of Context Bias
Vermont
JAMA 1997;277:628-628.
ABSTRACT