 |
|
Interpreting Surgical Trials With Subjective Outcomes
Avoiding UnSPORTsmanlike Conduct
David R. Flum, MD, MPH
JAMA. 2006;296:2483-2485.
In this issue of JAMA, 2 articles1-2 report results of the Spine Patient Outcomes Research Trial (SPORT). One study is a multicenter trial1 of patients with persistent disk-related pain and neurologic symptoms randomized to undergo diskectomy or receive usual care (most often, patient education, anti-inflammatory medication, and physical therapy, alone or in combination). Because the investigators expected high rates of refusal of randomization based on the differential risk of these 2 treatment strategies, they developed a parallel observational study of patients who qualified for the randomized trial but refused randomization.2 Patients in both the randomized trial and the observational cohort study were similar in almost all characteristics and were followed up in a similar fashion for 2 years.
The SPORT randomized trial had such a high proportion of patients who crossed over between treatment strategies (45%-60%) and such a significant degree of missing data (24%-27%) that an intention-to-treat analysis was not informative about the comparative efficacy of the strategies. The study showed that patients randomized to either operation or usual care improved considerably over the 2 years of the study and that patients who had an operation were more likely to have been severely affected by their disk pain than those who opted for usual care. Although it is tempting to consider "as-treated" analyses when looking for the effect of an intervention, such analyses are significantly confounded and should not be used to compare different strategies.3 For a different set of reasons, it is equally problematic to use the second, observational cohort study to understand the role of surgical intervention for discogenic pain.
When evaluating subjective outcomes such as pain, satisfaction, and function after health care interventions, it is critical to distinguish the effect of the intervention from the effect of the patient's expectation of the intervention. The impact of expectation on subjective outcome is commonly described as the placebo effect. The only way to distinguish the effect of a patient's positive expectations of the operation from the intervention itself is to blind patients to the treatment they receive and then randomize them to receive the intervention of interest or to receive an ineffective, sham intervention (placebo). While this technique is well accepted in studies of medications using inactive pills (placebos), simulated acupuncture, and nontherapeutic conversation in place of therapeutic psychiatric interventions, it has only occasionally been applied to surgical trials. This is unfortunate because the use of sham controls has been critical in understanding just how much patient expectation influences outcome after an operation. Given the proliferation of operative procedures for the treatment of subjective complaints like back pain, the need for sham-controlled trials has never been greater.
A dramatic demonstration of the power of suggestion in surgical outcomes can be found in one of the first sham-controlled surgical studies, performed in 1959. After multiple observational studies suggesting that ligation of the internal mammary artery was helpful in patients with coronary disease, Cobb et al4 randomized patients to operative arterial ligation or a sham procedure. Both groups improved after the intervention, but there were similar, if not greater, improvements in subjective measures such as exercise tolerance and nitroglycerin use in the sham surgical group.
More recently, after multiple case series demonstrated that patients with osteoarthritis of the knee improve after arthroscopic surgery, Moseley et al5 demonstrated just how much of that effect is related to the hopes, expectations, and beliefs of the patient. The investigators randomized 180 patients to undergo arthroscopy with debridement, arthroscopy with lavage, or sham arthroscopy. The power of expectation was so strong that patients could not determine if they were assigned to the treatment or sham groups—and all groups improved. At 2 years after randomization, all patients reported comparable pain scores and functional scores.
There have been several other arenas of invasive health care intervention in which observational trials have shown procedural success but sham-controlled trials demonstrate how much of that success is due to the placebo effect. Another sham-controlled study6 in patients with knee osteoarthritis demonstrated that patients benefit equally from irrigation of the joint and sham irrigation. Researchers found similar improvements in quality of life after direct brain injections of embryonic neurons or placebo in patients with advanced Parkinson disease.7 Heart failure patients undergoing transmyocardial laser revascularization or sham procedures6 had equal improvements in subjective outcomes. After hernia repair, there was equal improvement in pain control after cryoablation of nerves or sham interventions.8
These sham-controlled studies fundamentally question the utility of the procedures they are compared with and are critical to the health care community, but are they ethical? An ethical framework for the use of sham surgery has recently been proposed.9 This framework justifies placebo-controlled trials of invasive procedures when the risk of a placebo does not exceed a threshold of acceptable research risk and if the knowledge to be gained is substantial. Ethical justification of placebo-controlled trials hinges on the question of community exposure to invasive, high-cost procedures with associated risk.
Like all randomized trials, sham-controlled trials are justified only when genuine equipoise exists among clinicians about the merits of the intervention. Equipoise often develops when there is conflicting or weak evidence about the efficacy of the intervention or when clinical experience does not correspond to published reports. Clinical equipoise was at the heart of the SPORT randomized trial, but the authors reported that a sham-controlled trial was impractical and unethical, presumably because the risk of the sham would include general anesthesia (to truly blind the patients). In studies in which local anesthesia is part of the sham, one argument might be that the anesthesia is no greater risk than the risk associated with driving in a car to the hospital to receive a sugar pill. Given the current safety of general anesthesia, perhaps a similar justification could be made for sham-controlled spine surgery.
A sham-controlled trial would be particularly relevant for spine surgery because while there are objective outcomes that are relevant in spine surgery (operative complications, development of testable muscle weakness, neurologic complications), the more commonly occurring and relevant outcomes are subjective. The observational cohort study of the SPORT investigation found that patients who underwent spine surgery and patients who received usual care had significant improvements in subjective outcomes that persisted at 2 years. Patients who had an operation were much more bothered by their pain and disability at baseline compared with patients who agreed to nonoperative care and, despite this, had greater improvement in almost all parameters.
Although at first this finding suggests that surgery is more beneficial than usual care, this interpretation may be flawed. Patients who elected to have surgery were different in many ways than those who did not. A higher level of disease severity among operative-care patients might be considered a conservative bias in that treatment effects among patients with similar disability might be even greater. But if anything has been learned from the legacy of sham-controlled trials, these differences may also include a greater expectation of success among patients having the more invasive intervention. As in the studies of arthroscopy, improvement among patients having spine operations may be the result of their expectations of success rather than the procedure itself. How much of the observed difference in outcome in the SPORT trial is the result of these raised expectations can only be determined by studying the placebo effect through sham-controlled trials.
Why might a patient get better after sham surgery? The underlying components of raised expectations are complicated. They may include a higher level of emotional "investment" in surgical care compared with usual care based on the level of commitment that flows from a decision to have an operation and get through recovery. After the patient has accepted the risks of surgical intervention, the desire for improvement may drive perceptions about improvement. Patients who opt for surgery may also be different than others in their beliefs of the benefits of invasive intervention.
The surgeon's expectations and direction also may play an important role in patient improvement. For example, multiple case series have demonstrated the benefit of laparoscopic cholecystectomy, colon resection, and appendectomy compared with conventional approaches on subjective outcomes such as pain control, function, and readiness for discharge. In reality, the clinical care team influences patient and discharge expectations though coaching, communication, and management. Randomized trials of these 3 procedures8-10 that blind both the patients and the discharging clinicians to the treatment that patients received by placing very large, side-to-side abdominal wall dressings (and then using standardized discharge care and criteria) demonstrate little or no difference in these outcomes. When the clinician's differential expectations and "coaching" were removed by placing a large bandage on the abdominal wall, these subjective benefits disappeared. Studies that do not address both patient and clinician expectation on subjective outcome do not inform the clinical community about the true role of the intervention.
Helping balance the competing risks and benefits of operative and nonoperative approaches to discogenic pain and neurologic symptoms was the goal of the SPORT trial. Because of limitations in design and study operation, the proper role and benefits of these competing interventions are still unclear. Given the large number of patients potentially exposed to the risks of these strategies, a sham surgical trial may be the only effective and ethical next step.
AUTHOR INFORMATION
Financial Disclosures: None reported.
Editorials represent the opinions of the authors and JAMA and not those of the American Medical Association.
Author Affiliations: Department of Surgery, University of Washington, Seattle. Dr Flum (daveflum{at}u.washington.edu) is Contributing Editor, JAMA.
REFERENCES
|
1. Weinstein JN, Tosteson TD, Lurie JD, et al. Surgical vs nonoperative treatment for lumbar disk herniation: the Spine Patient Outcomes Research Trial (SPORT): a randomized trial. JAMA. 2006;296:2441-2450.
FREE FULL TEXT
2. Weinstein JN, Lurie JD, Tosteson TD, et al. Surgical vs nonoperative treatment for lumbar disk herniation: the Spine Patient Outcomes Research Trial (SPORT) observational cohort. JAMA. 2006;296:2451-2459.
FREE FULL TEXT
3. Flum DR. The asymptomatic hernia: "if it's not broken, don't fix it." JAMA. 2006;295:328-329.
FREE FULL TEXT
4. Cobb LA, Thomas GI, Dillard DH, Merendino KA, Bruce RA. An evaluation of internal-mammary-artery ligation by a double-blind technic. N Engl J Med. 1959;260:1115-1118.
ISI
| PUBMED
5. Moseley JB, O’Malley K, Petersen NJ, et al. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med. 2002;347:81-88.
FREE FULL TEXT
6. Bradley JD, Heilman DK, Katz BP, Gsell P, Wallick JE, Brandt KD. Tidal irrigation as treatment for knee osteoarthritis: a sham-controlled, randomized, double-blinded evaluation. Arthritis Rheum. 2002;46:100-108.
FULL TEXT
|
ISI
| PUBMED
7. McRae C, Cherin E, Yamazaki TG, et al. Effects of perceived treatment on quality of life and medical outcomes in a double-blind placebo surgery trial. Arch Gen Psychiatry. 2004;61:412-420.
FREE FULL TEXT
8. Callesen T, Bech K, Thorup J, et al. Cryoanalgesia: effect on postherniorrhaphy pain. Anesth Analg. 1998;87:896-899.
FREE FULL TEXT
9. Horng S, Miller FG. Ethical framework for the use of sham procedures in clinical trials. Crit Care Med. 2003;31(3 suppl):S126-S130.
FULL TEXT
|
ISI
| PUBMED
10. Basse L, Jakobsen DH, Bardram L, et al. Functional recovery after open versus laparoscopic colonic resection: a randomized, blinded study. Ann Surg. 2005;241:416-423.
FULL TEXT
|
ISI
| PUBMED
RELATED LETTERS
Sham Surgery in Clinical Trials
Peter Angelos
JAMA. 2007;297(14):1545-1546.
EXTRACT
| FULL TEXT
Sham Surgery in Clinical Trials—Reply
David R. Flum
JAMA. 2007;297(14):1546.
EXTRACT
| FULL TEXT
RELATED ARTICLES
Surgical vs Nonoperative Treatment for Lumbar Disk Herniation: The Spine Patient Outcomes Research Trial (SPORT): A Randomized Trial
James N. Weinstein, Tor D. Tosteson, Jon D. Lurie, Anna N. A. Tosteson, Brett Hanscom, Jonathan S. Skinner, William A. Abdu, Alan S. Hilibrand, Scott D. Boden, and Richard A. Deyo
JAMA. 2006;296(20):2441-2450.
ABSTRACT
| FULL TEXT
Surgical vs Nonoperative Treatment for Lumbar Disk Herniation: The Spine Patient Outcomes Research Trial (SPORT) Observational Cohort
James N. Weinstein, Jon D. Lurie, Tor D. Tosteson, Jonathan S. Skinner, Brett Hanscom, Anna N. A. Tosteson, Harry Herkowitz, Jeffrey Fischgrund, Frank P. Cammisa, Todd Albert, and Richard A. Deyo
JAMA. 2006;296(20):2451-2459.
ABSTRACT
| FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
|
The Role of NETT in Emphysema Research
Wise and Drummond
Proc Am Thorac Soc 2008;5:385-392.
ABSTRACT
| FULL TEXT
Gizmo Idolatry
Leff and Finucane
JAMA 2008;299:1830-1832.
FULL TEXT
Ethical Issues Confronted In Pulmonary Clinical Trials
Wise
Proc Am Thorac Soc 2007;4:200-205.
ABSTRACT
| FULL TEXT
Either Surgery or Nonoperative Treatment Led to Improvement in Intervertebral Disc Herniation
Mirza
JBJS 2007;89:1139-1139.
FULL TEXT
Sham Surgery in Clinical Trials
Angelos
JAMA 2007;297:1545-1546.
FULL TEXT
Is Surgery Necessary for Lumbar Disc Herniation?
JWatch General 2006;2006:1-1.
FULL TEXT
|
