Magnetic Resonance Angiography for the Evaluation of Lower Extremity Arterial Disease
A Meta-analysis
- Mark J. W. Koelemay, MD;
- Jeroen G. Lijmer, MD, PhD;
- Jaap Stoker, MD, PhD;
- Dink A. Legemate, MD, PhD;
- Patrick M. M. Bossuyt, MD, PhD
Abstract
Context Magnetic resonance angiography (MRA) is a rapidly evolving technique that has been reported to be accurate for assessment of lower extremity arterial disease.
Objective To obtain the best available estimates of the diagnostic performance of MRA in patients with lower extremity arterial disease.
Data Sources Studies published from January 1985 through May 2000 in English, German, or French, identified from the MEDLINE, EMBASE, and Current Contents databases.
Study Selection Studies were included that allowed construction of 2 × 2 contingency tables for detection of stenosis greater than 50% or occlusion with MRA or arteriography in patients with claudication or critical ischemia.
Data Extraction Two observers graded the following elements of study quality: consecutively enrolled patients, prospective study design, clear cut-off levels, blinded assessment, and clear description of MRA technique. Summary receiver operating characteristic analysis was performed to examine the influence of year of publication, all methodological criteria, arterial tract, number of subdivisions within arterial tracts, and MRA technique on diagnostic performance.
Data Synthesis Of 3583 studies initially identified, 34 were included that evaluated MRA in 1090 patients (72% men; median age, 65 years). Magnetic resonance angiography was highly accurate for assessment of all lower extremity arteries. Three-dimensional gadolinium-enhanced (3-D Gd) MRA improved diagnostic performance compared with 2-D MRA (relative diagnostic odds ratio, 2.8 [95% confidence interval, 1.2-6.4]), adjusted for number of subdivisions within arterial tracts. The estimated points of equal sensitivity and specificity were 94% and 90% for 3-D Gd MRA and 2-D MRA, respectively.
Conclusions Magnetic resonance angiography is highly accurate for assessment of the entire lower extremity for arterial disease. Three-dimensional Gd-enhanced MRA improves diagnostic performance compared with 2-D MRA.
- KEYWORDS:
- arterial occlusive diseases
- diagnostic imaging
- leg
- lower extremity
- magnetic resonance angiography
- meta-analysis
Over the past decade the importance of arteriography for assessment of lower extremity arterial disease has been decreased by the introduction of noninvasive techniques such as color Duplex ultrasonography1-2 and magnetic resonance angiography (MRA). Time-of-flight MRA is based on contrast formation by the inflow of unsaturated spins (blood) into surrounding saturated spins (perivascular tissue). Acquisition of images is possible in 2-dimensional (2-D) or 3-dimensional (3-D) mode. Two-dimensional MRA can cover large areas within reasonable time, but it is sensitive to changes in flow direction. This may be overcome by 3-D MRA, which is, however, limited by rapid saturation of flowing spins and requires contrast agents such as gadolinium to improve visualization.
Magnetic resonance angiography is not a standardized technique. Various MRA protocols have proved to be accurate for assessment of the lower extremity arteries compared with conventional arteriography (CA) or intra-arterial digital subtraction angiography (iaDSA). Although arteriography is the generally accepted reference standard, its accuracy for depiction of the infrapopliteal arteries is limited. Accurate assessment of these vessels could give MRA the potential to supplant arteriography before distal bypass surgery.
Before a technique such as MRA can be used for clinical decision making it should be evaluated in well-designed studies. The methodological quality of early studies evaluating MRA was mediocre, but it has improved in more recent studies.3 Our objective was to obtain the best estimates of the diagnostic accuracy of MRA for evaluation of lower extremity arterial disease by performing a systematic literature review and meta-analysis if possible.
METHODS
Data Sources and Study Selection
MEDLINE (from 1985), EMBASE (from 1988), and the Current Contents database were searched for publications through May 2000 in English, German, and French with magnetic resonance angiography as the sole key word. Based on located titles and abstracts, studies that evaluated MRA for assessment of the lower extremity arteries were selected. Cross-references were used for search completion.
Studies were included that compared MRA with CA or iaDSA as the reference standard in patients with claudication or critical limb ischemia for detection of greater than 50% stenosis or occlusion and that presented 2 × 2 contingency tables or data allowing their construction. If such data were available for only a subset of patients, these data were included. Studies on MRA for follow-up after percutaneous transluminal angioplasty (PTA) or surgery and duplicate publications were excluded.
Methodological Assessment and Data Extraction
Methodological quality was graded independently by 2 observers (M.J.W.K., J.S.) to categorize studies. The following elements for good study quality were scored: consecutively enrolled patients, prospective study design, clear description of MRA technique (providing sufficient detail to permit replication), clear definition of cut-off levels (ie, quantifying target lesions as ≤50%, >50% stenosis, or occlusion) and independent, blind assessment of MRA and arteriography results.4 Discrepancies in judgment were resolved by consensus. κ Values were calculated to express interobserver agreement on each element. In some studies procedures in addition to the reference standard were performed in some of the patients when inadequate images were obtained, such as CA supplemented by DSA techniques. Using multiple reference standards might influence diagnostic accuracy. For instance, if iaDSA depicts the infrapopliteal arteries better than CA does, an artery may appear patent on iaDSA and occluded on CA. If multiple reference standards were used and it was not stated that all patients underwent additional procedures, reference test enhancement bias was assumed to be present.
The lower extremity arteries were divided into 3 defined arterial tracts: the aortoiliac tract from the infrarenal aorta to the common femoral artery, the femoropopliteal tract from the common femoral artery to the trifurcation, and the infrapopliteal arteries from trifurcation to the pedal arteries. Data were abstracted to construct 2 × 2 tables for each tract. Results for a specific tract reported by multiple observers were averaged. If it was impossible to derive results for a defined tract, 2 × 2 tables were constructed for aggregate results. Studies comparing different MRA modalities within the same population were included as such.
Some authors made subdivisions within a defined arterial tract, which could overestimate diagnostic accuracy because false-positive or false-negative results will be overwhelmed by the artificially high number of correct results. Especially, the negative predictive value will be overestimated. We tried to adjust for this by estimating the number of subdivisions made in such studies. To do so, we divided the total number of observations from the 2 × 2 table by the number of patients studied.
Statistical Analysis
The raw data were summarized according to a method described previously.5-7 For each study, sensitivity, specificity, and diagnostic odds ratios (DORs) were calculated from the 2 × 2 tables. The DOR is a simple statistic to express the discriminative power of a test. It is defined as the ratio of sensitivity / (1 − sensitivity) over (1 − specificity) / specificity. In simpler terms, the DOR is the odds of a positive test result if the arterial segment genuinely is diseased, divided by the odds of a positive test result if the segment is not diseased. A DOR greater than 1 indicates that a test has discriminative power, which increases with the magnitude of the DOR. To prevent division by 0 when calculating the DOR, conventional correction was applied by adding 0.5 to each cell in the 2 × 2 tables.8
Homogeneity of the studies was tested with the Breslow-Day test for homogeneity of odds ratios.9 In case of homogeneity the data were pooled to calculate summary point estimates of sensitivity and specificity. In case of heterogeneity summary receiver operating characteristic (SROC) curves were constructed, which correct for variation due to differences in test thresholds in the original studies. These curves can be defined by a regression model DOR = α + βS, in which the intercept represents the corresponding DOR and S the variation due to differences in test threshold. Once the slope and intercept are estimated by the regression analysis, data can be transformed into an SROC curve with conventional axes of sensitivity (true-positive rate [TPR]) vs 1 − specificity (false-positive rate [FPR]) with the equation TPR = 1/{1 + exp[−α/(1 − β)]} × [(1 − FPR)/(FPR)](1 + β)/(1 − β).
Regression analysis allows exploration of the influence of covariates on diagnostic performance, which can be expressed as relative DORs. A stepwise weighted linear multivariable regression model with backward elimination was used, in which weights proportional to the reciprocal of the variance of the log DOR represented the within-study variation, while random effects between studies were estimated using restricted maximum likelihood estimation. P>.10 was used to remove variables from the model. Calculations were performed with S-plus statistical software (S-plus 2000, Mathsoft Inc, Cambridge, Mass) and SAS version 6.12 (SAS Institute Inc, Cary, NC).
RESULTS
Search Results and Methodological Assessment
We identified 3583 studies of MRA in general and 157 studies of MRA for assessment of lower extremity arterial disease. Among these were 46 studies that compared MRA with arteriography. Twelve studies were excluded because they did not state if patients had claudication or critical limb ischemia10-17 or used unclear cut-off levels.18-21 Table 1 lists the 34 included studies and their patient characteristics. Three studies were published in German22, 35, 47 and 1 was in French.48 Fourteen studies satisfied all methodological criteria.22-35 The κ values for agreement on study quality elements ranged between 0.48 and 0.55, except for agreement on independent assessment of MRA and arteriography (0.35).
Table 1. Characteristics of Included Studies and Their Patient Populations (N = 1090)*
Meta-analysis
The median sample size of the 34 studies was 25 patients (range, 13-115). The studies summarize the diagnostic performance of MRA in 1090 patients (72% men; median of the mean age, 65 years). Most of the patients (median, 83%) had claudication.
Table 2 lists the results from the individual studies. The low specificity in 1 study was attributed by the authors to tortuosity of the iliac arteries, clip artifacts, and stents in patients with prior interventions.39 Thirteen studies reported diagnostic accuracy as aggregate results (Table 3). From a study that simultaneously evaluated 4 different MRA protocols, the one with the highest accuracy was included.28 Homogeneity of the DOR was rejected by the Breslow-Day test (χ250 = 313,129; P<.001). Four studies with outlying results were detected by inspection of all data points in ROC space 4.35, 38-39,52 When these studies were excluded, homogeneity was also rejected.
Table 2. Diagnostic Accuracy of MRA for Detection of Stenosis >50% or Occlusion*
Table 3. Diagnostic Accuracy of MRA for Detection of Stenosis >50% or Occlusion: Aggregate Results*
Multivariable analysis was then performed to identify independent factors that influenced diagnostic performance and to construct SROC curves. The variables average age and proportion of men were not included in this analysis because 9 studies had missing data. Variables entered were year of publication, all methodological criteria, reference standard enhancement bias, arterial tract, number of subdivisions within arterial tracts, prevalence of target lesions, and MRA technique (2-D vs 3-D gadolinium-enhanced). Continuous variables were grouped according to tertiles. Irrespective of arterial tract, 3-D gadolinium-enhanced MRA had better discriminative power than 2-D MRA (Table 4). A high number of subdivisions also improved diagnostic performance. We constructed SROC curves for each MRA technique, adjusted for the number of subdivisions (Figure 1). Although this adjustment yields a conservative estimation, MRA is highly accurate for detection of stenosis greater than 50% or an occlusion in the entire lower extremity arterial tree. The estimated Q-points (where sensitivity and specificity are equal) were 94% for 3-D gadolinium-enhanced and 90% for 2-D MRA, respectively. These results did not change substantially when studies with outlying results were excluded.35, 38-39,52
Table 4. Relative Diagnostic Odds Ratios After Multivariable Analysis*
Figure. Summary Receiver Operating Characteristic Curve for MRA Detection of >50% Stenosis or Occlusion
Adjusted for number of subdivisions within defined arterial tracts. MRA indicates magnetic resonance angiography; 2-D, 2-dimensional; 3-D, 3-dimensional.
COMMENT
In a systematic review of the published literature, we found that MRA is highly accurate for detection of stenosis greater than 50% or occlusion within the entire lower extremity arterial tree. It could be expected that the advantage of 3-D gadolinium-enhanced MRA over 2-D MRA would be particularly strong within the tortuous aortoiliac tract,56 but we did not find a difference in relative performance between defined arterial tracts. The clinical significance of a 4% difference in Q-points remains to be established. Alternative explanations for our findings could be differences in spectrum of disease within the respective study populations or differences in MRA protocols. Unfortunately, too many data were missing on stage of disease, age, or sex to include these variables in the analysis. For the same reason, the influence of the use of maximum-intensity projection (9% missing data) alone or in combination with source data (71% missing or ambiguous data) to construct images and voxel size (43% missing) could not be analyzed. The studies included in this meta-analysis were all published within the past 8 years. In this short period, MRA techniques have changed significantly. Image acquisition time has been reduced from more than 1 hour39 to a few minutes.31 Susceptibility to flow artifacts and image quality have been improved by electrocardiogram triggering, gadolinium contrast, and subtraction techniques.29-30,32, 47 Moreover, the value of new contrast agents or MR sequences has not yet been tested in comparative studies. These innovations suggest that MRA techniques and their definitive role in clinical practice are still evolving.
Use of MRA has repeatedly detected infrapopliteal arteries not visible on CA that were suitable bypass recipients at surgical exploration.40, 57-59 Some consider arteriography inappropriate as a reference standard for these arteries and prefer intraoperative angiography. However, the incidence of angiographically occult patent vessels varies between centers.57 Moreover, in a multicenter trial, arteriography and MRA were equally accurate compared with intraoperative angiography or post-PTA arteriography,57 while in other studies arteriography was even more accurate than MRA.60-61 Not until randomized trials are performed that compare outcomes of distal bypass surgery based on arteriography and MRA will it be clear which modality is to be preferred.
Little is known about the value of MRA for clinical outcomes. The patients who benefit most from PTA or surgery have critical limb ischemia, not claudication. The majority of the studied patients had claudication, which emphasizes the need for comparative and management studies in patients with ischemic pain and tissue loss. Unfortunately, the included studies did not allow a subgroup analysis of diagnostic performance for each stage of disease. Several studies compared treatment plans made by different observers based on arteriography and MRA.10, 20, 25, 36, 41, 57-64 When MR angiograms were technically adequate, the concordance in proposed interventions exceeded 85%. Studies designed as such remain hypothetical because clinical decision making is subject to interindividual variability.62, 65-66
Whereas early comparative studies had methodological flaws such as unclear patient selection, inadequate description of inception cohort, no routine independent assessment of MRA and reference standard, and not reporting interobserver variability,3 most studies included in our analysis were of high methodological quality. Still, the judgment of individual study outcomes and our meta-analysis has its limitations. The interobserver agreement on rating individual elements of study quality was only fair, despite absolute agreement in 77% to 94%. This may be explained by the fact that κ values depend on the frequency of judgments in each category.67 In studies with a small sample size, defined arterial tracts were often subdivided into arbitrary segments. Our analysis shows that the diagnostic accuracy is overestimated in such studies. It would be helpful if such studies presented their data on a segment-to-segment basis, which allows the calculation of sensitivity and specificity and pooling of raw data for both each segment and each defined arterial tract. Thirteen studies presented accuracy as aggregate results. Although our analysis did not reveal differences in diagnostic accuracy among arterial tracts, presenting data as such obstructs the interpretation of MRA for assessment of a specific arterial tract. Consequently, it remains unclear if MRA is suitable for planning interventions within these tracts, such as PTA of the iliac arteries.
A final limitation of this study is the possibility of publication bias. We did not attempt to quantify the number of unpublished studies but realize that conclusions may be too optimistic when studies with favorable results are more likely to be submitted and published. It has been reported that exclusion of unpublished studies can yield a 15% larger intervention effect, but such data are not available for diagnostic research.68
In summary, we found that MRA is highly accurate for evaluation of lower extremity arterial disease. Its accuracy has been improved by 3-D gadolinium-enhanced techniques. More studies are needed in patients with critical limb ischemia. To facilitate the dissemination of MRA, standardization of study design and reporting of results is needed.
Acknowledgments
Author Contributions: Study concept and design: Koelemay, Legemate, Bossuyt.
Acquisition of data: Koelemay, Stoker.
Analysis and interpretation of data: Koelemay, Lijmer, Bossuyt.
Drafting of the manuscript: Koelemay, Stoker.
Critical revision of the manuscript for important intellectual content: Koelemay, Lijmer, Stoker, Legemate, Bossuyt.
Statistical expertise: Koelemay, Lijmer, Bossuyt.
Administrative, technical, or material support: Legemate.
Study supervision: Legemate, Bossuyt.
Acknowledgment: We thank Siem Heisterkamp, PhD, for statistical advice and software programming.
Corresponding Author and Reprints: Dink A. Legemate, MD, PhD, Department of Surgery, G4-111, Academic Medical Center, PO Box 22700, 1100 DE Amsterdam, the Netherlands (e-mail: d.a.legemate{at}amc.uva.nl).
