L E T T E R T O T H E E D I T O R
Meta-analyses: with confidence or prediction intervals?
Arnaud Chiolero•Vale´rie Santschi•Bernard Burnand•Robert W. Platt •
Gilles Paradis
Received: 13 August 2012 / Accepted: 3 October 2012 / Published online: 16 October 2012 Ó Springer Science+Business Media Dordrecht 2012
In meta-analyses, when data are pooled and analyzed using random effect models, it is standard to report a confidence interval (CI) around the effect estimate [1–3], as reported in several meta-analyses published in the European Jour-nal of Epidemiology [4–6]. Nevertheless, when heteroge-neity is substantial, some authors have proposed to report a prediction interval (PI) rather than a CI to have a better appreciation of the uncertainty around the effect estimate [7–9].
What is the meaning of confidence and prediction intervals? Using results from a meta-analysis demonstrat-ing the impact of pharmacist interventions on blood pres-sure [10], we explain how to use each of these intervals.
In a recent systematic review with meta-analyses of ran-domized controlled trials, we showed that pharmacist inter-ventions improve the management of major cardiovascular disease risk factors in outpatients, including hypertension, dyslipidemia, and smoking [10]. Interventions were led by the pharmacist alone or in collaboration with other health professionals (e.g., physicians or nurses) and included patient educational interventions, measurement of blood pressure, medication management and feedback to physi-cian, or educational interventions to healthcare profession-als. Heterogeneity in the effect of interventions was expected
and random effects models were used to estimate mean changes in blood pressure [2].
Out of 30 trials included in this review, blood pressure was the outcome in 19 studies including 10,479 patients. Phar-macist interventions were associated with clinically and statistically significant reductions of systolic (-8.1 mmHg [95 % CI -10.2 to -5.9]) (Fig.1) and diastolic blood pressure (-3.8 mmHg [95 % CI -5.3 to -2.3]) (Fig.2) compared with usual care. Nevertheless, a substantial het-erogeneity was observed in the effect of pharmacist inter-ventions for both systolic (I2= 76 %; I2 is a measure of between study heterogeneity not due to chance [2]) and diastolic blood pressure (I2= 85 %). Differences between studies in terms of type and intensity of interventions may explain this heterogeneity [10,11]. We computed PI and CI for the effect estimate of pharmacist interventions on blood pressure (Table1).
For both systolic and diastolic blood pressure, PI was much wider than CI. It is critical to realize that CI and PI do not estimate the same thing [8,9]. CI quantifies the accuracy of the mean and indicates where the mean effect is likely to be [8]. More particularly in our meta-analysis, CI indicates the uncertainty around the estimate of the average effect of pharmacist interventions. Since CI for systolic and CI for diastolic blood pressure do not contain the null value (Table1), we are confident that the effect on blood pressure is on average beneficial [2]. PI quantifies the dispersion (or distribution) of effect estimates of the interventions [8]. It means that in 95 % of cases the true effect of a new and unique study (from the same family of studies assessing the impact of a pharmacist intervention) will fall within the PI values. In our case, the PI contains values close to 0 mmHg for systolic blood pressure and above 0 mmHg for diastolic blood pressure (Table1). This means that, although phar-macists’ interventions are effective on average to decrease
A. Chiolero (&) V. Santschi B. Burnand
Institute of Social and Preventive Medicine (IUMSP), Lausanne University Hospital, Biopoˆle 2, route de la Corniche 10, 1010 Lausanne, Switzerland
e-mail: arnaud.chiolero@chuv.ch; valerie.santschi@gmail.com A. Chiolero V. Santschi R. W. Platt G. Paradis
Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montreal, Canada
B. Burnand
Swiss Cochrane, Lausanne, Switzerland
123
Eur J Epidemiol (2012) 27:823–825 DOI 10.1007/s10654-012-9738-y
blood pressure, some of these interventions may not be effective. On the other hand, each PI also contains largely negative values underlining that some pharmacist inter-ventions could have a very large effect on blood pressure.
In conclusion, CI and PI convey different but comple-mentary information. While our results indicate with high confidence that, on average, pharmacist interventions decrease blood pressure, they should also drive researchers to conduct further studies to determine which type of pharmacist intervention is the most effective to decrease blood pressure.
Conflict of interest There are no conflicts of interest and no specific source of funding.
References
1. Higgins JPT, Green S. Cochrane handbook for systematic sys-tematic reviews of interventions. Chichester: Wiley; 2008. 2. Riley RD, Higgins JP, Deeks JJ. Interpretation of random effects
meta-analyses. BMJ. 2011;342:d549.
3. Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions. Version 5.1.0 [updated March 2011]. The
Cochrane Collaboration 2011. Available from
www.cochrane-handbook.org.
4. Jiang Y, Ben Q, Shen H, Lu W, Zhang Y, Zhu J. Diabetes mellitus and incidence and mortality of colorectal cancer: a systematic review and meta-analysis of cohort studies. Eur J Epidemiol. 2011;26(11):863–76.
5. Wu SH, Liu Z, Ho SC. Metabolic syndrome and all-cause mor-tality: a meta-analysis of prospective cohort studies. Eur J Epi-demiol. 2010;25(6):375–84.
6. Bonovas S, Nikolopoulos G, Filioussi K, Peponi E, Bagos P, Sitaras NM. Can statin therapy reduce the risk of melanoma? A meta-analysis of randomized controlled trials. Eur J Epidemiol. 2010;25(1):29–35.
7. Higgins JP, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. J R Stat Soc Ser A Stat Soc. 2009;172:137–59.
8. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Predic-tion Intervals. In: Borenstein M, editor. IntroducPredic-tion to meta-analysis. Chap. 17. Chichester: Wiley; 2009. p. 127–33. 9. Kelley GA, Kelley KS. Impact of progressive resistance training
on lipids and lipoproteins in adults: another look at a meta-analysis using prediction intervals. Prev Med. 2009;49:473–5.
(-16.10, -0.00) (95% PI)
Overall effect (95% CI)
8 7 10 19 3 16 4 11 9 1 12 17 15 14 6 2 5 13 18 -8.05 (-10.20, -5.91) -3.00 (-3.81, -2.19) -6.00 (-9.35, -2.65) -8.90 (-15.14, -2.66) -10.90 (-14.43, -7.37) -8.70 (-14.31, -3.09) -4.65 (-9.35, 0.05) -12.80 (-17.19, -8.41) WMD (95% CI) [mmHg] -20.00 (-30.47, -9.53) -6.00 (-9.75, -2.25) -12.00 (-21.06, -2.94) -10.10 (-20.62, 0.42) -14.40 (-24.96, -3.84) -6.40 (-12.46, -0.34) -5.50 (-14.04, 3.04) -18.36 (-29.96, -6.76) -11.00 (-18.60, -3.40) -2.00 (-7.60, 3.60) -7.80 (-11.34, -4.26) -4.50 (-10.15, 1.15) 100.00 8.87 7.37 5.11 7.23 5.57 6.28 6.53 2.86 7.05 3.46 2.85 2.83 5.24 3.71 2.48 4.23 5.58 7.22 5.54 -8.05 (-10.20, -5.91) -3.00 (-3.81, -2.19) -6.00 (-9.35, -2.65) -8.90 (-15.14, -2.66) -10.90 (-14.43, -7.37) -8.70 (-14.31, -3.09) -4.65 (-9.35, 0.05) -12.80 (-17.19, -8.41) -20.00 (-30.47, -9.53) -6.00 (-9.75, -2.25) -12.00 (-21.06, -2.94) -10.10 (-20.62, 0.42) -14.40 (-24.96, -3.84) -6.40 (-12.46, -0.34) -5.50 (-14.04, 3.04) -18.36 (-29.96, -6.76) -11.00 (-18.60, -3.40) -2.00 (-7.60, 3.60) -7.80 (-11.34, -4.26) -4.50 (-10.15, 1.15) 100.00 8.87 7.37 5.11 7.23 5.57 6.28 6.53 2.86 7.05 3.46 2.85 2.83 5.24 3.71 2.48 4.23 5.58 7.22 5.54 0 -10 0 +10 Weight [%] -20 -30 Study [mmHg]
Fig. 1 Forest plot of the mean difference in systolic blood pressure with pharmacist care compared with usual care group. The diamond represents confidence interval (CI) around the effect estimate. The prediction interval (PI) is shown with lines extending from the diamond. WMD: weighted mean difference. Data are adapted from Santschi et al. [10]
(-9.99, 2.43) (95% PI)
Overall effect (95% CI)
8 7 10 19 3 16 4 11 9 1 12 17 15 14 6 2 5 13 18 Study -3.78 (-5.30, -2.25) 0.00 (-0.48, 0.48) -2.60 (-4.06, -1.14) -1.10 (-4.56, 2.36) -1.40 (-3.90, 1.10) -5.40 (-9.06, -1.74) -2.48 (-5.20, 0.24) -6.70 (-9.74, -3.66) -11.00 (-16.77, -5.23) -3.00 (-5.01, -0.99) -11.00 (-15.43, -6.57) -6.70 (-12.19, -1.21) -14.90 (-20.37, -9.43) -3.00 (-6.59, 0.59) 2.70 (-3.63, 9.03) -7.02 (-12.69, -1.35) 1.00 (-2.13, 4.13) 0.00 (-5.60, 5.60) -3.60 (-5.68, -1.52) -3.20 (-6.04, -0.36) 100.00 7.46 7.03 5.42 6.25 5.24 6.06 5.79 3.62 6.64 4.60 3.81 3.82 5.31 3.28 3.69 5.71 3.73 6.59 5.96 -3.78 (-5.30, -2.25) 0.00 (-0.48, 0.48) -2.60 (-4.06, -1.14) -1.10 (-4.56, 2.36) -1.40 (-3.90, 1.10) -5.40 (-9.06, -1.74) -2.48 (-5.20, 0.24) -6.70 (-9.74, -3.66) -11.00 (-16.77, -5.23) -3.00 (-5.01, -0.99) -11.00 (-15.43, -6.57) -6.70 (-12.19, -1.21) -14.90 (-20.37, -9.43) -3.00 (-6.59, 0.59) 2.70 (-3.63, 9.03) -7.02 (-12.69, -1.35) 1.00 (-2.13, 4.13) 0.00 (-5.60, 5.60) -3.60 (-5.68, -1.52) -3.20 (-6.04, -0.36) 100.00 7.46 7.03 5.42 6.25 5.24 6.06 5.79 Weight [%] 3.62 6.64 4.60 3.81 3.82 5.31 3.28 3.69 5.71 3.73 6.59 5.96 0 -10 0 [mmHg] -20 WMD (95% CI) [mmHg] +10
Fig. 2 Forest plot of the mean difference in diastolic blood pressure with pharmacist care compared with usual care group. The diamond represents confidence interval (CI) around the effect estimate. The prediction interval (PI) is shown with lines extending from the diamond. WMD: weighted mean difference. Data are adapted from Santschi et al. [10]
Table 1 Effect estimate of pharmacist interventions on systolic and diastolic blood pressure
Mean effect 95 % confidence interval 95 % prediction interval Systolic blood pressure [mmHg] -8.1 -10.2 to –5.9 -16.1 to 0.0 Diastolic blood pressure [mmHg] -3.8 -5.3 to –2.3 -10.0 to ?2.4
Data are pooled from 19 studies including 10,479 patients (adapted from Santschi et al. [10])
824 A. Chiolero et al.
10. Santschi V, Chiolero A, Burnand B, Colosimo AL, Paradis G. Impact of pharmacist care in the management of cardiovascular disease risk factors: a systematic review and meta-analysis of randomized trials. Arch Intern Med. 2011;171:1441–53.
11. Santschi V, Chiolero A, Paradis G, Colosimo AL, Burnand B. Pharmacist interventions to improve cardiovascular disease risk factors in diabetes: a systematic review and meta-analysis of randomized controlled trials. Diabetes Care. 2012, in press.
Meta-analyses 825
