Article
Reference
Development and validation of outcome prediction models for aneurysmal subarachnoid haemorrhage: the SAHIT multinational
cohort study
JAJA, Blessing N R, et al . & SAHIT collaboration
Abstract
OBJECTIVE: To develop and validate a set of practical prediction tools that reliably estimate the outcome of subarachnoid haemorrhage from ruptured intracranial aneurysms (SAH).
JAJA, Blessing N R, et al . & SAHIT collaboration. Development and validation of outcome prediction models for aneurysmal subarachnoid haemorrhage: the SAHIT multinational cohort study. BMJ , 2018, vol. 360, no. 8137, p. j5745
DOI : 10.1136/bmj.j5745 PMID : 29348138
Available at:
http://archive-ouverte.unige.ch/unige:126329
Disclaimer: layout of this document may differ from the published version.
Appendix: Supplementary tables [posted as supplied by author]
Table A: Excluded studies and the reasons for the exclusion
Study N Description Reason for Exclusion
Model Development
MAPS 228 The Matrix and Platinum Science (MAPS) trial compares the effectiveness of Matrix2 polyglycolic/polylactic acid biopolymer–modified coils with bare metal coils.
Data was not provided on Fisher grade of SAH clot; WFNS grade; and the outcomes of interest (mRS and GOS score).
LEEDS 117 Prospective single-center study from the University of Leeds, UK, to investigate neurocognitive outcomes following SAH
No data on premorbid history of hypertension, aneurysm size and location, and the GOS score DURHAM 105 Prospective single-center study from
the Durham University, UK, to investigate quality of life and neuro- behavioural outcomes following SAH
No data on premorbid history of hypertension, aneurysm size and the GOS score.
CHICAGO 75 Consecutive hospital series from the University of Chicago, USA.
No data elements on the key predictor variable (WFNS grade) and GOS score
Model Validation
ALISAH 47 The Albumin in Subarachnoid
Hemorrhage (ALISAH) Multicenter Pilot Clinical Trial was conducted at 6 North American centers to investigate the safety and tolerability of 25%
human albumin in patients with SAH.
We considered the study insufficiently powered for any meaningful validation analysis of the SAHIT prediction models. For example, there was no death among the study subjects.
CARAT 1010 The Cerebral Aneurysm Rerupture After Treatment (CARAT) study is a
ambidirectional cohort study of all patients with ruptured intracranial aneurysms treated with coil
embolization or surgical clipping at 9 high-volume centers in the United States from 1996 to 1998 to identify risk factors for rerupture after treatment of intracranial aneurysms.
Despite the large cohort, we were unable to use the dataset because data was unavailable on the key predictor – WFNS grade. Clinical severity was assessed according to the Hunt and Hess scale;
unfortunately, there is no validated procedure for imputing the latter grading scheme for the WFNS grade or vice versa.
Also, the outcome (GOS score) was assessed at discharge rather than at a definitive end point (e.g. 3 or 6 or 12 months post ictus).
Table B: Performance indices at leave-on-study-out cross validation (internal-external validation)
Core model Neuroimaging model Full model
Mortality R
2(%) CIL Slope R
2(%) CIL Slope R
2(%) CIL Slope CONS-I 5 -0 . 81 0 . 66 4 -0 . 74 0 . 60 4 -0 . 66 0 . 60
IHAST 11 -0.35 1.28 10 -0.28 1.05 10 -0.15 1.15
IMASH 10 -0.30 0.66 11 -0.17 0.68 14 -0.15 0.65
ISAT 11 -2.14 1.26 14 -1.96 1.38 14 -1.80 1.43
MASH 12 0.04 0.60 12 -0.01 0.64 18 0.13 0.75
TIRILAZAD 18 0.20 0.76 20 0.23 0.78 27 -0.13 0.72
D-SAT 28 0.24 1.00 27 0.15 0.97 25 0.45 1.04
SHOP 42 0.20 1.41 34 0.04 1.18 41 0.11 1.15
Unfavourable
CONS-I 15 0.41 0.76 17 0.53 0.82 15 0.48 0.79
IHAST 16 -0.30 1.23 17 -0.23 1.16 16 -0.13 1.19
IMASH 31 0.28 1.18 33 0.30 1.28 34 0.31 1.27
ISAT 11 0.48 0.80 11 0.55 0.79 12 0.62 0.68
MASH 25 0.25 0.79 24 0.16 0.84 28 0.17 0.90
TIRILAZAD 27 0.01 0.85 28 0.08 0.89 31 -0.04 0.86
D-SAT 37 -0.14 1.03 38 -0.17 1.06 37 0.00 1.11
SHOP 46 -0.09 1.20 40 -0.06 1.06 42 -0.13 1.04
CIL: Calibration-in-the-large, numbers closer to zero indicates better calibration. Slope:
Recalibration slope, numbers closer to 1 indicates better calibration.
Table C: Performance indices in the external validation cohorts
Core model Neuro model Full model
Mortality R
2(%) CIL Slope R
2(%) CIL Slope R
2(%) CIL Slope
BRANT 20 1.14 1.35 - - - - - -
EPO 7 0.10 0.60 6 0.38 0.55 20 0.09 0.79
Kurashiki 15 -1.26 0.87 18 -1.01 0.92 11 -0.92 0.66
SMH 35 0.06 1.17 31 0.21 1.07 38 0.55 1.23
SWISS-SOS 31 0.42 1.00 32 0.59 1.02 35 0.51 0.92
UTRECHT 11 0.05 0.70 13 0.24 0.78 12 0.51 0.79
Pooled 22 0.28 0.86 23 0.46 0.88 24 0.47 0.81
Unfavourable
BRANT 24 0.22 1.50 - - - - - -
EPO 26 0.49 1.14 0.27 0.38 1.13 25 0.30 0.93
Kurashiki 35 -0.38 1.19 37 -0.28 1.21 35 -0.27 1.18
SMH 49 -0.60 1.59 50 -0.69 1.54 52 -0.55 1.64
SWISS-SOS 32 -0.41 1.05 0.32 -0.45 1.04 35 -0.50 1.04
UTRECHT 32 -0.14 1.23 32 -0.18 1.23 31 -0.09 1.24
Pooled 30 -0.23 1.06 31 -0.25 1.07 31 -0.27 1.04
CIL: Calibration-in-the-large, numbers closer to zero indicates better calibration. Slope:
Recalibration slope, numbers closer to 1 indicates better calibration.
Table D: Results of test of potentially meaningful interaction effects Interaction term p-value for
Likelihood ratio test of interaction effect
P value for test of equality of AUC for functional outcome
P-value for test of equality of AUC for mortality outcome
Age by WFNS grade 0.007 0.56 0.22
Age by Hypertension 0.03 0.44 0.61
Aneurysm size by treatment 0.63 0.46 0.65
Aneurysm location by treatment
0.0001 0.08 0.27
P-values are uncorrected. AUC: Area under the receiver operator characteristics curve. Given the very large size of the study population there is the potential for type 1 error and it is difficult what sense to make of the interaction terms that are significant
Table E: Intercepts and coefficients for the different prediction models
Predictor Mortality Unfavourable outcome
Core Neuroimaging Full Core Neuroimaging Full
Intercept -4.918 -5.475 -5.350 -3.703 -4.175 -4.122 Age (per 10
years)
0.32 0.30 0.27 0.034 0.032 0.031
Hypertension 0.327 0.346 0.344 0.268 0.277 0.273
WFNS 1 - - - -
2 0.707 0.676 0.687 0.688 0.602 0.598 3 1.393 1.352 1.273 1.448 1.360 1.321 4 1.803 1.699 1.669 1.723 1.600 1.580
5 2.786 2.578 2.404 2.565 2.399 2.300
Fisher 1 - - - -
2 - -0.008 0.072 - 0.310 0.349
3 - 0.470 0.497 - 0.729 0.750
4 - 0.323 0.487 - 0.854 0.931
Location: ACA - - - - -
ICA - 0.220 0.222 - -0.105 -0.109
MCA - -0.100 -0.027 - -0.266 -0.247
PCA - 0.473 0.318 - 0.032 -0.033
Size: ≤ 12 mm - - - - -
13 – 24 mm - 0.658 0.481 - 0.222 0.136
≥ 25 mm - 1.178 0.370 - 0.529 0.131
Treatment:
Clipping
- - - - -
Coiling - - -0.390 - - -0.177
None - - 1.543 - - 0.842
ACA: Anterior cerebral aneurysms, including anterior communicating artery; ICA: Internal cerebral aneurysms;
MCA: Middle cerebral aneurysms; PCA: Posterior circulation aneurysms.
METHODS:
In all studies, a very small proportion of patients re-bled and some had the aneurysm revisited, the exception was the CONSCIOUS I trial cohort.
