source:
External Validation of Models Predicting Survival After Ruptured Abdominal Aortic Aneurysm Repair
EJVES Jan2015
What models are available nowadays, and how accurate they are?
Three models are been tested: updated Glasgow Aneurysm Score (GAS), the Vancouver scoring system, the Edinburgh Ruptured Aneurysm Score (ERAS), and the Hardman index
449 patients in ten hospitals with a RAAA (intervention between 2004 and 2011)
The updated GAS score – calculated with the formula: age (years) + 7 for cardiac comorbidity (defined as previous history of myocardial infarction, cardiac surgery, angina pectoris or arrhythmia) + 10 for cerebrovascular comorbidity (defined as previous history of stroke or transient ischemic attack) + 17 for shock (defined as an in hospital systolic blood pressure <80 mmHg) + 14 for renal insufficiency (defined as a pre-operative serum creatinine >160 μmol/L) + 7 for OR
predicted mortality rate (%) =
and the performance of GAS is as follows:
The Vancouver score – calculated with the formula: age (years)*0.062 + loss of consciousness (yes = 1/no = −1)*1.14 + cardiac arrest (yes = 1/no = −1)*0.6
predicted mortality rate (%) =
and the performance of Vancouver is as follows:
The ERAS score – calculated with the formula: +1 for best recorded in hospital Glasgow coma scale (GCS) <15, +1 for in hospital systolic blood pressure <90 mmHg, +1 for pre-operative hemoglobin level <5.6 mmol/L. A score of 0 or 1 corresponded with a predicted death rate of 30%, a score of 2 with a predicted death rate of 50%, and a score of 3 with a predicted death rate of 80%.
The Hardman index – calculated with the formula: +1 for age >76 years, +1 for in hospital loss of consciousness, +1 for a pre-operative serum creatinine >190 μmol/L, +1 for pre-operative serum hemoglobin level <5.6 mmol/L, +1 for electrocardiographic (ECG) ischemia (defined as ST segment depression greater than 1 millimeter or an associated T wave change determined by a senior cardiologist [RJGP]). A score of 3 or more corresponded with a predicted death rate of 100%.
OVERALL RESULTS:
The AUC of the updated GAS was 0.71 (95% confidence interval [CI] 0.66–0.76), of the Vancouver score was 0.72 (95% CI 0.67–0.77), and of the ERAS was 0.58 (95% CI 0.52–0.65).
After recalibration, predictions by the updated GAS slightly overestimated the death rate, with a predicted death rate 60% versus observed death rate 54% (95% CI 44–64%).
After recalibration, predictions by the Vancouver score considerably overestimated the death rate, with a predicted death rate 82% versus observed death rate 62% (95% CI 52–71%).
Performance of the Hardman index could not be assessed on discrimination and calibration, because in 57% of patients electrocardiograms were missing.
Decision making
The decision to withhold intervention in patients with a RAAA can be very difficult. Only extremely reliable models can be useful in clinical decision making and in identifying patients in whom withholding intervention might be considered.
For this purpose, a cut-off value for the predicted death rate was set at ≥95%. If the death rate was to be predicted accurately at 95%, the number needed to treat (NNT) would be 20. This cut-off value is arbitrary and could also have been 90% (NNT of 10) or 99% (NNT of 100). Different cut-off values can be used depending on the clinical situation.
None of the prediction models met the criterion of identifying patients in whom to withhold intervention. This disappointing conclusion is in agreement with previous validation studies.21, 22 and 23
Currently, the prediction models have insufficient accuracy to evaluate the chances of successful intervention and future studies should focus on improvement towards this aim. The usefulness of current prediction models lies in case mix comparisons between hospitals, and in a tailored prognosis for patients and relatives.