01 March 2021: Database Analysis
Comparison of Prognostic Value Among 4 Risk Scores in Patients with Acute Coronary Syndrome: Findings from the Improving Care for Cardiovascular Disease in China-ACS (CCC-ACS) ProjectJieleng Huang12BCEF, Xuebiao Wei23CD, Yu Wang2BE, Mei Jiang2BC, Yingwen Lin2BC, Zedazhong Su2BC, Peng Ran2B, Yingling Zhou2G, Jiyan Chen2AG*, Danqing Yu12AG
Med Sci Monit 2021; 27:e928863
BACKGROUND: Accurate risk assessment and prospective stratification are of great importance for treatment of acute coronary syndrome (ACS). However, the optimal risk evaluation systems for predicting different type of ACS adverse events in Chinese population have not been established.
MATERIAL AND METHODS: Our data were derived from the Improving Care for Cardiovascular Disease in China-ACS (CCC-ACS) Project, a multicenter registry program. We incorporated data on 44 750 patients in the study. We compared the performance of the following 4 different risk score systems with regard to prediction of in-hospital adverse events: the Global Registry for Acute Coronary Events (GRACE) risk score system; the age, creatinine and ejection fraction (ACEF) risk score system, and its modified version (AGEF), and the Canada Acute Coronary Syndrome (C-ACS) risk assessment system.
RESULTS: Admission AGEF risk score was a better prognosis index of potential for in-hospital mortality for patients with ST segment elevation myocardial infarction (STEMI) than GRACE risk score (AUC: 0.845 vs 0.819, P=0.012), ACEF (AUC: 0.845 vs 0.827, P=0.014), C-ACS (AUC: 0.845 vs 0.767, P<0.001). In patients with non-ST segment-elevation acute coronary syndrome (NSTE-ACS), there was no statistically significant difference between the GRACE risk scale and AGEF (AUC: 0.853 vs 0.832, P=0.140) for in-hospital death.
CONCLUSIONS: AGEF risk score showed a non-inferior utility compared with the other 3 scoring systems in estimating in-hospital mortality in ACS patients.
Keywords: acute coronary syndrome, percutaneous coronary intervention, Prognosis, Risk Assessment, Aged, Cardiovascular Diseases, Coronary Angiography, Heart Disease Risk Factors, Hospital Mortality, Middle Aged, Prospective Studies, quality improvement, Registries, Risk Factors
Despite remarkable improvements in the treatment of acute coronary syndromes (ACS), the mortality rate is still poor, at 5–10% according to some reports [1–3]. The risk of further cardiovascular complications following ACS is substantial . As such, ACS is a pivotal public health issue throughout the world. Therefore, accurate risk assessment and prospective stratification are of great importance for clinical management of ACS.
Clinical manifestations, electrocardiograms, biochemical analyses, and other quantifiable factors have been used to determine risk and management options for patients with ACS. A number of models and scores of varying degrees of complexity have been used in various studies to identify patients at high risk. Current European Society of Cardiology (ESC) clinical guidelines and the American College of Cardiology Foundation/American Heart Association (ACC/AHA) advocate use of Global Registry for Acute Coronary Events (GRACE) risk scores for risk assessment stratification [5–7]. Many studies have demonstrated the accuracy of GRACE risk scores for prediction of ACS-related mortality in hospital and during follow-up after discharge [8–10]. However, this risk model contains numerous independent variables, which limits its utility. Several simple cardiovascular risk scores have been proposed in recent years, including the age, creatinine, and ejection fraction (ACEF) risk score [11–13], AGEF, a modified version of the ACEF score [14,15], and Canada Acute Coronary Syndrome (C-ACS) score . These simplified risk models eliminate “overfitting” of many independent variables. However, these risk scores were originally designed for different purposes. The ACEF was designed to predict in-hospital outcomes and the C-ACS was designed to predict longer-term outcomes, while the AGEF was designed to assess contrast-induced nephropathy. In clinical practice, the use of these scores is often generalized to other ailments, and they are often not used for their original purposes. Furthermore, GRACE, ACEF, AGEF, and C-ACS have not been compared in large patient cohorts in China.
This study aimed to compare the predictive and discriminatory abilities of these 4 risk scores with respect to in-hospital outcomes for ACS patients. Our findings in this study are built upon a collaboration between the American Heart Association (AHA) and Chinese Society of Cardiology (CSC): Improving CCC Project (Care for Cardiovascular Disease in China).
Material and Methods
STUDY DESIGN AND POPULATION:
Data from a multicenter registry project focusing on upgrading the quality of treatment and nursing for ACS patients were used in this study. The study setting and facilities strategy of the CCC project are provided at length in a previous publication . In each hospital, the first 20–30 ACS inpatient cases in each month were consecutively recruited to this study. Clinical information was acquired using a standard data-gathering website (Oracle Clinical Remote Data Capture, Oracle). Patient characteristics, medical histories, symptoms on arrival, in-hospital treatments and procedures, discharge medications, and secondary prevention information were collected. During November 2014 and June 2017, 63 641 patients diagnosed as having ACS from 150 hospitals were enrolled in the project. Of these patients, 44 750 were incorporated in this research after excluding 18 891 (3.3%) patients due to lack available admission serum creatinine data, left ventricular ejection fraction (LVEF) data, and GRACE risk scores (Figure 1).
DEFINITIONS AND RISK SCORES:
ST segment elevation myocardial infarction (STEMI), in line with the 2010 STEMI guideline , was defined as the existence of typical stethalgia and accompanying symptoms lasting ≥30 minutes but <12 hours. In addition, there had to be at least 2 contiguous leads with ST segment elevation ≥1 mm or a new or undetermined duration of left branch bundle block with a ≥2-fold increase in cardiac enzymes (troponin I or T). Non-ST segment elevation (NSTE)-ACS was determined on the basis of the primary discharge diagnosis of non-ST segment elevated myocardial infarction (NSTEMI) and unstable angina. Non-ST segment elevated myocardial infarction ACS was defined in line with the diagnostic and management guidelines published by the CSC . The diagnostic criteria for unstable angina were as follows: (1) ischemic symptoms at rest or variant angina; new-onset (ie, within 1 month) angina; ischemic symptoms became more frequent, severe, or prolonged, or did not respond to nitroglycerin in recent months for patients with stable angina; (2) myocardial ischemia detected by electrocardiogram or other examination; (3) coronary artery stenosis ≥70% with a need for coronary intervention . Hypertension was diagnosed when there was a high blood pressure history, taking antihypertensive medicine, and accompanied by systolic blood pressure (SBP) ≥140 mmHg, or diastolic blood pressure (DBP) ≥90 mmHg. Diabetes was defined as having a history of a diabetes, taking hypoglycemic agents during prior hospitalization, or glycated hemoglobin A1c concentration 6.5% and over at discharge.
GRACE risk scores consist of medical history, findings at hospital presentation, and findings during hospitalization. Components include age, heart rate, Killip class, SBP, cardiac arrest, ST segment deviation, serum creatinine, and cardiac biomarker status . ACEF scores were estimated with the following equation, available in the publication in which the model was defined: Age/EF (%)+1 (if preoperative serum creatinine value >2.0 mg/dL) . AGEF risk scores were estimated with the equation age/EF (%)+1 point for each 10 mL/min decreased in creatinine clearance (CrCl) below 60 mL/min/1.73 m2 (up to 6 points) . LVEF was the ejection fraction value recorded before the index percutaneous coronary intervention (PCI). C-ACS risk scores were assigned based on whether the heart rate exceeded 100 beats per minute, age 75 years and older, systolic blood pressure lower than 100 mm Hg, or Killip grade II–IV . If the answer is yes, 1 point was scored for each item.
To compare differences among the 4 risk scores, we classified patients into tertiles using the data collected in this study. Patients in tertiles I, II, and III were defined as low-, moderate-, and high-risk patient populations, respectively. The study endpoints were all-cause mortality and in-hospital major adverse clinical events (MACEs). Major adverse clinical events were set to any combined with cardiogenic death, recrudescent myocardial infarction, stent thrombogenesis, and apoplexy. In addition, in-hospital major bleeding  was also recorded, which was defined as hemorrhage in brain and retroperitoneum, a 4 g/dL and over reduction in hemoglobin levels, or hemorrhage requiring transfusion and surgical management.
Continuous data are expressed as mean±standard deviations (SD) or median and quartile ranges. Categorical data are exhibited as counts and percentages. The area under the ROC curve (AUC) and 95% confidence intervals (CI) among different risk scores for predicting adverse events were calculated by using receiver an operating characteristic (ROC) model. SPSS software program version 19.0 (SPSS, Inc.; Chicago, IL, USA) for Windows was used for statistical processing in our study. Bilateral
BASELINE CLINICAL CHARACTERISTICS:
After excluding 1948 patients with missing values for serum creatinine, 13 651 patients due to insufficient data for LVEF, and 3292 patients due to lack of GRACE risk scores, 44 750 patients were incorporated in this study, of whom 64.6% were diagnosed with STEMI and 35.4% presented with NSTE-ACS (NSTE-ACS). The average age of the sample population was 62.63±12.39 years, and 75.6% were males. The baseline clinical characteristics of the study population is provided in Table 1. PCI was performed in 10 149 (64.1%) patients in the NSTE-ACS group and 23 757 (82.1%) patients in the STEMI group. The majority of the patients underwent dual antiplatelet treatment with full anticoagulation. During hospitalization, death occurred in 468 patients (1.0%). Major adverse clinical events occurred in 1510 (9.5%) patients during the hospitalization period in the NSTE-ACS group and in 3079 (10.6%) patients in the STEMI group. Major bleeding happened in 846 (2.9%) patients in the STEMI group during hospitalization and 310 (2.0%) patients in the NSTE-ACS group.
Based on the 4 risk scores, the in-hospital death (%) rates for STEMI patients in the low-, moderate-, and high-risk groups were 0.3, 1.2, and 4.2, respectively, based on C-ACS risk score (P<0.001, Figure 2B); the mortality rates were 0.1, 0.5, and 2.8, respectively, based on ACEF risk score (P<0.001, Figure 2B); the mortality rates were 0.1, 0.4, and 2.7, respectively, based on AGEF risk score (P<0.001, Figure 2B); and the mortality rates were 0.2, 0.4, and 2.6, respectively, based on GRACE risk score (P<0.001, Figure 2B). Similar results were observed in NSTE-ACS patients (Figure 2A). Incidence of MACEs and major hemorrhage during hospitalization stratified by different risk scores are shown in Supplemenatry Figures 1 and 2, indicating that the incidences in high-risk patients were significant higher.
RISK MODEL DISCRIMINATION:
Table 2 shows the ROC curve comparison for in-hospital adverse events. The predictive accuracies of the 4 risk scores are presented in Figure 3. In the NSTE-ACS group, the AUCs for in-hospital death were 0.853, 0.827, 0.832, and 0.766 for GRACE, ACEF, AGEF, and C-ACS risk score, respectively (Figure 3A). The C-ACS score exhibited the lowest predictive ability. AUC differences for C-ACS and ACEF, AGEF, and GRACE were 0.060, 0.066, 0.087, respectively. The abilities of GRACE, ACEF, and AGEF risk models to assess in-hospital deaths in the NSTE-ACS group were insignificant (Table 2).
In the STEMI group, the AGEF risk model (AUC=0.845; 95% CI 0.825–0.864; P<0.001, Figure 3B) exhibited better predictive potential for in-hospital mortality than GRACE risk score on admission (AUC=0.819; 95% CI 0.796–0.842; P<0.001, Figure 3B), ACEF risk score (AUC=0.827; 95% CI 0.806–0.849; P<0.001, Figure 3B), or C-ACS risk score (AUC=0.767, 95% CI 0.740, 0.793; P<0.001; Figure 3B). The AUC differences between the C-ACS model and ACEF, AGEF, and GRACE were 0.061, 0.078, 0.053, respectively (Table 2).
Supplementary Figures 3 and 4 summarize the discriminative ability of these 4 risk models to predict MACEs and major hemorrhage. GRACE risk scores exhibited greater predictive power for in-hospital MACEs than the other 3 risk scores in both the STEMI and NSTE-ACS groups. In addition, GRACE and AGEF risk scores exhibited similar discriminative ability for major bleeding.
For mortality, the prognostic power of different scores was compared in patients who did, or did not, undergo PCI. In patients receiving PCI, the C-ACS score exhibited a lower inpatient death discrimination ability than GRACE (AUC: 0.759 vs 0.834, P=0.015), ACEF (AUC: 0.759 vs 0.811, P=0.015), and AGEF (AUC: 0.759 vs 0.827, P=0.014) risk scores. Similar results were observed in patients who did not undergo PCI (Figure 4). Moreover, the predictive power of each risk score system for MACEs and major bleeding were also compared (Table 3, Supplementary Figures 5, 6).
This was a retrospective study based on prospectively collected information. Missing data and confounding factors might have influenced the results. Furthermore, the CCC-ACS project only recruits ACS patients in the highest level of public hospital, and there is no information on patients who died before arriving at the hospital. This may lead to potential selection bias in this cohort study, thereby reducing the in-hospital mortality rates of patients. Thus, the patients and outcomes of this study may not reflect experiences elsewhere in the healthcare system. More precise results need to be validated in a broader range of care settings. In addition, AUC and 95% CI were used to compare predicted values of different scores, but the results also found that ROC curves intersected at some points. Since the size of the overall AUC was compared in this study, and the AGEF risk score had the largest AUC, it must be acknowledged that AGEF risk score may be less sensitive and specific than the other 3 models at some points. Finally, the study lacked follow-up data. Therefore, the predictive capability of the different scores with regard to long-term prognosis was not evaluated.
The predictive and discriminatory abilities of different risk scores with respect to in-hospital clinical outcomes in Chinese ACS patients were compared in our study. In STEMI patients, AGEF was significantly superior to GRACE and ACEF in predicting in-hospital death, while there was no significant difference for the NSTE-ACS group. Overall, AGEF risk score showed a non-inferior utility to the other 3 scoring systems in predicting in-hospital mortality in ACS patients.
FiguresFigure 1. Flow diagram for the selection of the study population. Figure 2. Rates of in-hospital death in the low-, moderate-, and high-risk groups, according to the GRACE, ACEF, AGEF, and C-ACS risk scores. Figure 3. Receiver operating characteristics (ROC) curves showing the discriminative ability of the risk scales for the predictive ability of in-hospital death in patients with NSTE-ACS (A) and STEMI (B). Figure 4. Receiver operating characteristics (ROC) curves showing the discriminative ability of the risk assessments of in-hospital death in patients undergoing PCI (A) or non-PCI (B).
1. Szummer K, Wallentin L, Lindhagen L, Improved outcomes in patients with ST-elevation myocardial infarction during the last 20 years are related to implementation of evidence-based treatments: Experiences from the SWEDEHEART registry 1995–2014: Eur Heart J, 2017; 38(41); 3056-65
2. Rosselló X, Huo Y, Pocock S, Global geographical variations in ST-segment elevation myocardial infarction management and post-discharge mortality: Int J Cardiol, 2017; 245; 27-34
3. Granger CB, Goldberg RJ, Dabbous O, Predictors of hospital mortality in the global registry of acute coronary events: Arch Intern Med, 2003; 163(19); 2345-53
4. Eagle KA, Lim MJ, Dabbous OH, A validated prediction model for all forms of acute coronary syndrome: Estimating the risk of 6-month postdischarge death in an international registry: JAMA, 2004; 291(22); 2727-33
5. Fox KA, Anderson FA, Dabbous OH, Intervention in acute coronary syndromes: Do patients undergo intervention on the basis of their risk characteristics? The Global Registry of Acute Coronary Events (GRACE): Heart, 2007; 93(2); 177-82
6. Fox KA, Carruthers KF, Dunbar DR, Underestimated and under-recognized: the late consequences of acute coronary syndrome (GRACE UK-Belgian Study): Eur Heart J, 2010; 31(22); 2755-64
7. D’Ascenzo F, Biondi-Zoccai G, Moretti C, TIMI, GRACE and alternative risk scores in Acute Coronary Syndromes: A meta-analysis of 40 derivation studies on 216,552 patients and of 42 validation studies on 31,625 patients: Contemp Clin Trials, 2012; 33(3); 507-14
8. Elbarouni B, Goodman SG, Yan RT, Validation of the Global Registry of Acute Coronary Event (GRACE) risk score for in-hospital mortality in patients with acute coronary syndrome in Canada: Am Heart J, 2009; 158(3); 392-99
9. Kao YT, Hsieh YC, Hsu CY, Comparison of the TIMI, GRACE, PAMI and CADILLAC risk scores for prediction of long-term cardiovascular outcomes in Taiwanese diabetic patients with ST-segment elevation myocardial infarction: From the registry of the Taiwan Society of Cardiology: PLoS One, 2020; 15(2); e0229186
10. Komiyama K, Nakamura M, Tanabe K, In-hospital mortality analysis of Japanese patients with acute coronary syndrome using the Tokyo CCU Network database: Applicability of the GRACE risk score: J Cardiol, 2018; 71(3); 251-58
11. Ranucci M, Castelvecchio S, Menicanti L, Risk of assessing mortality risk in elective cardiac operations: Age, creatinine, ejection fraction, and the law of parsimony: Circulation, 2009; 119(24); 3053-61
12. Stähli BE, Wischnewsky MB, Jakob P, Predictive value of the age, creatinine, and ejection fraction (ACEF) score in patients with acute coronary syndromes: Int J Cardiol, 2018; 270; 7-13
13. Andò G, Morabito G, de Gregorio C, Trio O, The ACEF score as predictor of acute kidney injury in patients undergoing primary percutaneous coronary intervention: Int J Cardiol, 2013; 168(4); 4386-87
14. Andò G, Morabito G, de Gregorio C, Age, glomerular filtration rate, ejection fraction, and the AGEF score predict contrast-induced nephropathy in patients with acute myocardial infarction undergoing primary percutaneous coronary intervention: Catheter Cardiovasc Interv, 2013; 82(6); 878-85
15. Capodanno D, Marcantoni C, Ministeri M, Incorporating glomerular filtration rate or creatinine clearance by the modification of diet in renal disease equation or the Cockcroft-Gault equations to improve the global accuracy of the Age, Creatinine, Ejection Fraction [ACEF] score in patients undergoing percutaneous coronary intervention: Int J Cardiol, 2013; 168(1); 396-402
16. Huynh T, Kouz S, Yan AT, Canada Acute Coronary Syndrome Risk Score: A new risk score for early prognostication in acute coronary syndromes: Am Heart J, 2013; 166(1); 58-63
17. Hao Y, Liu J, Liu J, Rationale and design of the Improving Care for Cardiovascular Disease in China (CCC) project: A national effort to prompt quality enhancement for acute coronary syndrome: Am Heart J, 2016; 179; 107-15
18. Guideline for diagnosis and treatment of patients with ST-elevation myocardial infarction: Zhonghua Xin Xue Guan Bing Za Zhi, 2010; 38; 675-90
19. Guidelines for the diagnosis and treatment of non-ST-segment elevation acute coronary syndromes: Zhonghua Xin Xue Guan Bing Za Zhi, 2012; 40; 353-67 [in Chinese]
20. Mathews R, Peterson ED, Chen AY: Am J Cardiol, 2011; 107(8); 1136-43
21. Tang EW, Wong CK, Herbison P, Global Registry of Acute Coronary Events (GRACE) hospital discharge risk score accurately predicts long-term mortality post acute coronary syndrome [published correction appears in Am Heart J, 2007;154(5):851]: Am Heart J, 2007; 153(1); 29-35
22. Fox KA, Dabbous OH, Goldberg RJ, Prediction of risk of death and myocardial infarction in the six months after presentation with acute coronary syndrome: Prospective multinational observational study (GRACE): BMJ, 2006; 333(7578); 1091
23. Brieger D, Fox KAA, Fitzgerald G, Predicting freedom from clinical events in non-ST-elevation acute coronary syndromes: The Global Registry of Acute Coronary Events: Heart, 2009; 95(11); 888-94
24. Luo J, Dai L, Li J, Risk evaluation of new-onset atrial fibrillation complicating ST-segment elevation myocardial infarction: A comparison between GRACE and CHA2DS2-VASc scores: Clin Interv Aging, 2018; 13; 1099-109
25. Hannan EL, Racz M, Culliford AT, Risk score for predicting in-hospital/30-day mortality for patients undergoing valve and valve/coronary artery bypass graft surgery: Ann Thorac Surg, 2013; 95(4); 1282-90
26. Wykrzykowska JJ, Garg S, Onuma Y, Value of age, creatinine, and ejection fraction (ACEF score) in assessing risk in patients undergoing percutaneous coronary interventions in the ‘All-Comers’ LEADERS trial: Circ Cardiovasc Interv, 2011; 4(1); 47-56
27. Di Serafino L, Borgia F, Maeremans J, The age, creatinine, and ejection fraction score to risk stratify patients who underwent percutaneous coronary intervention of coronary chronic total occlusion: Am J Cardiol, 2014; 114(8); 1158-64
28. Levey AS, Coresh J, Greene T, Using standardized serum creatinine values in the modification of diet in renal disease study equation for estimating glomerular filtration rate [published erratum appears in Ann Intern Med, 2008;149(7):519]: Ann Intern Med, 2006; 145(4); 247-54
29. Stevens LA, Schmid CH, Greene T: Am J Kidney Dis, 2010; 56(3); 486-95
30. Kalaycı A, Oduncu V, Geçmen Ç, A simple risk score in acute ST-elevation myocardial infarction: Modified ACEF (age, creatinine, and ejection fraction) score: Turk J Med Sci, 2016; 46(6); 1688-93
31. Pogorevici A, Citu IM, Bordejevic DA, Caruntu F, Tomescu MC, Canada acute coronary syndrome score was a stronger baseline predictor than age ≥75 years of in-hospital mortality in acute coronary syndrome patients in western Romania: Clin Interv Aging, 2016; 11; 481-88
32. Latif F, Kleiman NS, Cohen DJ, In-hospital and 1-year outcomes among percutaneous coronary intervention patients with chronic kidney disease in the era of drug-eluting stents: A report from the EVENT (Evaluation of Drug Eluting Stents and Ischemic Events) registry: JACC Cardiovasc Interv, 2009; 2(1); 37-45
33. Ix JH, Mercado N, Shlipak MG, Association of chronic kidney disease with clinical outcomes after coronary revascularization: The arterial revascularization therapies study (ARTS): Am Heart J, 2005; 149(3); 512-19
34. Xu H, Yang Y, Wang C, Association of hospital-level differences in care with outcomes among patients with acute ST-segment elevation myocardial infarction in China: JAMA Netw Open, 2020; 3(10); e2021677
35. Chen JY, He PC, Liu YH, Association of parenteral anticoagulation therapy with outcomes in Chinese patients undergoing percutaneous coronary intervention for non-ST-segment elevation acute coronary syndrome [published erratum appears in JAMA Intern Med, 2019;179(2):280]: JAMA Intern Med, 2019; 179(2); 186-94
36. Chen M, Kan J, Zhang JJ, Improvement of clinical outcome in patients with ST-elevation myocardial infarction between 1999 and 2016 in China: The Prospective, Multicentre Registry MOODY study: Eur J Clin Invest, 2020; 50(2); e13197
28 Nov 2023 : Clinical ResearchImpact of Fluorodeoxyglucose-Positron Emission Tomography/Computed Tomography on Therapeutic Decisions and ...
Med Sci Monit In Press; DOI: 10.12659/MSM.942122
28 Nov 2023 : Clinical ResearchLong-Term Outcomes of Decompression and Grafting in Acute Pathological Proximal Femur Fractures in Children...
Med Sci Monit In Press; DOI: 10.12659/MSM.943031
27 Nov 2023 : Clinical ResearchComparison of Outcomes from Emergency Admissions to a Major Trauma Center in Turkey of 1646 Elderly Patient...
Med Sci Monit In Press; DOI: 10.12659/MSM.942916
Most Viewed Current Articles
13 Nov 2021 : Clinical ResearchAcceptance of COVID-19 Vaccination and Its Associated Factors Among Cancer Patients Attending the Oncology ...
Med Sci Monit 2021; 27:e932788
30 Dec 2021 : Clinical ResearchRetrospective Study of Outcomes and Hospitalization Rates of Patients in Italy with a Confirmed Diagnosis o...
Med Sci Monit 2021; 27:e935379
14 Dec 2022 : Clinical ResearchPrevalence and Variability of Allergen-Specific Immunoglobulin E in Patients with Elevated Tryptase Levels
Med Sci Monit 2022; 28:e937990
08 Mar 2022 : Review articleA Review of the Potential Roles of Antioxidant and Anti-Inflammatory Pharmacological Approaches for the Man...
Med Sci Monit 2022; 28:e936292