Research

Comparison of COVID-GRAM, 4C Mortality, qSOFA, SIRS, NEWS, and MEWS in Predicting Mortality in COVID-19

10.4274/BMJ.galenos.2023.2022.7-10

  • Hakan Aydın
  • Halil Doğan
  • Mehmet Özgür Erdoğan

Received Date: 27.07.2022 Accepted Date: 02.03.2023 Med J Bakirkoy 2023;19(1):111-118

Objective:

Mortality prediction methods are still controversial about coronavirus disease-2019 (COVID-19) pneumonia. This study aimed to compare the efficacy of the the quick Sequential Organ Failure Assessment, systemic inflammatory response syndrome (SIRS), Modified Early Warning score (MEWS), National Early Warning score, 4C mortality, and COVID-GRAM critical illness risk score (COVID-GRAM), scoring systems in predicting 28-day mortality in adult patients with COVID-19.

Methods:

This single-center, retrospective, observational cohort study included patients presenting to a pandemic hospital between November 2021 and December 2021. Inclusion criteria were patients aged 18 years or older, patients with positive reverse transcription-polymerase chain reaction test, and thoracic computed tomography imaging. The receiver operating characteristic analysis was performed to examine the diagnostic accuracy of the investigated scoring systems in predicting 28-day mortality. Statistical analyses were performed using the SPSS and MedCalc software packages. A p-value of <0.5 was considered statistically significant.

Results:

The study was conducted in 846 patients. The median age of the patients included in the study was 49 (36-75) years, and the rate of male patients was 46.3% (n=392). The rate of pneumonia detection was 85.1% (n=720). The hospitalization rate was 49.6% (n=420), the admission rate to the intensive care unit was 7.4% (n=63), and the 28-day mortality rate was 5.7% (n=48). The highest area under the curve (AUC) values for 28-day mortality prediction was obtained from COVID-GRAM (AUC: 0.935) and 4C mortality (AUC: 0.922) scores, while the lowest AUC values were calculated in SIRS (AUC: 0.756) and MEWS (AUC: 0.805).

Conclusion:

According to our results, COVID-GRAM may be the first-choice scoring system in the emergency department for predicting the 28-day mortality associated with COVID-19.

Keywords: COVID-19, scoring systems, COVID-GRAM, 4C mortality, qSOFA

INTRODUCTION

The worldwide effects of coronavirus disease-2019 (COVID-19) still continue in many variations and profoundly affect the entire healthcare system, including the habits of patients (1-4). Most COVID-19 patients recover with outpatient treatmen; however, some develop pneumonia and acute respiratory distress syndrome (ARDS) (5). The incidence of the critical disease has been found to be 5% in all COVID-19 patients and 15% in severe illness (6). On the other hand, as new waves continue to emerge in various regions of the globe, health systems are facing an increasing resource crisis (7). Predicting which patients will have a poor prognosis might aid in resource allocation. In the current COVID-19 phase, scoring systems have emerged as a critical tool for determining which patients need a more aggressive approach and which require a more moderate approach (8,9).

In patients with sepsis or septic shock, the Surviving Sepsis Campaign’s 2021 guideline supports the use of systemic inflammatory response syndrome (SIRS), the quick Sequential Organ Failure Assessment (qSOFA), the National Early Warning score (NEWS), and the Modified Early Warning score (MEWS) (10). However, the 2020 COVID-19 guideline produced as part of the same campaign makes no proposal for a comparable COVID-19 scoring system (11). As a result, further study in this area is necessary. Despite the availability of several traditional and new scoring methods, their applicability in patients with COVID-19 has been debated (12). While only a few of these systems have been subjected to external validation, most have reported insufficient results in enough patients (12). COVID-GRAM is a scoring system established to identify patients with COVID-19 who need invasive mechanical ventilation in the intensive care unit (ICU) and to predict death (13). This scoring method considers ten distinct criteria and categorizes individuals as having a low, moderate, or high risk of developing a serious disease. Another scoring method, the 4C mortality score, was validated using data from 35,463 patients gathered from 260 hospitals in collaboration with the International Severe Acute Respiratory and Emerging Infections Consortium and the World Health Organization (WHO) Clinical Characterization Protocol (14). Numerous researches have shown the usefulness of both rating systems.

The purpose of this retrospective observational research was to compare the effectiveness of the qSOFA, SIRS, MEWS, and NEWS scoring systems, which are recommended in the sepsis guidelines, to the COVID-GRAM and 4C mortality scoring systems, which were designed particularly for COVID-19. Our secondary objective was to ascertain which of these scoring systems was superior to the others and to examine their practical applicability.


METHODS

The Ethical Committee of the University of Health Sciences Türkiye, Bakırköy Dr. Sadi Konuk Training and Research Hospital (decision no: 2021-23-26, date: 06.12.2021) and the Turkish Ministry of Health authorized this study. The research was planned as a retrospective, observational, cohort study at a single site.

Setting

This investigation was undertaken at Prof. Dr. Murat Dilmener Emergency Hospital, which was constructed by the Turkish Ministry of Health during the COVID-19 pandemic era. During the pandemic phase, the hospital takes around 110 ambulances per day and treats approximately 1,000 outpatients with COVID-19. The research period was from November 1, 2021, to December 5, 2021. Due to the retrospective nature of the study, the requirement for informed consent was waived; however, informed consent about the risks of COVID-19 and all treatment modalities (including cardiopulmonary resuscitation) was obtained from all patients or their varices at their first visit. Additionally, all individual information has been securely protected (by unlinking identifying information from the main dataset) and made available to researchers only. All the data were analyzed anonymously.

Protocol

The research comprised patients aged 18 years and older who had a positive real-time reverse transcription-polymerase chain reaction (RT-PCR) test on a nasopharyngeal swab sample and had thoracic computed tomography (CT). The following exclusion criteria were used: a) unavailable or missing medical data, b) pregnancy, c) repeated hospitalizations within 15 days, and d) a prior lung infection or surgery, tuberculosis history, or imaging results consistent with COVID-19 pneumonia. All patients admitted to the pandemic hospital had the RT-PCR test. The participants were chosen using the process of thorough case analysis. The admission to the inpatient ward or ICU was determined using the Turkish Ministry of Health’s COVID-19 Diagnosis and Treatment Guideline and the WHO’s criteria for critical and severe disease. Thus, severe sickness was defined as the presence of severe clinical symptoms of pneumonia (fever, cough, dyspnea, and rapid breathing) in addition to one of the following criteria: respiration rate of more than 30 breaths per min, significant respiratory distress, or oxygen saturation lower than 90%. Critical illness was defined as the presence of ARDS or respiratory failure necessitating ventilation, sepsis, or septic shock.

The research gathered data from the hospital's automation system. Two separate doctors submitted data concurrently using a standard data form, and the results were compared using blinded selection. The research eliminated these patients with missing or erroneous data. For patient selection, the whole case analysis process was used. All metrics (blood pressure, vital signs, and CT results) were gathered from the patient’s first visit to the emergency room. Scores were determined using the data form for each of the six distinct scoring systems examined in the research. Analyses were conducted in accordance with these computations. The duration of the disease was defined as the time period from the beginning of symptoms (fever, malaise, and cough) and presentation to the emergency room. On thoracic CT, pneumonia was diagnosed due to the presence of lung infiltration. The measurements and variables of the scoring systems are summarized in Table 1.

Statistical Analysis

Visual (histograms and probability graphs) and analytical (Kolmogorov-Smirnov/Shapiro-Wilk tests) approaches were used to determine the variables' compliance to the normal distribution. The premise of high normalcy was violated in terms of age, scoring system scores, vital parameters, and laboratory data. The Mann-Whitney U test was used to compare mortality across groups for numerical data and the chi-square test (or Fisher’s Exact test, if applicable) for categorical data. The median and interquartile range (25th-75th percentile) values for numerical data were provided, while categorical data were expressed as frequencies and percentages. After computing the scores for each patient, the receiver operating characteristic (ROC) curve analysis and area under the curve (AUC) values were calculated to determine the scoring systems’ overall effectiveness in predicting 28-day mortality. The ideal threshold was determined for each score using the Youden index. In additionally, the sensitivity and specificity, positive and negative probability ratios, positive and negative predictive values, and accuracy of each score were calculated at the ideal threshold. The Hanley-McNeil test was used to examine the significance of the variations in the AUC values of the scores: Under the ROC curve, areas with a threshold Z ratio of 1.96 were judged distinct. Statistical analysis was performed with IBM SPSS Statistics for Windows, Version 26.0 (Armonk, New York), and MedCalc Statistical Software, Version 19.0.6 (MedCalc Software bvba, Ostend, Belgium; 2019). Statistical significance was defined as a p-value of.05 or less.

Outcomes

The primary outcome measure was each scoring system’s diagnostic accuracy in predicting 28-day mortality. The secondary outcome was to compare the effectiveness of scoring systems in predicting 28-day mortality.


RESULTS

This study included data from 846 patients who met the inclusion and exclusion criteria. The median age of the patients included in the research was 49 (36-75) years, and 46.3% (n=392) of the patients were male. The median duration of sickness was 8 (6-10) days. The detection rate of pneumonia was 85.1% (n=720). The hospitalization rate was 49.6% (n=420), the admission rate to the ICU was 7.4% (n=63), and the 28-day mortality rate was 5.7% (n=48). Among hospitalized patients, the median length of stay was 9 (6-13) days.

The survival group had a median age of 47 (36-63) years, whereas the non-survivor group had a median age of 72 (65-89) years (Table 1). Among the vital signs, oxygen saturation (p<0.001), diastolic blood pressure (p=0.030), heart rate (p=0.045), and respiratory rate (p<0.001) were substantially different between the survivor and non-survivor group; however, neither systolic blood pressure or temperature was significantly different (Table 2). A statistically significant difference in white blood cell, neutrophil-lymphocyte ratio, urea, lactate dehydrogenase, direct bilirubin, and C-reactive protein levels was identified between the survivor and mortality groups. Table 2 includes information on the study population. Except for chronic kidney disease and malignancy, the mortality group had significantly higher rates of coronary artery disease (p<0.001), congestive heart failure, (p=0.002), chronic obstructive pulmonary disease (p=0.002), chronic neurological disease (p=0.001), hypertension (p=0.001), and diabetes (p=0.009).

The ROC analysis was used to determine the scoring system's diagnostic accuracy in predicting 28-day mortality (Table 3, Figure 1). All scores have statistically significant AUC values (AUC >0.75). The COVID-GRAM (0.935) and 4C mortality (0.922) systems had the greatest AUC values for the 28-day mortality prediction, whereas SIRS (0.756) and MEWS had the lowest AUC values (0.805). The sensitivity, specificity, positive and negative probability ratios, positive and negative predictive values, and accuracy of ideal clinical thresholds are shown in Table 3, and comparisons of AUC values of scoring systems are presented in Table 4.


DISCUSSION

COVID-GRAM, 4C mortality, and NEWS were shown to be the most effective scoring systems for early prediction of death and ICU admission in COVID-19 (AUC >0.9). Additionally, qSOFA, MEWS, and SIRS were shown to be effective. While COVID-GRAM had the greatest AUC, NEWS showed the greatest sensitivity, and qSOFA demonstrated the greatest specificity. COVID-19 urgently requires the implementation of suitable grading methods for the identification of severely sick patients. COVID-GRAM is a ten-parameter scoring system (15). COVID-GRAM was shown to be a significant predictor of mortality and the requirement for intubation in validation trials [odds ratio: 4.16, 95% confidence interval (CI): 1.8-9.5] (15). COVID-GRAM seems to be a scoring system that may be used to forecast critically sick individuals considering the findings of our investigation (AUC: 0.935, 95% CI: 0.916-0.951). This scoring system, however, excludes indicators such as D-dimer ferritin, creatinine, and age that are used to predict death in COVID-19 (16). As a result, a new grading system based on these biomarkers may be more successful. On the other side, to use a scoring system such as COVID-GRAM, enough medical resources must be available. Additionally, the ideal score differs by area, because various resources are accessible in different regions of the globe during a pandemic. This poses a disadvantage in terms of COVID-broad GRAM’s adoption. As a consequence, in order for the pandemic to cease, the disease’s influence must reduce or disappear globally. Another successful scoring method that we discovered in our research is the 4C mortality scoring system, which comprises eight factors. It has previously been shown to be a good predictor of death in a derivation cohort of 35,463 patients (AUC: 0.79 and 95% CI: 0.78-0.79) and a validation cohort of 22,361 patients (AUC: 0.79 and 95% CI: 0.78-0.79). (AUC: 0.77, 95% CI: 0.76-0.77) (17). Additionally, this scoring system was shown to be a substantial predictor of death in our research (AUC: 0.922, 95% CI: 0.902-0.940). The variables employed in this scoring system include commonly accessible blood parameters that are easily determined, which contributes to the system’s accessibility (17).

Clinicians have long used predictive scores to predict prognosis in patients with severe pneumonia, and research is ongoing (18-20). Sepsis caused by COVID-19 pneumonia has been included in the WHO’s pandemic definition criteria for critical patients. The Surviving Sepsis Campaign’s 2021 guideline shows a strong case of using SIRS, NEWS, or MEWS in patients with sepsis or septic shock rather than qSOFA (10). In 2020, this campaign produced a separate COVID-19 recommendation but made no reference to the effectiveness of any early warning system for this condition (11). SIRS involves blood parameters, but NEWS, MEWS, and qSOFA scores may be simply computed using Glasgow coma scale and vital signs readily available in practically any emergency room. On the other hand, due to the minimal number of factors and scorecards, the qSOFA score is the simplest to compute. All of these early warning systems were shown to be effective in predicting COVID-19-associated mortality in our research (AUC >0.75). NEWS was determined to be better than the other three scoring systems in specific. Additionally, whereas 4C mortality, COVID-GRAM, and SIRS all need laboratory testing, qSOFA, MEWS, and NEWS do not (21). Early diagnosis of severely sick patients is one of the most potent tools in doctors’ arsenals when it comes to fighting pandemics. The optimal strategy would be to combine all existing scoring methods to create a single basic scoring system that produces findings with minimal variability and is useful in large population studies. COVID-19 is a pandemic virus with new variations emerging daily and presenting with various clinical symptoms and prognoses. As a result, no scoring system can be confidently stated to be optimal without knowing what the future holds. However, when we evaluated data from a pandemic hospital, we discovered that COVID-GRAM was the most effective scoring method for predicting COVID-19-related death. Thus, until the findings of novel variations or substantial clinical studies are acquired, this grading method may be used with confidence.

The study’s retrospective design is a significant weakness. On the other hand, while we focused on scoring systems recommended by the Surviving Sepsis Campaign for patients with sepsis patients and those developed specifically for COVID-19, there are numerous additional parameters, including biomarkers and imaging findings, that are used to detect critical patients in COVID-19. Additionally, our findings reflect our region’s shifting worldwide incidence of the illness throughout the continuing epidemic, as well as diverse patient features. Additionally since the information was derived from a pandemic hospital, it included patients with a moderate to bad prognosis, particularly because many patients with a poor prognosis were sent to our center from other hospitals. Finally, our findings may have been influenced by the removal of patients whose data could not be obtained. The best scoring method will be determined via more thorough multicenter trials that integrate all scores used in COVID-19 for mortality and prognosis prediction.


CONCLUSION

According to our results, COVID-GRAM, when available, can be the first-choice scoring system in the effective prediction of mortality associated with COVID-19. However, in the presence of limited medical resources, NEWS would also provide reliable data for this purpose.

ETHICS

Ethics Committee Approval: The Ethical Committee of the University of Health Sciences Türkiye, Bakırköy Dr. Sadi Konuk Training and Research Hospital (decision no: 2021-23-26, date: 06.12.2021) and the Turkish Ministry of Health authorized this study.

Informed Consent: Due to the retrospective nature of the study, the requirement for informed consent was waived; however, informed consent about the risks of COVID-19 and all treatment modalities (including cardiopulmonary resuscitation) was obtained from all patients or their varices at their first visit.

Authorship Contributions

Surgical and Medical Practices: H.A., H.D., M.Ö.E., Concept: H.A., H.D., Design: H.A., H.D., M.Ö.E., Data Collection or Processing: H.A., H.D., M.Ö.E., Analysis or Interpretation: H.A., M.Ö.E., Literature Search: H.A., M.Ö.E., Writing: H.A., H.D., M.Ö.E.

Conflict of Interest: No conflict of interest was declared by the authors.

Financial Disclosure: The authors declared that this study received no financial support.

 


  1. Cucinotta D, Vanelli M. WHO Declares COVID-19 a Pandemic. Acta Biomed 2020;91:157-60.
  2. İlhan B, Bozdereli Berikol G, Aydın H, Arslan Erduhan M, Doğan H. COVID-19 outbreak impact on emergency trauma visits and trauma surgery in a level 3 trauma center. Ir J Med Sci 2022;191:2319-24.
  3. İlhan B, Berikol GB, Doğan H. Impact of COVID-19 Outbreak on Pediatric Trauma Cases in a Tertiary Trauma Center. Med J Bakirkoy 2022;18:70-6.
  4. Aydin H, Doğan H. COVID-19 outbreak impact on discharge against medical advice from the ED: A retrospective study. Am J Emerg Med 2022;57:21-6.
  5. Wu C, Chen X, Cai Y, Xia J, Zhou X, Xu S, et al. Risk Factors Associated With Acute Respiratory Distress Syndrome and Death in Patients With Coronavirus Disease 2019 Pneumonia in Wuhan, China. JAMA Intern Med 2020;180:934-43.
  6. Cummings MJ, Baldwin MR, Abrams D, Jacobson SD, Meyer BJ, Balough EM, et al. Epidemiology, clinical course, and outcomes of critically ill adults with COVID-19 in New York City: a prospective cohort study. Lancet 2020;395:1763-70.
  7. Grasselli G, Pesenti A, Cecconi M. Critical Care Utilization for the COVID-19 Outbreak in Lombardy, Italy: Early Experience and Forecast During an Emergency Response. JAMA 2020;323:1545-6.
  8. Akdur G, Daş M, Bardakci O, Akman C, Sıddıkoğlu D, Akdur O, et al. Prediction of mortality in COVID-19 through combing CT severity score with NEWS, qSOFA, or peripheral perfusion index. Am J Emerg Med 2021;50:546-52.
  9. Liu S, Yao N, Qiu Y, He C. Predictive performance of SOFA and qSOFA for in-hospital mortality in severe novel coronavirus disease. Am J Emerg Med 2020;38:2074-80.
  10. Evans L, Rhodes A, Alhazzani W, Antonelli M, Coopersmith CM, French C, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021. Intensive Care Med 2021;47:1181-247.
  11. Alhazzani W, Møller MH, Arabi YM, Loeb M, Gong MN, Fan E, et al. Surviving Sepsis Campaign: guidelines on the management of critically ill adults with Coronavirus Disease 2019 (COVID-19). Intensive Care Med 2020;46:854-87.
  12. Wynants L, Van Calster B, Collins GS, Riley RD, Heinze G, Schuit E, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ 2020;369:m1328.
  13. Liang W, Liang H, Ou L, Chen B, Chen A, Li C, et al. Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19. JAMA Intern Med 2020;180:1081-9.
  14. Ali R, Qayyum F, Ahmed N, Haroon MZ, Irshad R, Sajjad S, et al. Isaric 4c Mortality Score As A Predictor Of In-Hospital Mortality In Covid-19 Patients Admitted In Ayub Teaching Hospital During First Wave Of The Pandemic. J Ayub Med Coll Abbottabad 2021;33:20-5.
  15. Yildiz H, Yombi JC, Castanares-Zapatero D. Validation of a risk score to predict patients at risk of critical illness with COVID-19. Infect Dis (Lond) 2021;53:78-80.
  16. Steinberg E, Balakrishna A, Habboushe J, Shawl A, Lee J. Calculated decisions: COVID-19 calculators during extreme resource-limited situations. Emerg Med Pract 2020;22:CD1-5.
  17. Knight SR, Ho A, Pius R, Buchan I, Carson G, Drake TMet al. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score. BMJ 2020;370:m3339.
  18. Aydın H, Doğan H, Özüçelik DN, Koçak M, Gül OA. Viral prevalence, clinical profiles and comparison of severity scores for predicting the mortality of adults with severe acute respiratory infections. Turk J Med Sci 2019;49:862-71.
  19. Doğanay F, Ak R. Performance of the CURB-65, ISARIC-4C and COVID-GRAM scores in terms of severity for COVID-19 patients. Int J Clin Pract 2021;75:e14759.
  20. Chu SE, Seak CJ, Su TH, Chaou CH, Tseng HJ, Li CH. Prognostic accuracy of SIRS criteria and qSOFA score for in-hospital mortality among influenza patients in the emergency department. BMC Infect Dis 2020;20:385.
  21. Latten GHP, Polak J, Merry AHH, Muris JWM, Ter Maaten JC, Olgers TJ, et al. Frequency of alterations in qSOFA, SIRS, MEWS and NEWS scores during the emergency department stay in infectious patients: a prospective study. Int J Emerg Med 2021;14:69.