INTRODUCTION
The worldwide effects of coronavirus disease-2019 (COVID-19) still continue in many variations and profoundly affect the entire healthcare system, including the habits of patients (1-4). Most COVID-19 patients recover with outpatient treatmen; however, some develop pneumonia and acute respiratory distress syndrome (ARDS) (5). The incidence of the critical disease has been found to be 5% in all COVID-19 patients and 15% in severe illness (6). On the other hand, as new waves continue to emerge in various regions of the globe, health systems are facing an increasing resource crisis (7). Predicting which patients will have a poor prognosis might aid in resource allocation. In the current COVID-19 phase, scoring systems have emerged as a critical tool for determining which patients need a more aggressive approach and which require a more moderate approach (8,9).
In patients with sepsis or septic shock, the Surviving Sepsis Campaign’s 2021 guideline supports the use of systemic inflammatory response syndrome (SIRS), the quick Sequential Organ Failure Assessment (qSOFA), the National Early Warning score (NEWS), and the Modified Early Warning score (MEWS) (10). However, the 2020 COVID-19 guideline produced as part of the same campaign makes no proposal for a comparable COVID-19 scoring system (11). As a result, further study in this area is necessary. Despite the availability of several traditional and new scoring methods, their applicability in patients with COVID-19 has been debated (12). While only a few of these systems have been subjected to external validation, most have reported insufficient results in enough patients (12). COVID-GRAM is a scoring system established to identify patients with COVID-19 who need invasive mechanical ventilation in the intensive care unit (ICU) and to predict death (13). This scoring method considers ten distinct criteria and categorizes individuals as having a low, moderate, or high risk of developing a serious disease. Another scoring method, the 4C mortality score, was validated using data from 35,463 patients gathered from 260 hospitals in collaboration with the International Severe Acute Respiratory and Emerging Infections Consortium and the World Health Organization (WHO) Clinical Characterization Protocol (14). Numerous researches have shown the usefulness of both rating systems.
The purpose of this retrospective observational research was to compare the effectiveness of the qSOFA, SIRS, MEWS, and NEWS scoring systems, which are recommended in the sepsis guidelines, to the COVID-GRAM and 4C mortality scoring systems, which were designed particularly for COVID-19. Our secondary objective was to ascertain which of these scoring systems was superior to the others and to examine their practical applicability.
METHODS
The Ethical Committee of the University of Health Sciences Türkiye, Bakırköy Dr. Sadi Konuk Training and Research Hospital (decision no: 2021-23-26, date: 06.12.2021) and the Turkish Ministry of Health authorized this study. The research was planned as a retrospective, observational, cohort study at a single site.
Setting
This investigation was undertaken at Prof. Dr. Murat Dilmener Emergency Hospital, which was constructed by the Turkish Ministry of Health during the COVID-19 pandemic era. During the pandemic phase, the hospital takes around 110 ambulances per day and treats approximately 1,000 outpatients with COVID-19. The research period was from November 1, 2021, to December 5, 2021. Due to the retrospective nature of the study, the requirement for informed consent was waived; however, informed consent about the risks of COVID-19 and all treatment modalities (including cardiopulmonary resuscitation) was obtained from all patients or their varices at their first visit. Additionally, all individual information has been securely protected (by unlinking identifying information from the main dataset) and made available to researchers only. All the data were analyzed anonymously.
Protocol
The research comprised patients aged 18 years and older who had a positive real-time reverse transcription-polymerase chain reaction (RT-PCR) test on a nasopharyngeal swab sample and had thoracic computed tomography (CT). The following exclusion criteria were used: a) unavailable or missing medical data, b) pregnancy, c) repeated hospitalizations within 15 days, and d) a prior lung infection or surgery, tuberculosis history, or imaging results consistent with COVID-19 pneumonia. All patients admitted to the pandemic hospital had the RT-PCR test. The participants were chosen using the process of thorough case analysis. The admission to the inpatient ward or ICU was determined using the Turkish Ministry of Health’s COVID-19 Diagnosis and Treatment Guideline and the WHO’s criteria for critical and severe disease. Thus, severe sickness was defined as the presence of severe clinical symptoms of pneumonia (fever, cough, dyspnea, and rapid breathing) in addition to one of the following criteria: respiration rate of more than 30 breaths per min, significant respiratory distress, or oxygen saturation lower than 90%. Critical illness was defined as the presence of ARDS or respiratory failure necessitating ventilation, sepsis, or septic shock.
The research gathered data from the hospital's automation system. Two separate doctors submitted data concurrently using a standard data form, and the results were compared using blinded selection. The research eliminated these patients with missing or erroneous data. For patient selection, the whole case analysis process was used. All metrics (blood pressure, vital signs, and CT results) were gathered from the patient’s first visit to the emergency room. Scores were determined using the data form for each of the six distinct scoring systems examined in the research. Analyses were conducted in accordance with these computations. The duration of the disease was defined as the time period from the beginning of symptoms (fever, malaise, and cough) and presentation to the emergency room. On thoracic CT, pneumonia was diagnosed due to the presence of lung infiltration. The measurements and variables of the scoring systems are summarized in Table 1.
Statistical Analysis
Visual (histograms and probability graphs) and analytical (Kolmogorov-Smirnov/Shapiro-Wilk tests) approaches were used to determine the variables' compliance to the normal distribution. The premise of high normalcy was violated in terms of age, scoring system scores, vital parameters, and laboratory data. The Mann-Whitney U test was used to compare mortality across groups for numerical data and the chi-square test (or Fisher’s Exact test, if applicable) for categorical data. The median and interquartile range (25th-75th percentile) values for numerical data were provided, while categorical data were expressed as frequencies and percentages. After computing the scores for each patient, the receiver operating characteristic (ROC) curve analysis and area under the curve (AUC) values were calculated to determine the scoring systems’ overall effectiveness in predicting 28-day mortality. The ideal threshold was determined for each score using the Youden index. In additionally, the sensitivity and specificity, positive and negative probability ratios, positive and negative predictive values, and accuracy of each score were calculated at the ideal threshold. The Hanley-McNeil test was used to examine the significance of the variations in the AUC values of the scores: Under the ROC curve, areas with a threshold Z ratio of 1.96 were judged distinct. Statistical analysis was performed with IBM SPSS Statistics for Windows, Version 26.0 (Armonk, New York), and MedCalc Statistical Software, Version 19.0.6 (MedCalc Software bvba, Ostend, Belgium; 2019). Statistical significance was defined as a p-value of.05 or less.
Outcomes
The primary outcome measure was each scoring system’s diagnostic accuracy in predicting 28-day mortality. The secondary outcome was to compare the effectiveness of scoring systems in predicting 28-day mortality.
RESULTS
This study included data from 846 patients who met the inclusion and exclusion criteria. The median age of the patients included in the research was 49 (36-75) years, and 46.3% (n=392) of the patients were male. The median duration of sickness was 8 (6-10) days. The detection rate of pneumonia was 85.1% (n=720). The hospitalization rate was 49.6% (n=420), the admission rate to the ICU was 7.4% (n=63), and the 28-day mortality rate was 5.7% (n=48). Among hospitalized patients, the median length of stay was 9 (6-13) days.
The survival group had a median age of 47 (36-63) years, whereas the non-survivor group had a median age of 72 (65-89) years (Table 1). Among the vital signs, oxygen saturation (p<0.001), diastolic blood pressure (p=0.030), heart rate (p=0.045), and respiratory rate (p<0.001) were substantially different between the survivor and non-survivor group; however, neither systolic blood pressure or temperature was significantly different (Table 2). A statistically significant difference in white blood cell, neutrophil-lymphocyte ratio, urea, lactate dehydrogenase, direct bilirubin, and C-reactive protein levels was identified between the survivor and mortality groups. Table 2 includes information on the study population. Except for chronic kidney disease and malignancy, the mortality group had significantly higher rates of coronary artery disease (p<0.001), congestive heart failure, (p=0.002), chronic obstructive pulmonary disease (p=0.002), chronic neurological disease (p=0.001), hypertension (p=0.001), and diabetes (p=0.009).
The ROC analysis was used to determine the scoring system's diagnostic accuracy in predicting 28-day mortality (Table 3, Figure 1). All scores have statistically significant AUC values (AUC >0.75). The COVID-GRAM (0.935) and 4C mortality (0.922) systems had the greatest AUC values for the 28-day mortality prediction, whereas SIRS (0.756) and MEWS had the lowest AUC values (0.805). The sensitivity, specificity, positive and negative probability ratios, positive and negative predictive values, and accuracy of ideal clinical thresholds are shown in Table 3, and comparisons of AUC values of scoring systems are presented in Table 4.
DISCUSSION
COVID-GRAM, 4C mortality, and NEWS were shown to be the most effective scoring systems for early prediction of death and ICU admission in COVID-19 (AUC >0.9). Additionally, qSOFA, MEWS, and SIRS were shown to be effective. While COVID-GRAM had the greatest AUC, NEWS showed the greatest sensitivity, and qSOFA demonstrated the greatest specificity. COVID-19 urgently requires the implementation of suitable grading methods for the identification of severely sick patients. COVID-GRAM is a ten-parameter scoring system (15). COVID-GRAM was shown to be a significant predictor of mortality and the requirement for intubation in validation trials [odds ratio: 4.16, 95% confidence interval (CI): 1.8-9.5] (15). COVID-GRAM seems to be a scoring system that may be used to forecast critically sick individuals considering the findings of our investigation (AUC: 0.935, 95% CI: 0.916-0.951). This scoring system, however, excludes indicators such as D-dimer ferritin, creatinine, and age that are used to predict death in COVID-19 (16). As a result, a new grading system based on these biomarkers may be more successful. On the other side, to use a scoring system such as COVID-GRAM, enough medical resources must be available. Additionally, the ideal score differs by area, because various resources are accessible in different regions of the globe during a pandemic. This poses a disadvantage in terms of COVID-broad GRAM’s adoption. As a consequence, in order for the pandemic to cease, the disease’s influence must reduce or disappear globally. Another successful scoring method that we discovered in our research is the 4C mortality scoring system, which comprises eight factors. It has previously been shown to be a good predictor of death in a derivation cohort of 35,463 patients (AUC: 0.79 and 95% CI: 0.78-0.79) and a validation cohort of 22,361 patients (AUC: 0.79 and 95% CI: 0.78-0.79). (AUC: 0.77, 95% CI: 0.76-0.77) (17). Additionally, this scoring system was shown to be a substantial predictor of death in our research (AUC: 0.922, 95% CI: 0.902-0.940). The variables employed in this scoring system include commonly accessible blood parameters that are easily determined, which contributes to the system’s accessibility (17).
Clinicians have long used predictive scores to predict prognosis in patients with severe pneumonia, and research is ongoing (18-20). Sepsis caused by COVID-19 pneumonia has been included in the WHO’s pandemic definition criteria for critical patients. The Surviving Sepsis Campaign’s 2021 guideline shows a strong case of using SIRS, NEWS, or MEWS in patients with sepsis or septic shock rather than qSOFA (10). In 2020, this campaign produced a separate COVID-19 recommendation but made no reference to the effectiveness of any early warning system for this condition (11). SIRS involves blood parameters, but NEWS, MEWS, and qSOFA scores may be simply computed using Glasgow coma scale and vital signs readily available in practically any emergency room. On the other hand, due to the minimal number of factors and scorecards, the qSOFA score is the simplest to compute. All of these early warning systems were shown to be effective in predicting COVID-19-associated mortality in our research (AUC >0.75). NEWS was determined to be better than the other three scoring systems in specific. Additionally, whereas 4C mortality, COVID-GRAM, and SIRS all need laboratory testing, qSOFA, MEWS, and NEWS do not (21). Early diagnosis of severely sick patients is one of the most potent tools in doctors’ arsenals when it comes to fighting pandemics. The optimal strategy would be to combine all existing scoring methods to create a single basic scoring system that produces findings with minimal variability and is useful in large population studies. COVID-19 is a pandemic virus with new variations emerging daily and presenting with various clinical symptoms and prognoses. As a result, no scoring system can be confidently stated to be optimal without knowing what the future holds. However, when we evaluated data from a pandemic hospital, we discovered that COVID-GRAM was the most effective scoring method for predicting COVID-19-related death. Thus, until the findings of novel variations or substantial clinical studies are acquired, this grading method may be used with confidence.
The study’s retrospective design is a significant weakness. On the other hand, while we focused on scoring systems recommended by the Surviving Sepsis Campaign for patients with sepsis patients and those developed specifically for COVID-19, there are numerous additional parameters, including biomarkers and imaging findings, that are used to detect critical patients in COVID-19. Additionally, our findings reflect our region’s shifting worldwide incidence of the illness throughout the continuing epidemic, as well as diverse patient features. Additionally since the information was derived from a pandemic hospital, it included patients with a moderate to bad prognosis, particularly because many patients with a poor prognosis were sent to our center from other hospitals. Finally, our findings may have been influenced by the removal of patients whose data could not be obtained. The best scoring method will be determined via more thorough multicenter trials that integrate all scores used in COVID-19 for mortality and prognosis prediction.
CONCLUSION
According to our results, COVID-GRAM, when available, can be the first-choice scoring system in the effective prediction of mortality associated with COVID-19. However, in the presence of limited medical resources, NEWS would also provide reliable data for this purpose.
ETHICS
Ethics Committee Approval: The Ethical Committee of the University of Health Sciences Türkiye, Bakırköy Dr. Sadi Konuk Training and Research Hospital (decision no: 2021-23-26, date: 06.12.2021) and the Turkish Ministry of Health authorized this study.
Informed Consent: Due to the retrospective nature of the study, the requirement for informed consent was waived; however, informed consent about the risks of COVID-19 and all treatment modalities (including cardiopulmonary resuscitation) was obtained from all patients or their varices at their first visit.
Authorship Contributions
Surgical and Medical Practices: H.A., H.D., M.Ö.E., Concept: H.A., H.D., Design: H.A., H.D., M.Ö.E., Data Collection or Processing: H.A., H.D., M.Ö.E., Analysis or Interpretation: H.A., M.Ö.E., Literature Search: H.A., M.Ö.E., Writing: H.A., H.D., M.Ö.E.
Conflict of Interest: No conflict of interest was declared by the authors.
Financial Disclosure: The authors declared that this study received no financial support.