Abstract
Aim
In patients with community-acquired pneumonia, we observed that the impact of the age variable commonly used in Pneumonia Severity scores to predict mortality risk is disproportionate within the pneumonia severity index (PSI). In this context, our study aimed to evaluate and compare the effectiveness of pneumonia severity scores, adjusted for demographic factors such as age and gender, in predicting intensive care unit (ICU) admission and mortality, compared to PSI and CURB-65.
Materials and Methods
In 2019, data on patients diagnosed with pneumonia in the emergency department were obtained through a retrospective review. The CURB-65, PSI, age-modified PSI (mPSI), and gender- modified PSI (gmPSI) scores were calculated. The predictive performance of these scores for ICU admission and mortality was statistically analyzed.
Results
A total of 363 patients were included in the study, of whom 205 (56.4%) were male. Additionally, comparisons of the newly developed age-mPSI and gmPSI with the standard PSI and CURB-65 demonstrated no statistically significant difference in predicting 30-day mortality or ICU admission rates (p>0.05).
Conclusion
The findings of this study indicate that the newly developed mPSI and gmPSI scoring systems yield results comparable to those of the standard PSI scoring system.
Introduction
Community-acquired pneumonia (CAP) is a common disease with a lifetime prevalence of 20-30% in developing countries and 3-4% in developed countries (1). The mortality rate among hospitalized and diagnosed patients has been reported to vary between 4% and 21% in different settings (2). It has been reported that this rate is higher, even up to 50%, among patients admitted to the intensive care unit (ICU) (3). Treatment in patients with CAP is determined according to the general condition and prognosis of the patient, and the follow-up and treatment plan of the patient varies according to the severity of the disease (4). Therefore, various scoring systems are used to determine the prognosis. The most commonly used assessment tools are CURB-65 (confusion, blood urea nitrogen, respiratory rate, blood pressure) and Pneumonia Severity index (PSI). CURB-65 is a classification system used to predict the risk of mortality and is frequently preferred in emergency departments because it is easy to remember and contains few criteria (3). PSI is a scoring system developed by Fine et al. (5) that includes 20 parameters and is divided into 5 categories according to 30-day prognosis predictions. CURB-65 and the PSI are widely used to predict mortality in CAP (6). However, there are many studies reporting that PSI performs better than CURB-65 in predicting mortality (7, 8).
Nevertheless, clinical applications have revealed certain limitations of this scoring system within the context of emergency departments (8, 9). For instance, one study demonstrated that mortality and ICU admission rates were lower among patients categorized as PSI Class I/II but were paradoxically higher among those with a CURB-65 score of 1 (10). Furthermore, evidence suggests that approximately 27% of patients in PSI Class I-III may necessitate ICU admission and 40% of patients classified as low-risk were hospitalized (11, 12). Although elevated scores in such systems are typically indicative of more severe illness, it is important to note that younger to middle-aged patients with low scores may also require ICU-level care. In particular, the age-related risk factor incorporated into the PSI scoring system may result in disproportionately high scores among older patients while potentially underestimating severity in younger individuals.
Many studies in the literature have shown that sex hormones play an important role in the immune response and that women have a stronger immune response than men. In general, women have a stronger response to pathogens and produce higher levels of interferon and antibodies (13, 14). We aimed to assess the influence of age and sex on the performance of the PSI scoring system in this study. In particular, the impact of the newly developed age- and sex-modified, modified-PSI (mPSI) and gender- modified PSI (gmPSI) on clinical outcomes, including mortality and ICU admission rates, was analyzed.
Materials and Methods
This retrospective study was conducted in the emergency department of a tertiary hospital located in a provincial center that experiences approximately 420,000 patient visits annually. It received ethical approval from the University of Health Sciences Türkiye, Dışkapı Yıldırım Beyazıt Training and Research Hospital Clinical Research Ethics Committee (decision number: 103/13, date: 25.01.2021). Our research was designed in accordance with the Standards for Reporting of Diagnostic Accuracy Studies statement (15).
Information was extracted from electronic medical records and patient charts. A retrospective chart review was conducted by two emergency medicine specialists, each possessing a minimum of three years of experience.
Between January 1, 2019, and December 31, 2019, patients were diagnosed with pneumonia in the emergency medicine clinic. All patients over the age of 18 years with a diagnosis of pneumonia were included in the study. The inclusion criteria were based on ICD-10 codes (International Statistical Classification of Diseases, J15 and its subcodes, J15.8, J15.9) retrieved from the hospital’s data system. Individuals with a history of hospitalization within the preceding seven days, a diagnosis of cystic fibrosis, tuberculosis, or Human immunodeficiency virus, a documented history of concomitant pulmonary embolism, or a history of trauma at the time of admission, as well as those with incomplete data in their records, were excluded from the study. Additionally, patients with high mortality at the time of admission and those diagnosed with pneumonia were not included in the study.
Patients diagnosed with pneumonia were initially stratified according to the CURB-65 score, the PSI, and the criteria for ICU admission (1). The PSI scoring system was subsequently modified by excluding the age and gender variables. To adjust for age-related risk, 10 points were allocated to patients aged 65 years and older, whereas no points were assigned to those under 65 years. This revised scoring model was termed the mPSI.
In the calculation of the gmPSI, female patients aged 65 years and older were assigned 10 points, while male patients in the same age group were allocated 20 points. No points were assigned to either male or female patients under the age of 65.
The predictive performance of the scoring systems (PSI, mPSI, and gmPSI) in estimating hospitalization and mortality outcomes was systematically evaluated. Patients were subsequently categorized into two distinct age groups: younger adults (18-64 years) and older adults (≥65 years), enabling a comprehensive comparison between these cohorts.
Statistical Analysis
Statistical analyses were conducted using the SPSS 22.0 software package. The normality of continuous variables was assessed through the Kolmogorov-Smirnov test, supplemented by descriptive statistical measures of Skewness and Kurtosis.
Descriptive statistics were presented as frequencies and percentages for categorical variables, while continuous variables were summarized using medians and (interquartile range 25th-75th percentiles). For the analysis of differences in numerical variables between two independent groups, the Mann-Whitney U test was employed. Comparisons of proportions between independent groups were conducted using chi-square analysis.
ROC analyses were conducted to evaluate the predictive efficacy of continuous variables in estimating 30-day mortality and the need for intensive care. A p-value of <0.05 was considered the threshold for statistical significance.
Results
A total of 450 patients were initially enrolled in the study; however, 87 were excluded due to incomplete data. Among the remaining participants, 205 (56.4%) were male. Hospitalization rates did not differ significantly between male and female patients (p=0.248).
The stratified pneumonia scores based on 30-day mortality and ICU requirements for all patients are presented in Table 1. Patients who died or required ICU admission were older and exhibited higher scores across all measurements (p<0.05 for all comparisons) (Table 1).
ROC analysis was conducted to identify the threshold values of pneumonia scores distinguishing between deceased and surviving patient groups. Among the scores, PSI demonstrated the highest area under the curve (AUC), while CURB-65 exhibited the lowest. All AUC values were statistically significant (p<0.001 for all comparisons) (Table 2, Figure 1A).
In the ROC analysis conducted to determine the threshold values of pneumonia scores differentiating patients admitted to the ICU from those not admitted, the mPSI demonstrated the highest AUC, while the CURB-65 showed the lowest. All AUC values were statistically significant (p<0.001 for all comparisons) (Table 3, Figure 1B).
The best cut-off values, along with the corresponding sensitivity and specificity of the PSI, mPSI, and gmPSI scores for predicting mortality and ICU admission, are presented in Table 4. The sensitivity and specificity values for all three scoring systems were found to be comparable (Table 4).
Patients were categorized into two age groups: 18-64 years (young) and ≥65 years (geriatric). In the ROC analysis conducted to identify the threshold values of pneumonia scores for predicting mortality within the young age group, the PSI demonstrated the highest AUC, whereas CURB-65 exhibited the lowest. All AUC values were statistically significant (p<0.001) (Table 5, Figure 2A).
In the ROC analysis conducted to determine the threshold values of pneumonia scores for predicting mortality in the geriatric group, PSI and mPSI exhibited the highest AUC values, while CURB-65 demonstrated the lowest (Table 6, Figure 2B).
In the ROC analysis conducted to identify the threshold values of pneumonia scores for predicting ICU admission in the young patient group, the highest AUC value was observed for the mPSI, while the lowest was associated with the PSI (Table 7, Figure 3A).
In the ROC analysis conducted to determine the threshold values of pneumonia scores for intensive care admission in the geriatric population, the highest AUC value was observed for the PSI score, while the lowest was associated with the CURB-65 score (Table 8, Figure 3B).
Discussion
Treatment, prognosis, and care location decisions are often facilitated by scoring systems, which, despite inherent challenges and limitations, serve as valuable tools in the clinical decision-making process. These systems primarily aim to minimize unnecessary hospital admissions while ensuring appropriate patient placement and management. The PSI, a widely utilized scoring system, comprises 20 variables encompassing patient age, history of nursing home residence, comorbid conditions, vital signs, laboratory results, chest radiographic findings, and oxygenation status. The assessment is conducted in two distinct stages. In this scoring system, the age variable is directly assigned based on the numerical age in male patients, while, for female patients, it is calculated as age minus 10. Consequently, this approach may result in low scores for young patients with severe respiratory failure in the absence of comorbidities, while generating high scores for very elderly patients. Such discrepancies can create confusion for clinicians and potentially lead to inappropriate hospitalization and treatment decisions.
The literature contains numerous studies evaluating the sensitivity and specificity of CURB-65 and PSI scores in predicting mortality and the necessity for ICU admission (16-18). In a study conducted in Iran, CURB-65 demonstrated higher accuracy in predicting mortality and the need for intensive care hospitalization in patients with CAP (17). In contrast, studies conducted by Akça et al. (17) and Satici et al. (18) found that PSI exhibited superior performance in predicting mortality.
In a retrospective study involving 1,902 patients, Zhang et al. (19) categorized the patients into three age groups: 18-64 years, 65-84 years, and ≥85 years. Although PSI Class III is considered a low-risk category for mortality, they observed a mortality rate of 7.5% in the 18-64 age group, compared to 2.1% in the 65-84 age group and 2.6% in those aged ≥85 years. Accordingly, they suggested that PSI Class III may not be appropriate for classification as a low mortality risk in younger patients. They reported that while PSI was more sensitive than CURB-65 in predicting mortality, it misclassified the low-risk group among younger patients, and the discriminative power of both CURB-65 and PSI diminished with increasing age (19). In a prospective study involving 987 patients, Chen et al. (20) categorized the patients into age groups as defined in the study and observed mortality rates of 20% in the 18-64 age group, 11% in the 65-84 age group, and 11.5% among those aged 85 years and over. They demonstrated that 77.8% of the patients who died in the young adult group were categorized as low-risk, a significantly higher proportion compared to those categorized as low-risk in the elderly and very old groups. They attributed the poor performance of the PSI score in elderly patients to the significant influence of the age variable and proposed that applying a non-linear adjustment to the age variable would enhance the PSI score’s accuracy. Simonetti et al. (21) also observed in their prospective study that the effectiveness of PSI and CURB-65 in predicting mortality diminished with advancing age. In our clinical practice, we found it necessary to critically evaluate the age component of the PSI score. Our observations indicated that some very elderly patients did not require intensive care despite having a high PSI score, whereas certain younger patients required intensive care despite a low PSI score.
In our study, a novel scoring system was developed and implemented to address the confusion caused by existing scoring systems for patients requiring hospitalization in the ward or ICU, or those who can be managed as outpatients. Based on the data obtained from these scoring systems, we found that the highest AUC value for predicting 30-day mortality was associated with PSI, while the lowest was observed with CURB-65.
Comparisons according to the patients’ age groups indicated that the highest AUC value for 30-day mortality in the 18-64 age group was related to PSI, and the highest AUC value for the need for ICU admission was related to mPSI. We found that the highest AUC value for 30-day mortality in the geriatric age group was related to both PSI and mPSI, and the highest AUC value for the need for ICU admission was related to PSI. PSI scoring was more significant than mPSI scoring in terms of the need for intensive care admission in the geriatric age group. However, the results of both scores were similar in the geriatric age group in terms of mortality. In the 18-64 age group, the significance of mPSI scoring in intensive care admissions led to the conclusion that the age factor had an effect on determining the need for admission. However, after a more detailed examination of the findings, we found that there was no significant difference between the scoring systems.
Study Limitations
Our study has a retrospective design, and the consequent data loss is one of our limitations. There may be missing data due to incorrect diagnosis code records. Due to the retrospective nature of the study, certain variables like confusion may have been under-documented in the medical records, which could have resulted in lower CURB-65 and PSI scores. There are limitations in our study because patient data were collected in a single center and the study was conducted with a specific sample size.
Conclusion
In our study, we have demonstrated that the mPSI and gmPSI scoring systems, can reduce hospital burden and additional costs by decreasing unnecessary intensive care admissions, similar to PSI. Additionally, they exhibit comparable sensitivity to PSI, which is used as the standard for predicting early mortality. Therefore, we believe that its application in both elderly and younger patients could facilitate a more objective assessment.
