Introduction

Nearly 14% of Coronavirus disease-2019 (COVID-19) patients suffer from acute hypoxic respiratory failure (ARF), which is usually managed by a high nasal flow cannula (HNFC) to restore oxygenation (1). HNFC is a heated circuit that delivers humidified air via a non-invasive nasal cannula. Despite success, a subset of patients on HNFC become non-responders eventually requiring an invasive approach of intubation or mechanical ventilation (MV) (2).

From the very beginning of the COVID-19 pandemic, emergency physicians have been struggling, especially in resource constraint set-ups, to deal with the overwhelming number of COVID-19 patients requiring MV for survival (3,4). Therefore, the challenge is to find new ways to identify patients who are likely to end up in HNFC failure and be weaned by MV (2). In this context, some authors have recently evaluated the ROX index as a predictor of intubation in COVID-19 patients; however, the validation of this tool is subject to further studies (5,6).

The ROX algorithm measures the ratio of oxygen saturation (FiO₂) and predicts the need to intubate and mechanically ventilate patients with ARF. It was first described in 2016 by Roca et al. (5) in a multicenter prospective cohort of 157 patients with ARF on HFNC. The authors found that score <4.88, measured at 12^th hour after the onset of HFNC showed significant risk for the need of invasive treatment (5). These results were also supported by a subsequent study analyzing the diagnostic accuracy of the ROX index in 191 patients with pneumonia treated with HFNC (7). In addition, the superiority of ROX for predicting invasive treatment was reported by a large-scale FLORALI cohort (8).

These findings indicate that the ROX index is an easy-to-use clinical method that could act as an early warning system for HNFC failure in COVID-19-related ARF patients. Additionally, early warning for invasive treatment approaches would allow hospitals to meet the unprecedented demand for ventilators during the COVID-19 surge by implementing an effective resource allocation strategy, thus saving many lives. To the best of our knowledge, the ROX index has not yet been applied in COVID-19 patients; hence, we aimed to validate the diagnostic accuracy of the ROX algorithm for predicting HNFC failure and the need for invasive ventilation (IV) in COVID-19 patients. Furthermore, our secondary goal was to assess the risk of invasive management in patients with different comorbidities.

Materials and Methods

Study Design and Setting

A multicenter prospective cohort was designed after obtaining ethical approval from the Ziauddin University Ethics Review Committee (ERC#1701219FHPAT, date: 10.02.2020). Informed consent was obtained from all participants. The subjects were recently admitted and treated for COVID-19 between March 2020 and December 2021 in critical care units of six major tertiary care setups in district east, Karachi, Pakistan. The demographic data were obtained from the principal investigator via a research questionnaire, while the medical records and history were used to determine pre-existing conditions.

Patients

A total of 123 new cases with laboratory confirmed COVID-19 infection were selected for investigation. The selection procedure was carried out using the non-probability consecutive sampling technique. We employed very stringent inclusion criteria and recruited only those patients who were subjected to HNFC on admission based on their oxygenation parameters. The patients were placed on HFNC if SpO₂ fell below 93% and FiO₂~30% with evident respiratory distress despite being oxygenated. For HFNC, FiO₂ to sustain SpO₂ above 92% was initiated at a flow rate of 50 to 60 L/min. For the protective measures, a heat moisture exchange filter was used between the nasal cannula and the non-IV (NIV) device, and a protective pad was applied on the nasal area to avoid skin wear and tear. The exclusion criteria were based on all those who were already intubated or mechanically ventilated on admission, those on conventional oxygen therapy, and patients with mild to moderate COVID-19 disease.

The patients were followed for intubation or MV until discharge from the hospital or death. The criteria for intubation on HNFC subjects were FiO₂ >60% and signs of respiratory distress or other organic dysfunctions. The intubated patients were reassessed every 30 to 60 min for improvement in ventilatory parameters. To minimize microbial transmission, a high-efficiency particulate arrestable filter was employed on the exhalation output of the MV.

Procedure

The ROX parameters SpO₂, FiO₂, and respiratory rate were measured and documented at time 0, i.e., on admission and at the 2^nd, 6^th, and 12^th hour after the onset of HFNC. The final ROX score was calculated for each of the aforementioned hours using the formula: SpO₂/FiO₂*, %/respiratory rate, breaths/min. The ROX index was interpreted as described by Blez et al. (9) and Roca et al. (5) i.e. ≥4.88 for lower risk of intubation while <4.88 suggest high risk of HNFC failure.

Statistical Analysis

Data were assessed for normality using the Shapiro-Wilk test. For analysis, we developed the control and case groups on the basis of the outcome. The control group comprised patients maintained on HNFC, and the case group comprised patients with the outcome of HNFC failure requiring IV (i.e., intubation/MV). Continuous variables are presented in terms of median or interquartile ranges, and categorical variables are presented as frequencies (%) and absolute numbers (n). The sensitivity, specificity, positive predictive value, and negative predictive value were calculated to determine the diagnostic accuracy of the ROX index at time 0 (on admission), 2^nd, 6^th, and 12^th hour, while the strength of association was assessed by Fisher’s exact test. The risk of invasive treatment in patients with comorbidities was analyzed, and the statistical power of significance was estimated for each independent variable. All statistical analyses were performed using MedCalc software version 20, and p value ≤0.05 was considered statistically significant.

Results

Of the 123-COVID-19 cohorts, 49 (39%) patients suffered failure of HFNC and were classified as cases. All these cases were managed by endotracheal intubation (ETI) and MV. The overall mortality was 19 (15%)in the case group. Our baseline data indicated that there were more males (73%) than females (27%) in our study. The age bracket for all subjects ranged from 33 to 84 years, and the median age was recorded as 66 years.

On the analysis of ROX index, we found the range in the overall population at time 0 was 3.5-5.7, at 2^nd hour was 3.2-6.5, at 6^th hour was 3.0-8.2, and at the 12^th hour was 2.9-10.1. The medians for these ranges were 3.80, 4.90, 4.80, and 5.0 for 0, 2, 6, and 12 h readings, respectively. Moreover, the ROX data monitored at different hours (0, 2, 6 and 12) were all statistically related to the cases. We observed a consistent rise in the diagnostic accuracy of the ROX index to predict IV at subsequent time intervals. The highest diagnostic value of the ROX index was recorded for 12^th hour with a sensitivity of 91.8% and a specificity of 89.1%. The validity indices were also supported by receiver operating curve analysis with the maximum area under the curve (AUC; 0.905) for the ROX index at 12^th hour (Figure 1). Table 1 shows the diagnostic value of ROX index in discriminating HFNC failure at 0, 2^nd, 6^th and 12^th hour.

Our clinical data showed that most of the studied patients had some comorbidity (83%). The main comorbidities associated with intubation in COVID-19 patients were increased age, hypertension, diabetes mellitus, chronic kidney disease, asthma, and chronic obstructive pulmonary disease. Besides cardiovascular diseases and malignancy, all comorbidities showed a significant increase in the risk for the need for invasive treatment at 95% confidence interval (CI). However, chronic kidney disease showed the highest risk (2.52, 95% CI: 1.75-3.63) for the invasive approach. Table 2 presents the risk estimates for invasive oxygenation and the association of comorbidities.

Discussion

HNFC has gained popularity in recent years as a first-line treatment for COVID-19-related ARF. Owing to the positive impact of HNFC on gas exchange parameters, masking of the deteriorating clinical picture has become increasingly common, resulting in delayed decision for interventional treatments. As a consequence, fatal outcomes are often encountered in critically ill COVID-19 patients (10). Hence, early warning tools for the indication of failure of NIV have been an area of research interest. The ROX index is one such tools that has shown promising results (6).

The ROX index was first described by Roca et al. (5) in the pre-COVID era as a method to predict HNFC failure in pneumonia patients. The authors recommended that a score below 4.88 is an indication for a high risk of intubation, whereas a score higher than 4.88 suggests a good outcome. Subsequently, a multiethnic study from Europe successfully validated the ROX score of 4.88 as a cut-off to predict outcome (11). Both investigators used readings taken at 0, 2, 6, and 12 h from the start of conventional NIV treatment to calculate final ROX scores (5,11). In contrast, Rodriguez et al. (12) calculated ROX scores for intervals up to 48 h and found higher values as cut-off values for stratification of HFNC patients.

Among early COVID-19 studies, Blez et al. (9) evaluated the diagnostic performance of the ROX index using a predefined cut-off >4.88 for success of HNFC in COVID-19 patients and found 81% sensitivity. In later studies, Suliman et al. (13) documented that the ROX index is an independent predictor of IV with 90.2% sensitivity and 75% specificity in patients suffering from severe COVID-19 infection on admission. Additionally, in a multicenter study, Vega et al. (6) reported that the ROX index at 12 h is the best predictor of intubation or MV (AUC of 0.7916) in COVID-19 patients; however, the study found a higher threshold value (<5.99) for stratification and suggested that a cut-off of less than 4.88 could be used as an indicator for HNFC failure in non-COVID patients. In another study, Luis et al. (14) found that ROX index 4.88 both at 2 and 12 h increased the risk of ventilatory failure and poor outcome. Nonetheless, a significant statistical relationship between the ROX index and intubation was concordant in all studies, indicating the external validity of ROX at any time interval during management (6,9,13,14).

In this study, we used a pre-described cut-off <4.88 as the predictor of invasive treatment and found good accuracy of the ROX index for discriminating HNFC failure in COVID-19 patients. This finding is consistent with those of Vega et al. (6) and Luis et al. (14). However, in a later analysis, the authors reported a higher threshold for discrimination than that described earlier (6). Nevertheless, investigators included patients with moderate ARF who were treated outside the intensive care unit (ICU), which might be the reason for high ROX scores contributing to an elevated threshold (6). Also, similar to Vega et al. (6), we noted greater diagnostic performance of ROX at 12^th and 6^th hour. As most intubation occurs after 12 h of HNFC, we suggest that the ROX index could be an ideal tool to assess the need for IV in COVID-19 patients.

In addition to diagnostic performance, our secondary goal was to assess the likelihood of intubation with comorbidities. We found that underlying hypertension and diabetes were the leading conditions in our series. Similar observations were made in an early Chinese study and a meta-analysis estimating the prevalence of comorbidities in COVID-19 patients (15,16). We also noted that most patients who required IV belonged to the elderly age group. In general, older age is a predictor of adverse outcomes in viral infections, and this might be the case in COVID-19 disease (17).

Our statistical data indicated that older adults with different underlying health conditions are at a higher risk of severe COVID-19 outcomes. In concordance, the preliminary literature also showed that almost 70% of the COVID-19 patients on ventilator had some form of comorbidity (18). In addition, pooled data from the United States suggested that people with underlying health conditions have significant odds of developing a severe form of COVID-19 disease (19). Among the underlying conditions, we found that almost all comorbidities were significantly associated with a high risk of intubation; however, chronic kidney disease poses the greatest threat to intubation during hospitalization. In contrast, most authors found that the risk of invasive management is highest in diabetic patients (16). We suggest two reasons for our observations. First, patients with end-stage renal disease are typically immunocompromised, leading to infections with severe outcomes. Alternatively, the low prevalence of chronic renal conditions in previous cohorts might have contributed to this discrepancy.

Study Limitations

This research has both strengths and limitations. To the best of our knowledge, this is the first study in Pakistan to evaluate the ROX index in a clinical setting, providing hospitals with an opportunity to implement a simple and non-invasive method for better care and resource allocation. Moreover, the multicentric design allows diverse strata of population to be analyzed in this cohort. Furthermore, we followed prospective sampling, which has the advantage of examining multiple effects and is also tailored to control confounders, thus minimizing bias. Another important aspect of this study is to assess the impact of comorbidity on the risk of invasive treatment modality besides validating the ROX index.

In addition, the present research has some limitations. First, apart from the ETI, other invasive approaches such as; orotracheal intubation were not considered as these modalities are less frequently offered in ICUs and none of the subjects underwent any of these procedures. Second, we characterized the comorbidities based on history and recent medical records; however, fresh investigations to analyze the current status of disease was not done. In addition, we did not inquire about the duration of underlying diseases for correlation. Lastly, the sample size could have increased further to provide more external validity to our observations.

Conclusion

In summary, this multicenter study indicated that the ROX index could prove vital in guiding clinical decisions regarding IV in critically ill COVID-19 patients. Indeed, our observations suggested that ROX score especially at 12^th and 6^th hour from the start of HNFC, could anticipate the need for invasive management with high accuracy. Furthermore, pre-existing comorbidities have a significant impact on adverse outcomes in COVID-19 patients; however, renal diseases, hypertension, and diabetes are the major risk factors for clinical deterioration requiring IV.

Ethics

Ethics Committee Approval: A multicenter prospective cohort was designed after obtaining ethical approval from the Ziauddin University Ethics Review Committee (ERC#1701219FHPAT, date: 10.02.2020).

Informed Consent: Informed consent was obtained from the study participants.

Authorship Contributions

Surgical and Medical Practices - Concept - Design - Data Collection or Processing - Analysis or Interpretation - Literature Search - Writing: N.T., S.H., F.A.B.

Conflict of Interest: No conflict of interest was declared by the authors.

Financial Disclosure: The authors declared that this study received no financial support.