Chest Computed Tomography (CT) Severity Scales in COVID-19 Disease: A Validation Study

Background

The computed tomography (CT) scan of the chest is the primary imaging modality used in the evaluation of patients with SARS-CoV-2 infection [1–18]. Several studies have shown a relationship between the extent of inflammatory lung changes seen in a chest CT scan and the clinical course of a disease and prognosis [19–22]. The quantitative assessment of lung lesions plays a key role in patient triage and therapeutic decision-making processes [19–22]. To date, no standardized and widely used method has been established for the quantitative assessment of pulmonary lesions in COVID-19. During a pandemic, automatic segmentation methods facilitating precise assessment of the volume of inflammatory lesions are not always available. Therefore, studies assessing the prognostic value of CT scans have been mostly based on the use of dedicated scoring scales [19–21].

Kunwei et al [20] first presented the Total Severity Score, which the authors used to assess each of the 5 lobes of both lungs for the presence of inflammatory abnormalities such as mixed ground-glass opacities, ground-glass opacities, and consolidations. Using the Total Severity Score, each lobe is assessed using a range of 0 to 4 points, depending on the percentage of lobe involvement: 0 indicates 0% involvement; 1 indicates 1% to 25%; 2 indicates 26% to 50%; 3 indicates 51% to 75%; and 4 indicates 76% to 100%. The Total Severity Score is then obtained by summing the points of the 5 lobes (range 0 to 20 points).

The Chest CT Score was introduced by Li et al [21]. In their study, both lungs were divided into 5 lobes, with each lobe being assessed individually. Each lobe is assigned a score of 0 to 5, depending on the percentage of lobe involvement: 0 indicates 0% involvement; 1 indicates less than 5%; 2 indicates 5% to 25%; 3 indicates 26% to 49%; 4 indicates 50% to 75%; and 5 indicates more than 75% involvement.

The Chest CT Severity Score was proposed by Yang et al [19]]. The score is an adaptation of a method used in 2005 during the SARS epidemic. The scale makes use of lung opacification as the marker for the extent of lung involvement. Using the Chest CT Severity Score, the 18 segments of both lungs are divided into 20 regions with the posterior apical segment of the left upper lobe being subdivided into posterior and apical segmental regions, and the anteromedial basal segment of the left lower lobe being subdivided into basal and anterior segmental regions. Next, in those 20 lung regions, lung opacities are subjectively evaluated using a chest CT. Each of the regions could be assigned a score of 0 for 0% parenchymal opacification involvement; 1 point for 1% to 50% involvement; and 2 points for 51% to 100% involvement. The overall score is determined by the summing the points of the 20 lung segment regions, with a total score ranging from 0 to 40 points.

The objective of our study was to validate the 3 aforementioned scales, which we used to assess the extent of lung inflammation in patients with SARS-CoV-2 infection, in terms of interobserver agreement and the correlation of scores with the clinical condition of the patient on the day of the study.

Material and Methods

A total of 77 chest CT scans collected from 77 consecutive patients hospitalized with SARS-CoV-2 infection at the Central Clinical Hospital of the Ministry of the Interior in Warsaw were analyzed in this study. CT scans were performed on a 16-detector CT scanner, with patients in the supine position during a single inspiratory breath-hold, with the scanning range from the apex of the lungs in the costophrenic angle. The CT scan parameters were as follows: 80 to 120 KVp, 100 mAs; slice thickness of 1.2 mm; and pitch of 1.2. The CT scans were reviewed for lung parenchyma using a working station equipped with OsiriX MD v.8.0.2 software at a window width of 1500 HU and a level of −600 HU.

Two radiologists (B.M. and D.P.) assessed the scans independently. Each of them had 7 years of professional experience. The radiologists were informed on the positive results of the RT-PCR tests for SARS-CoV-2, while having no access to the results of other laboratory tests, clinical data, or previous imaging scans.

Three scales were applied to assess chest CT scans: the Total Severity Score, ranging from 0 to 4 points for each of the 5 lung lobes (total score of 0 to 20 points); the Chest CT Score, ranging from 0 to 5 points for each of the 5 lung lobes (total score of 0 to 25 points); and the Chest CT Severity Score, ranging from 0 to 2 points for each of the 20 lung segments (total score of 0 to 40 points).

To assess the interobserver agreement of CT scan assessments, the Cohen’s κ and intraclass correlation coefficient (ICC) were calculated. Owing to the ordinal nature of the scales, the weighted Cohen’s κ coefficient was applied. The weights were selected using the Fleiss-Cohen method [23]. The ICC (3, k) coefficients were calculated relative to absolute agreement. The weighted κ values were interpreted according to the method of McHugh [24], while the ICCs were interpreted according to the method of Koo and Li [25]. Agreement was defined as moderate (κ >0.4–0.6), good (κ >0.6–0.8), or very good (κ >0.8–1.0).

To analyze correlations between the patient clinical condition on the day of the scan and the extent of pulmonary lesions seen in the chest CT scan, all patients whose tests were included in the study were evaluated using the Modified Early Warning Score, which covers measurements of systolic blood pressure, heart rate, respiratory rate, body temperature, and level of consciousness [26] (Table 1). The Pearson linear correlation coefficient was used for the analysis of the correlation between the extent of inflammatory lesions and the clinical condition of the patient. The correlation coefficient was defined as low (r=0–0.3), moderate (r=0.3–0.5), strong (r=0.5–0.7), or very strong (r=0.7–1).

This study was approved by the Bioethics Committee of the Central Clinical Hospital of the Ministry of the Interior in Warsaw.

Results

The interobserver agreement for Total Severity Score assessment was very good. For the overall assessment, the κ was 0.944 (95% confidence interval [CI] 0.948–0.941) and the ICC was 0.948 (95% CI 0.949–0.947). There were no significant differences between the interobserver agreement within the 5 individual lung lobes: the upper-right lobe, κ=0.938 and ICC = 0.944; the middle-right lobe, κ=0.945 and ICC=0.947; the lower-right lobe, κ=0.937 and ICC=0.944; the upper-left lobe, κ=0.937 and ICC=0.944; and the lower-left lobe, κ=0.938 and ICC=0.944 (Table 2).

For the overall assessment using the Chest CT Score, the κ was 0.909 (95% CI 0.914–0.904) and the ICC was 0.919 (95% CI 0.919–0.918). The interobserver agreement of assessments within the 5 individual lung lobes was similar: the upper-right lobe, κ=0.910 and ICC=0.915; the middle-right lobe, κ=0.914 and ICC=0.917; the lower-right lobe, κ=0.907 and ICC=0.913; the upper-left lobe, κ=0.913 and ICC=0.917; and the lower-left lobe, κ=0.907 and ICC=0.914 (Table 3).

The 40-point Chest CT Severity Score was characterized by very good rater-to-rater agreement. For the overall assessment, the κ was 0.888 (95% CI 0.895–0.886) and the ICC was 0.899 (95% CI 0.899–0.898). Comparable agreement was obtained for the assessments of the right lung (κ=0.888, 95% CI 0.895–0.886 and ICC=0.898, 95% CI 0.899–0.898) and the left lung (κ=0.891, 95% CI 0.892–0.884 and ICC=0.899, 95% CI 0.899–0.898) (Table 4).

Of the 3 scales used to assess the extent of lung lesions in the course of SARS-CoV-2 infection, the strongest positive correlation with patient clinical condition as expressed using the Modified Early Warning Score was observed for the Chest CT Severity Score (r=0.627 for Radiologist 1 and r=0.653 for Radiologist 2). The Pearson’s r values for Radiologist 1 and Radiologist 2 were 0.613 and 0.637 for the Chest CT Score and 0.561 and 0.586 for the Total Severity Score, respectively (Table 5).

Discussion

The 3 analyzed scales had very good interobserver agreement. The slight differences observed in the conformity of the assessments might have been due to the complexity of the scales. The highest level of agreement was obtained for the Total Severity Score (0–20 total points; 0–4 points for each lung lobe; κ=0.944 and ICC=0.948). For the Chest CT Score (0–25 total points; 0–5 points for each lobe), the κ was 0.909 and the ICC was 0.919. For the most complex scale, the Chest CT Severity Score (0–40 total points; 0–2 points for each segment), the κ was 0.888 and the ICC was 0.899. The agreement levels for the Total Severity Score and the Chest CT Severity Score scales were similar to those reported by other authors (ICC of 0.976 and 0.925, respectively) [19,20]. The Chest CT Score had not yet been validated for interobserver agreement in COVID-19 patients.

The analysis of correlations between the clinical condition of the patient expressed using the Modified Early Warning Score and the extent of inflammatory lesions observed on CT scan revealed a strong positive correlation for each of the 3 scales analyzed. We observed a relationship between the detail of the scale and the level of correlation with a patient’s clinical condition. The strongest positive relationship was observed for the Chest CT Severity Score (r=0.627 and 0.653). For the Chest CT Score, Pearson’s r values were 0.613 and 0.637, and the weakest correlation was observed for the least detailed Total Severity Score (r=0.561 and 0.586).

Conclusions

The analysis of the conformity of the assessments of pulmonary lesions developed over the course of COVID-19 by the same 2 independent radiologists was characterized by very good agreement of the assessments using each of the 3 analyzed scales. The highest interobserver agreement was achieved with the Total Severity Score. However, the Chest CT Severity Score was distinguishable from among the 3 scales in terms of correlation with patient clinical condition. Further tests are required to validate these scales in the context of disease course prediction and prognosis. The apparent relationship between the detail of the scale and the level of correlation with patient clinical condition highlights the value of accurate and quantitative assessments of pulmonary lesions in patients with SARS-CoV-2 infection. Our findings point to the need for and the importance of the wide availability of automated image segmentation methods and the capability for volumetric assessments of disease-involved and unremarkable pulmonary parenchyma.