A Prospective Observational Study Conducted at a Single Center in Japan to Validate a Previously Developed Predictive Formula of In-Hospital Mortality for Patients Aged ≥65 Years with Endogenous Diseases Transported by Ambulance

Background

Japan has reached a level of super-aged society that is unprecedented worldwide [1]. In 2020, the percentage of the population aged ≥65 years in Japan reached 28.4% whereas that in other developed countries, including in Europe and North America, was 18.3% [1]. The aging population in Japan is still growing and is expected to exceed 30% by 2030 [1]. This situation has led to a yearly increase in the number of patients aged ≥65 years who are transported by ambulance, which reached 60% in 2019 [2]. Additionally, 60% of these patients were admitted to the hospital, and 1.8% of those aged ≥65 years who were transported by ambulance died [2]. An accurate assessment of the prognosis of older patients on arrival to the emergency room could be essential to give them the opportunity to make decisions regarding their treatment [3]. Although the prognosis of older patients is greatly affected by the presence of underlying diseases or concomitant physical or cognitive impairments, precise information on those conditions is not always available during emergency transport or it is difficult to gather this information from patients themselves or from family members in an emergency setting [4]. Therefore, accurate prediction of the prognosis of older emergency patients using data gathered immediately after their arrival to the emergency room by only observing them could greatly help patients and their family members to understand the patient’s situation and choose appropriate treatments.

In 2019, we conducted a retrospective cohort study among inpatients aged ≥65 years with endogenous diseases transported by ambulance, in which we developed a predictive formula for patient in-hospital mortality [5]. The study included 357 inpatients aged ≥65 years who were transported to an acute care hospital in Japan owing to diseases or conditions other than trauma, suicide, fracture, addiction, or suffocation; 37 of these patients died in the hospital. The patients were divided into 2 groups: those with in-hospital death and those surviving to discharge. Multivariate logistic regression analysis was performed using variables available immediately after transport that were significant in the univariate analysis. As a result, the number of covariates was subsequently determined to be 4 based on the sample size, and the following covariates were selected: age ≥90 years, disturbance of consciousness, shock index (SI) [6] ≥1, and oxygen administration ≥5 L/min. Multivariable analysis revealed that those 4 variables were associated with greater mortality, and their regression coefficients were 1.1, 1.0, 1.4, and 1.3, respectively [5]. Subsequently, we developed a predictive formula for patient in-hospital mortality using regression coefficients in multivariable logistic regression analysis, which treated each item as one point using approximation of those regression coefficients and totaled them for a maximum score of 4 [5]. Although this formula showed that the mortality rates and positive likelihood ratio of patients with scores of 3 were 39% and 7.53, respectively, according to our previous retrospective study [5], we failed to evaluate the formula’s discriminatory ability and calibration because of a lack of external validation. External validation was required for more accurate verification of the formula, despite the area under the receiver operating characteristic curve (AUC) of 0.805 that was derived from the internal validation of the previous study.

Therefore, in this prospective study conducted at a single center in Japan, we aimed to validate the previously developed predictive formula of in-hospital mortality for inpatients aged ≥65 years who were transported by ambulance for endogenous diseases.

Material and Methods

ETHICAL CONSIDERATIONS:

This study conforms to the ethical guidelines for medical and health research involving human subjects issued by the Ministry of Health, Labour and Welfare and the Ministry of Education, Culture, Sports, Science, and Technology in Japan. This study was approved by the research ethics committee of the Yuai-Kai Foundation and Oda Hospital (No. 20181015). The study was registered at the University Hospital Medical Information Network (UMIN) at www.umin.ac.jp (UMIN ID: UMIN000046346). Informed written consent was obtained from all patients or their family members, and the anonymity of patients was protected.

STUDY DESIGN AND PATIENTS:

This single-center prospective observational study was conducted at the Yuai-Kai Foundation and Oda Hospital, an acute care hospital in a suburban area in the southern part of Japan (Appendix), which is the same hospital in which our previous study was conducted. Participants were consecutive inpatients aged ≥65 years with endogenous diseases, all of whom were transported by ambulance and subsequently admitted to Oda Hospital during the period April 2019 to March 2020. Patients who met the following criteria were excluded: exogenous diseases (included trauma, suicide, fracture, addiction, and suffocation), cardiopulmonary arrest on arrival, death in the emergency room, and transferred to another hospital for more advanced treatment within 6 hours after emergency room admission. The primary outcome was set as in-hospital mortality after admission to the hospital. The settings, inclusion and exclusion criteria, and outcomes were the same as those in the development study.

DATA AND DATA SOURCES:

All collected items were clearly defined before the investigation and recorded in formatted research templates on the patients’ medical charts. The data on arrival or at admission included: age, sex, arrival time, history of previous visits and admissions, having a referral letter or not, vital signs (blood pressure, heart rate, respiration rate, oxygen saturation), amount of oxygen administration and who decided the amount, level of consciousness (Glasgow Coma Scale [GCS] score), past medical history and medications, Bedriddenness rank [7,8], Cognitive function score [7,8], ABC Dementia Scale [9–11], and Barthel Index [12]. Bedriddenness ranks are an official Japanese assessment tool to assess the degree of impairment in daily activities among aged people, classified into 9 grades using 4 classification steps [7,8]. Cognitive function scores are an official Japanese assessment tool of dementia and cognitive impairment among aged people, which are classified into 8 grades using 5 classification steps [7,8]. The ABC Dementia Scale is a scale of dementia severity developed in Japan, which comprises 13 questions [9–11]. We defined patients “taking antihypertensive medications” as those taking one or more of the following medicines: α-blockers, β-blockers, calcium channel blockers, angiotensin II receptor blockers, angiotensin-converting enzyme inhibitors, diuretics, nitrates, and eplerenone.

On admission and after discharge, the research physician and research assistants reviewed the electronic medical records concerning the following items: infusion volume at 24 hours after hospital arrival, use of vasopressor agents, date of discharge, final diagnosis, in-hospital mortality, and date of death. Other items were also evaluated and recorded by nurses in the emergency room. Research assistants checked the presence of missing data for the above items after admission and reminded the attending nurse to complete the record. In the development study, disturbance of consciousness was assessed using the Japan Coma Scale and the medical information in the chart, whereas in this study, it was assessed using GCS.

PREDICTIVE FORMULA:

The predictive formula is as below:

The predictive formula, which we developed in our previous study in 2019, comprised 4 items: age ≥90 years, disturbance of consciousness, SI ≥1, and oxygen administration ≥5 L/min. The predictive score was calculated with those conditions treated as 1 point each and subsequently totaling these for a maximum score of 4 [5]. We used the formula we developed in that previous study, without modification.

STATISTICAL ANALYSIS:

We divided patients into 2 groups: those with in-hospital death (Group D) and patients surviving to discharge (Group S). We calculated the median and interquartile range (IQR) for continuous variables using the Mann-Whitney U test. For categorical variables, the proportion was calculated using the chi-square test. Multivariable logistic regression analysis was performed using the 4 categorical variables comprising the prediction formula developed in our previous study, defining in-hospital death as the primary outcome. After the predictive score for each case was calculated, we then calculated mortality, sensitivity, specificity, and the stratum-specific likelihood ratio according to each score, AUC of the formula, and shrinkage coefficient. Subgroup analysis was conducted by comparing AUCs of the predictive formula between the 2 groups divided by the presence or absence of items other than the 4 items used in the formula. Analyses were performed using IBM SPSS statistics version 25 (IBM Corp., Armonk, NY, USA). The significance level was set at p<0.05. The missing values for each variable were excluded from the analysis.

The required sample size for the two-tailed test was calculated to be 23 patients in Group D and 229 in Group S from the expected AUC of 0.7 and ratio of the number of patients belonging to Group D to that of Group S of 10, based on our previous study, with an α error 5% and a β error 10%.

Results

PATIENTS’ BACKGROUND:

During the study period, 732 patients were transported to the emergency room. In total, 325 patients were eligible for analysis after excluding 211 patients aged <65 years, 170 patients who were not admitted to Oda Hospital, and 26 patients with exogenous diseases or cardiopulmonary arrest (Figure 1). The patient backgrounds are shown in Table 1. The median age of the patients was 84 (IQR: 78–91) years with 107 (33%) patients aged ≥90 years. Among those patients, 139 (43%) were men. Forty-two patients (12%) died during hospitalization. A list of patients’ medications at the time of transport to the emergency room (Supplementary Table 1) and the breakdown of International Classification of Diseases Tenth Revision (ICD-10) diagnosis codes (Supplementary Table 2) are shown in the supporting material. According to the ICD-10 classification, 67 patients (20%) had “J”, i.e., respiratory system diseases, with aspiration pneumonia and acute pneumonia, accounting for 7% each. Following those, 58 patients (17%) had “I”, i.e., circulatory system diseases and 35 patients (10%) had “K”, i.e., digestive system diseases.

UNIVARIATE ANALYSIS:

According to the definition in univariate analysis of in-hospital death as the primary outcome, significant differences between Group D and Group S were found for age (89 [IQR: 84–94] vs 84 [IQR: 77–91] years, p=0.001), age ≥90 years (48% vs 31%, p=0.030), use of vasopressors (17% vs 6%, p=0.009), number of prescribed medicines (8 [IQR: 5–10] vs 6 [IQR: 3–8], p=0.031), disturbance of consciousness (57% vs 11%, p<0.001), oxygen saturation (96% [IQR: 93–98%] vs 97% [IQR: 95–99%], p=0.004), quick Sequential Organ Failure Assessment (q-SOFA) score 2 points or higher (38% vs 13%, p<0.001), oxygen administration of 5 L/min or higher (40% vs 17%, p<0.001), Barthel Index (15 [IQR: 0–36] vs 60 [IQR: 20–95], p<0.001), ABC Dementia Scale score (69 [IQR: 53–104] vs 101 [IQR: 68–117], p<0.001), Bedriddenness rank (p<0.001), and Cognitive function score (p=0.005) (Table 1).

MULTIVARIATE ANALYSIS:

In multivariable logistic regression analysis (forced-entry method), defining in-hospital death as the primary outcome using the 4 items adopted in the predictive formula developed in our previous study, only disturbance of consciousness (odds ratio [OR]: 2.6 (95% confidence interval [(95% CI]: 1.3–5.2), p=0.008) and oxygen administration 5 L/min or higher (OR: 2.6 (95% CI: 1.3–5.4), p=0.009) showed significant associations with in-hospital mortality (Table 2). The AUC of the prediction formula was 0.670 (95% CI: 0.574–0.767) (Figure 2), and the incidence of observed in-hospital mortality was consistent with the predicted incidence calculated using the prediction formula, with a shrinkage coefficient of 0.759 (Figure 3). The mortality rate and stratum-specific likelihood ratios for scores in the predictive formula of 3 and 4 were 40.9% and 4.66, and 66.7% and 13.48, respectively (Figure 4).

SUBGROUP ANALYSIS:

In the subgroup analysis, the AUC of the predictive formula was higher than 0.7 in patients with a duration of hospitalization shorter than the median of 12 days; fluid infusion <2000 mL within 24 hours; Barthel Index equal to or higher than the median of 55; Bedriddenness rank of normal, J, or A; and ABC Dementia Scale equal to or greater than the median of 97 (Figure 5).

Discussion

In this study, we conducted a prospective external validation of our predictive formula for in-hospital mortality among older patients with endogenous diseases, which we had developed in our previous study [5]. The predictive formula adopted 4 items: aged ≥90 years, disturbance of consciousness, SI ≥1, and oxygen administration ≥5 L/min. This formula is extremely easy to use, treating each positive item as one point and totaling them, making it convenient and useful in busy clinical settings [5]. The present study showed that the AUC of the formula was 0.670, with high sensitivity for a score of 1; high specificities for scores 2, 3, and 4; and high stratum-specific likelihood ratio for a score of 4. These findings demonstrate that the formula has sufficiently reliable discrimination ability for in-hospital mortality among older patients immediately after ambulance transport. Previous studies by other researchers on predictive models of emergency patient mortality included factors similar to those in the present study, such as age, consciousness disturbance, blood pressure, and oxygen saturation index [13]. However, those models used the APACHE (Acute Physiology and Chronic Health Evaluation) II diagnostic category, which includes mechanical ventilation or laboratory findings, making these impossible to use immediately after transport, targeting adults of all ages. In addition, there is no previous research on predictive models for in-hospital mortality among emergency patients aged ≥65 years, as in our model. Therefore, the present study is highly valuable because we show the high reliability and accuracy of the predictive model for mortality among older patients, using data that are available immediately after being transported by an ambulance.

Although the SI is useful in predicting patient mortality [14], in the present study, we found no significant difference between patients who died during hospitalization (Group D) and those who survived to discharge (Group S). The SI, which is easily calculated using heart rate and blood pressure [15], is reported to be an important prognostic index for patients with trauma, bleeding, cardiovascular disease, or sepsis [16] and could be extremely useful in predicting 30-day mortality in patients of all ages [14]. However, the SI could be less informative regarding the condition of older patients than that of younger ones because of age-related decline in physiological function, which would change the heart rate or blood pressure under stress [17]. Additionally, the total number of oral medications taken by patients at the time of emergency transport was significantly higher in Group D, although there was no significant difference in the number of antihypertensive medications between the 2 groups (Table 1). Therefore, the SI might not reflect the true physical condition of older patients because multiple medications prevent the normal fluctuation of vital signs under stress [14]. Additionally, subgroup analysis in this study showed a higher AUC of the formula in patients who had better capability in terms of Activities of Daily Living (ADLs) than those with poorer ADL capability. It is reported that older patients with poor ADL capability tend to show a lower standard deviation for heart rate variability and fewer heart rate fluctuations than those with normal ADL capability [18]. This could prevent the SI from reflecting the precise condition of older patients in the present study, with a large proportion of patients with poor ADL capacity (Supplementary Table 3) slightly lowering the accuracy of the predictive formula. This study also showed that the value of the age-adjusted SI [19,20] (calculated as age×SI), the usefulness of which has recently been reported, was significantly higher in Group D by univariate analysis (Table 1). Therefore, adopting an age-adjusted SI could improve the prediction accuracy of our formula.

Subgroup analysis in this study also showed a higher AUC of the predictive formula in patients with no or only mild dementia than in those with severe dementia (Figure 5). Although we defined disturbance of consciousness as GCS ≤14 in the present study, it could be inappropriate to judge the presence or absence of disturbance of consciousness in patients with cognitive impairment based on GCS alone because of their possible inability to follow instructions from medical staff owing to communication problems. A similar problem is indicated regarding judgment using q-SOFA score because it also includes disturbance of consciousness as an assessment item [21]. Additionally, delirium could easily develop in older patients with dementia, 76% of whom are not correctly diagnosed at the time of emergency transport [22]. Therefore, it might be difficult to differentiate transient disturbance of consciousness caused by delirium from usual manifestations caused by baseline dementia as the cause of communication problems. In this study, up to 65% of patients had some degree of cognitive impairment (Table 1), which could have been the cause of a slightly lower AUC in the formula than that predicted in our previous study [5]. Thus, the prediction accuracy of this formula could be slightly inferior when used in a population with many patients who have dementia.

There are several limitations in this study. First, the study was conducted at a single center, which was the same hospital as that in our previous study in which we developed the prediction formula. Therefore, it would be desirable to conduct further studies to validate the formula in different populations treated in different departments and with different diseases or severities of illness; it is especially important to include a substantial number of patients with more severe conditions showing a score 4, which was rather limited in this study. Although aged ≥90 years was not significantly associated with in-hospital mortality in the present study, other factors not included in the predictive formula, such as diagnoses (diseases or classification of organs involved), baseline diseases frequently affecting older patients, or medication status [23], could have influenced the results. There was no protocol to determine the amount of oxygen administered, which was at the discretion of paramedics in the ambulance. We failed to evaluate whether patients had a living will or palliative care plan, which could have affected the findings of this study.

Conclusions

Our previously developed predictive formula for in-hospital mortality, which comprises variables available immediately after emergency transport, showed sufficiently reliable discriminatory ability in prospective external validation. However, some caution is required when using this formula in the presence of certain factors, such as multiple medications, advanced age, or low ADL capability, which could prevent fluctuation of vital signs under physical stress, as well as dementia, which could disrupt evaluation of the level of consciousness.

Supplementary Metarials

Appendix

CHARACTERISTICS OF THE YUAI-KAI FOUNDATION AND ODA HOSPITAL:

The hospital contains 11 departments with 111 beds for acute care. The departments are: Internal Medicine, General Medicine, General Surgery, Cardiovascular Surgery, Neurosurgery, Otorhinolaryngology, Plastic Surgery, Dermatology, Radiology, Anesthesiology, and Rehabilitation. The facility has no Orthopedic Surgery department. The hospital is located in the city of Kashima, in Saga Prefecture in southern Japan. It covers an approximate population of 90,000 people, and treats approximately 3,200 inpatients each year, with a mean length of stay of 11.6 days.

The median age of inpatients for the overall hospital is 76 years (quartile 62–85). The breakdown of diseases of patients admitted to Department of Internal Medicine by organ is as follows: respiratory system, 26.0%; digestive system, 26.0%; cardiovascular system, 10.4%; renal/urinary system, 6.4%; and hematology system, 5.3%.