Validity and Reliability of the Thai Version of the Delirium Diagnostic Tool-Provisional (DDT-Pro-TH) in Perioperative Patients With Fragility Hip Fractures

Introduction

In Thailand, the aging population has been steadily increasing, leading to a substantial rise in fragility hip fractures (FHF) among older adults [1]. Such fractures greatly affect quality of life and mobility [2]. They typically are managed through surgical intervention, such as internal fixation or hip replacement procedures, to facilitate a return to pre-injury ambulatory function [3]. However, complications such as urinary tract infections, pressure sores, and delirium [4] are frequently observed.

Delirium is an acute disorder of cognition (particularly attention), higher-level thinking, and circadian rhythms that commonly affects older patients [5]. It not only delays recovery but also contributes to prolonged hospital stays, increased healthcare costs, and a higher risk of long-term disability or mortality, thereby imposing substantial burdens on families, caregivers, and the healthcare system [6]. The reported prevalence of delirium in geriatric patients undergoing hip fracture surgery typically ranges from 20% to 34% worldwide [7,8]. This prevalence may be even higher in certain populations; for example, studies in Thailand have shown rates between 31% and 45% [4,9]. Diagnosis is generally based on the Diagnostic and Statistical Manual of Mental Disorders, a globally accepted diagnostic standard that requires psychiatric expertise. Its Fifth Edition (DSM-5) [10] is available in Thai. Brief screening methods, such as the Confusion Assessment Method (CAM) [11] and the Memorial Delirium Assessment Scale [12], are also widely used, although they perform better with specific training and expertise.

The Delirium Diagnostic Tool-Provisional (DDT-Pro) [13,14] is a novel instrument designed to simplify delirium diagnosis without requiring expertise in neuropsychiatric assessment because of its highly structured design, which also permits severity ratings. By assessing only 3 symptoms representing each core domain of delirium – vigilance, comprehension, and sleep-wake cycle – the DDT-Pro has demonstrated good accuracy and reliability in international settings, including the United States, Colombia, Spain, and Korea [13–16]. A recent international study indicated very high validity (area under the curve [AUC]=90% vs DSM-5 Text Revision) in the intensive care unit when appropriate translations were used [17]. However, it has not yet been translated or validated for use in older Thai patients or individuals with FHF. Therefore, this study aimed to validate and assess the reliability of the Thai version of the DDT-Pro for delirium diagnosis (DDT-Pro-TH) against DSM-5 criteria as the reference standard.

Material and Methods

ETHICS, DESIGN, AND PARTICIPANTS:

Upon approval from the Siriraj Institutional Review Board (COA no. Si 353/2022) and registration of the clinical trial protocol (NCT06504251), a cross-sectional study was conducted at a university-based national tertiary referral center from September 2022 to March 2024 following the Standards for Reporting Diagnostic Accuracy Studies (STARD) guideline.

The inclusion criteria were age 65 years and older, diagnosis with FHF, and hip fracture occurrence within 7 days before admission to Siriraj Hospital. Exclusion criteria were an inability to communicate in Thai; a diagnosis of cancer before or during treatment for a hip fracture (which may present as brain metastasis); pre-existing schizophrenia; pre-existing dementia or moderate to severe cognitive impairment (to avoid confounding the assessment and diagnosis of acute delirium); and diagnosis with coronavirus disease 2019 (COVID-19) during admission (due to challenges in evaluation).

VALIDITY ASSESSMENT:

The sample size for evaluating concurrent validity was determined using the 95% confidence interval (CI) for the AUC. PASS version 14 software (NCSS LLC, Kaysville, UT, USA), used to calculate the required sample size, indicated that 99 participants were needed. The expected AUC estimate was set at a minimum of 0.700, with a 95% CI ranging from 0.600 to 0.900.

RELIABILITY ASSESSMENT: For internal consistency reliability, the sample size calculation was based on the 95% CI for Cronbach’s alpha, following the methodology described by Bonett and Wright [18]. The planned Cronbach’s alpha was 0.7, with a desired width of 0.2 and a 95% CI. The resulting confidence interval ranged from 0.5861 to 0.7875, requiring a sample size of 90 participants. For inter-rater reliability, the sample size was based on the 95% CI for the Cohen’s kappa statistic. According to the work of McHugh [19], 66 participants were required for this analysis.

To ensure comprehensive evaluation of the validity and reliability of the DDT-Pro-TH, at least 66 participants were required for inter-rater reliability, at least 90 for internal consistency, and at least 99 for concurrent validity. Considering the expected 10% dropout rate, we enrolled 110 patients with FHF to maintain statistical power and reliability.

THE DELIRIUM DIAGNOSTIC TOOL-PROVISIONAL (DDT-PRO) INSTRUMENT AND ITEMS:

The DDT-Pro consists of 3 items that assess core domains of delirium: attention (vigilance), higher-level thinking skills (comprehension), and circadian function (sleep–wake cycle). Each item is scored from 0 to 3, yielding a total score of 0 to 9. A DDT-Pro score of 6 points or lower indicates delirium [14]. Vigilance and comprehension are rated via direct testing of the patient, whereas the sleep–wake cycle is rated based on the previous 12 to 24 h using any available information source. DDT-Pro items were derived from 2 well-validated delirium instruments, the Delirium Rating Scale-Revised-98 [20] and the Cognitive Test for Delirium [21]; they include 2 different forms (A and B) [14] to reduce potential learning effects during repeated assessments.

Translation of the DDT-Pro involved forward and backward translation to ensure accuracy and validity. The initial English-to-Thai translation was performed by a translator proficient in both languages in the translation unit of the Faculty of Arts, Chulalongkorn University, Bangkok, Thailand. A multidisciplinary team – including orthopedic specialists, a geriatrician, an anesthesiologist, and a physiatrist – reviewed the translation and resolved discrepancies. The agreed-upon Thai version was then back-translated into English by a translation expert proficient in Thai. The back-translated version was compared with the original to identify adjustments, which were reviewed by the same team of experts and Consultation-Liaison Psychiatry delirium specialists Drs. Trzepacz (developer of the DDT-Pro) and Franco. The final DDT-Pro-TH and a DSM-5 evaluation form were used as assessment tools in this study.

RECRUITMENT AND INFORMED CONSENT:

Patients with FHF admitted to the hospital for surgery who met the inclusion criteria were invited to participate. The attending physician thoroughly explained the research procedures and protocols, encouraged patients or proxies to ask questions, and ensured that they fully understood the process. Participation was voluntary, and informed consent was obtained through a certified Letter of Intent to Participate in the Research Project and a Document Explaining Research Participation, both approved by the Siriraj Hospital Human Research Ethics Committee.

DATA COLLECTION PROCEDURES:

Interviews using pre-prepared instruments (the DDT-Pro-TH and DSM-5 [10]) were conducted to collect patient data preoperatively and on postoperative days 1 and 2. Data collection occurred when patients reported a resting pain score of 0–2 and had sufficient time to complete the evaluation form. Evaluation schedules depended on researcher availability; most assessments were conducted in the late afternoon. Each of the 110 participants underwent an initial pre-surgery evaluation the day before the procedure using both the DDT-Pro-TH and the DSM-5 delirium reference standard, followed by reassessments on postoperative day 1 and 2.

A geriatrician assessed each patient using the DSM-5 reference standard. For validation of the translated tool, an orthopedic resident and a clinical researcher each assessed the patient using different DDT-Pro-TH forms (A or B) at slightly different times. Each evaluator was independent and blinded to the other’s results. DSM-5 and DDT-Pro-TH assessments were always performed within the same 4-h period.

COLLECTION OF BASIC PARTICIPANT INFORMATION:

Basic demographic and clinical information were collected, including age, sex, educational background, functional ambulatory status, time from initial hospital admission to surgery, fracture site (eg, neck, intertrochanter), surgical method (eg, fixation, arthroplasty), pre-injury mobility status, and length of hospital stay.

STATISTICAL ANALYSIS:

Statistical analyses were performed using SPSS Statistics for Windows, version 30.0 (IBM Corp., Armonk, NY, USA). P-values <0.05 were considered statistically significant. Patient demographic and clinical characteristics were summarized using descriptive statistics. Categorical variables are presented as numbers and percentages; normally distributed continuous variables are reported as mean±standard deviation.

VALIDITY:

Concurrent validity, a type of criterion validity, was determined by comparing the DDT-Pro-TH results with the DSM-5 assessment administered simultaneously. The AUC of the receiver operating characteristic (ROC) curve, with its corresponding 95% CI, was used to measure diagnostic accuracy. An AUC near 1.0 indicated excellent performance, whereas an AUC between 0.7 and 0.9 suggested moderate to good performance. A narrow 95% CI indicated greater certainty in the model’s classification performance; a wider CI indicated less certainty [22].

RELIABILITY:

The reliability of the DDT-Pro-TH was evaluated in terms of internal consistency and inter-rater reliability; it is reported for each of the 3 assessments and the combined dataset. Internal consistency reliability measured the degree of interrelatedness among the items within the tool. Cronbach’s alpha was used, with an ideal range of 0.70 to 0.95. Values below 0.70 indicated insufficient correlation among items, whereas values above 0.95 indicated redundancy [23]. Inter-rater reliability measured consistency and agreement between DDT-Pro-TH raters using Cohen’s kappa, which ranges from −1 to 1. A value of 1 indicated perfect agreement, 0 indicated agreement by chance, and −1 indicated complete disagreement [19].

Results

PATIENT CHARACTERISTICS AND GENERALIZABILITY:

From September 2022 to March 2024, 110 patients with FHF participated in the study, 13 (11.8%) of whom were diagnosed with delirium per DSM-5. All patients completed DDT-Pro-TH assessments. The sample distribution is shown in Figure 1. Demographic and clinical characteristics are presented in Table 1.

VALIDITY:

The concurrent validity AUC of the DDT-Pro-TH for DSM-5 delirium diagnosis was 0.959 (95% CI, 0.903–0.987) (see Figure 2 for ROC distribution). This performance indicates a high level of accuracy in distinguishing between positive and negative cases. The result was statistically significant (P<0.001). The ROC curve showed a steep rise, achieving high sensitivity at 92.3% (95% CI, 64.0–99.8) with minimal loss of specificity (87.3%; 95% CI, 79.4–93.4) at the cutoff of 6 points or lower. This finding reflects robust performance consistent with recommended cutoffs from other translations.

RELIABILITY:

Internal consistency reliability demonstrated a Cronbach’s alpha of 0.819 (95% CI, 0.783–0.851) (Table 2), indicating strong measurement consistency throughout the study period when rating days were combined. The preoperative assessment showed high reliability, with Cronbach’s alpha of 0.795 (95% CI, 0.718–0.853). On postoperative day 1, reliability was excellent, with Cronbach’s alpha of 0.861 (95% CI, 0.808–0.900); on postoperative day 2, reliability remained high, with Cronbach’s alpha of 0.787 (95% CI, 0.707–0.848). These results demonstrate the robustness of measurements across time points.

Inter-rater reliability was assessed using Cohen’s kappa, with an overall value of 0.974 (95% CI, 0.962–0.982), indicating excellent agreement throughout the study period (Table 3). Each assessment day also demonstrated excellent agreement: preoperative Cohen’s kappa was 0.973 (95% CI, 0.961–0.981), postoperative day 1 Cohen’s kappa was 0.995 (95% CI, 0.993–0.997), and postoperative day 2 Cohen’s kappa was 0.982 (95% CI, 0.974–0.988). These findings highlight the consistency and reliability of assessments across time points.

Discussion

In this study, we successfully translated the DDT-Pro into Thai, producing the DDT-Pro-TH, a culturally relevant instrument for delirium diagnosis. Concurrent validity, internal consistency, and inter-rater reliability in the present sample of patients with FHF revealed strong performance. The DDT-Pro-TH is designed to assist various non-expert healthcare providers with delirium diagnosis by assessing 3 key symptoms – vigilance, comprehension, and sleep–wake cycle – each representing a core domain of delirium [24,25]. Our findings demonstrate the high reliability and validity of the DDT-Pro-TH on each individual assessment day, both preoperatively and postoperatively, as well as when scores were combined across the study period.

The tool demonstrated excellent concurrent validity relative to the DSM-5, with an AUC of 96%, and identified all patients with delirium in this population. Its specificity of 87.3% at the previously determined cutoff value of 6 points or lower reflects high accuracy in excluding delirium; there was only a small proportion of false positives. Sensitivity was 92.3%. However, prior studies have shown that subsyndromal delirium can present with scores of 6 and 7, suggesting that some false positives in this study reflected that milder state [13]. Our results underscore the effectiveness of the DDT-Pro-TH as a screening tool for timely detection of delirium, supporting prompt clinical intervention and potentially improved outcomes in this vulnerable patient group.

When compared with the CAM in other reports, the DDT-Pro has exhibited similarly strong diagnostic performance, as well as notable differences. Although the CAM has reported sensitivity of 94% and specificity of 89%, its performance considerably varies depending on patient type and rater training and expertise. Sensitivity can be as low as 13% when administered by nurses to surgical patients [11,26,27], which is a substantial limitation for a brief screening tool. In contrast, the DDT-Pro-TH demonstrated high sensitivity (92.3%) in detecting delirium among patients with FHF, ensuring that no cases were missed, although specificity was slightly lower (87.3%). Its high sensitivity may offer an advantage in targeted settings where accurate and early detection is essential, particularly during perioperative care for patients with FHF who exhibit high risk for delirium and its associated in-hospital morbidities.

Additionally, previous validation studies of the DDT-Pro have shown high diagnostic accuracy (ROC AUC=93.8–99%) across various patient populations and strong inter-rater reliability (0.87) between physicians and nurses. Upon direct comparison with the CAM in geriatric inpatients, the DDT-Pro demonstrated higher sensitivity than the CAM (90% vs 64%), particularly in the dementia subgroup (75% vs 62–64%); its specificity was slightly lower (86.7% vs 95.3%) than when the DDT-Pro was administered by a nurse and the CAM was administered by an expert consultation-liaison psychiatrist [26]. Among patients with comorbid dementia, the DDT-Pro demonstrated sensitivity of 91% and specificity of 77% (vs 61.8% and 90.0% for the CAM) [26]. These findings indicate that although both tools are effective, they may identify delirium differently, making the DDT-Pro-TH a valuable alternative for use by non-expert clinicians in perioperative settings.

The Four ‘A’s Test (4AT) is another widely used delirium screening tool, with overall sensitivity and specificity values of 88% each [11,28]. A validation study directly comparing the DDT-Pro with the 4AT in geriatric patients from a skilled nursing facility with a high prevalence of comorbid dementia revealed discordant diagnoses between the 2 tools. The DDT-Pro demonstrated better screening performance, with sensitivity exceeding that of the 4AT by more than 20 percentage points (77.2% vs 54.4%) [15].

During its original validation in patients with acute traumatic brain injury, the DDT-Pro demonstrated accuracy of 99.4%, sensitivity of 100%, and specificity of 94.4% using a cutoff of 6 points or lower when administered by rehabilitation nurses [14]. In a study of predominantly intensive care unit patients with COVID-19, the DDT-Pro predicted mortality and was correlated with a COVID-19 severity scale [29]. A recent intensive care unit validation study also showed excellent performance using a cutoff of 6 points or lower, with accuracy of 90% [17].

The DSM-5 delirium prevalence in our FHF population was 11.8% (n=13). A meta-analysis evaluating postoperative delirium in older adults undergoing hip fracture surgery revealed a wide prevalence range of 16% to 40% when diagnoses were based on DSM-III and DSM-IV, with variability attributed to differences in medical care across study centers [7]. Delirium in patients with hip fracture is linked to mortality, prolonged hospitalization, cognitive decline, functional deterioration, and increased healthcare costs [7,30].

At Siriraj Hospital (our institution), the relatively low delirium prevalence may be partly explained by the presence of a well-established fracture liaison service. This multidisciplinary team – which includes geriatricians, orthopedic surgeons, anesthesiologists, physiatrists, and specialized nurses – focuses on prevention and early detection of complications such as delirium [31]. The fracture liaison service adopts a proactive approach by addressing risk factors early, optimizing perioperative care, and implementing tailored management strategies. Early interventions, such as pain control, hydration, mobilization, and cognitive monitoring, are integral to the protocol. This model not only reduces delirium incidence but also improves patient outcomes, shortens hospital stays, and decreases healthcare costs, demonstrating the value of multidisciplinary care in managing FHF [9].

In most countries, consultation-liaison psychiatrists are the primary diagnosticians of delirium because it is a neuropsychiatric disorder. When these individuals are not available, other clinicians may be involved in detection. The advantage of the DDT-Pro lies in its use of robust performance metrics for provisional diagnosis while maintaining simplicity and a structured format, enhancing accuracy even among non-expert raters. Because delirium symptoms can overlap with or be confounded by symptoms of many other neuropsychiatric conditions, the DDT-Pro is recommended as an initial step in diagnosis, after which a psychiatrist should perform a thorough differential diagnosis of other psychiatric conditions to ensure accurate delirium diagnosis, etiologic assessment, and management.

Despite its strengths, this study has several limitations. First, the relatively small number of delirium cases (n=13) may limit the generalizability of the findings, emphasizing the need for larger sample sizes to provide more robust estimates of prevalence and diagnostic performance. This lower delirium frequency is particularly notable when compared with a study conducted at Siriraj Hospital more than a decade ago (2010–2012), which showed a higher delirium incidence of 45% [4]. Second, the study was conducted at a single center, which may restrict the applicability of the results to other healthcare settings or populations. The exclusion of patients with severe comorbidities, advanced dementia, or communication barriers further limits generalizability. However, the DDT-Pro has shown strong performance, even among patients with dementia [15,26]. Third, the cross-sectional design may not fully capture the dynamic and fluctuating nature of delirium, potentially overlooking important temporal variations in its presentation. Notably, a prior study showed improvement in DDT-Pro scores with clinical improvement of delirium on days 4 and 5 [26]. Finally, assessments of DDT-Pro accuracy may depend on the diagnostic criteria or reference standards used to confirm delirium, which can vary across studies and settings. The DDT-Pro demonstrated higher performance metrics when the Delirium Rating Scale–Revised-98 was used as the reference standard, instead of DSM-5 [26].

Conclusions

In this study, we successfully translated, cross-culturally adapted, and validated the DDT-Pro-TH for perioperative patients with FHF. The DDT-Pro-TH demonstrated strong psychometric properties, with high internal consistency reliability (Cronbach’s alpha=0.819), excellent inter-rater reliability (Cohen’s kappa=0.974), and excellent concurrent validity (AUC=0.959) relative to DSM-5 assessment. These findings indicate that the DDT-Pro-TH is a reliable and valid tool for diagnosing delirium in this vulnerable population, offering a practical, non-specialist alternative to the DSM-5 for improved delirium detection and management.