Responsiveness of walking-based outcome measures after multiple sclerosis relapses following steroid pulses

Background

The selection of outcome measures to be applied is one of the most important factors in designing a clinical trial. The outcome measures present a major factor influencing the duration of a trial, likelihood of detecting a therapeutic effect, size of the sample and acceptance of study results [1]. Selection of outcome measures is particularly difficult for multiple sclerosis (MS) trials because MS affects patients in many different ways. Among many difficulties, walking problems are very common and important for MS patients; about three-quarters of them experience mobility problems [2,3] and they consider walking as the most valuable bodily function [4]. Also, new developments in the pharmacotherapy to improve walking in MS patients have occurred [5,6]. Therefore, the assessment of walking ability in MS patients is of a great importance for clinical research and practice.

Assessment of walking ability in MS patients began in 1955 with the first measure of disease severity, the Kurtzke Disability Status Scale [7]. Since that time, there has been a proliferation of numerous generic and genuine walking-based measures for MS patients. They are either clinician-based, patient-based, timed over fixed distance, measure the maximum distance a person can walk over a specific time interval, or use motion sensors such as accelerometers [8]. There is a considerable body of literature on the reliability and validity of walking-based measures but, regardless of their clinical importance, the studies of their responsiveness are either deficient or mostly related to rating scales [9–12].

Responsiveness is ability of a measure to detect clinically important change or change over time, and is evaluated by various responsiveness statistics. Typically, responsiveness is determined by comparing before and after scores of interventions expected to produce a change in health. As the interpretation of P values is somewhat binary and sample size-dependent [13], it has become common to report responsiveness as an effect size, or standardized change score, by converting change scores into standard deviation units. Effect size and P values are limited indicators of responsiveness because they are inseparably linked to the magnitude of change induced by the intervention [14]. This can be partly overcome by comparing measures head-to-head in the same sample, which keeps sample and treatment effect constant and enables researchers to compare the relative responsiveness of competing measures [15].

The aims of this study were: to examine the effects of IVMP on the recovery of walking ability in patients experiencing MS relapse; to compare, head-to-head, the responsiveness of some walking-based measures applied for MS trails; and to consider their potential implications for clinical trials.

Material and Methods

PATIENTS:

The patients involved in this study were recruited from the Department of Neurology, University Hospital Center Split, from July 2008 to December 2010. The research was approved by local Ethics Committee and all participants gave their informed consent. A total of 54 consecutive patients who were selected for IVMP (1000 mg/day over 3 days) due to MS relapse and who met the inclusion criteria were asked to participate in the study. Three patients refused to take part, 1 patient did not appear for control testing, and 1 patient was found to be noncompliant with wearing the accelerometer; therefore 49 patients were included in the statistical analyses. During the study, 6 patients experienced more then 1 relapse meeting the inclusion criteria. Due to minimizing the potential impact of practice effects on the results, only their first relapse was considered. A relapse was defined as either new nervous system deficits or worsening of previous ones lasting at least 24 hours [16]. All patients complained of walking difficulties caused by MS relapse and had clinical involvement, individually or combined, of the corticospinal tract (paresis, hyperreflexia, spasticity, extensor plantar response), posterior columns or medial lemnisci tract (proprioception) and cerebellum (cerebellar ataxia) affecting 1 or both lower limbs. The new nervous system deficits or worsening of previous ones were assessed strictly by clinical examination. MRI studies for documenting location and size of the MS lesions responsible for the relapses were not included. The inclusion criteria were: 1) age 18 years or over; 2) definite diagnosis of relapsing-remitting MS (RRMS) [17]; 3) EDSS [18] ≤6.5 at the time of the inclusion; 4) relapse with onset of symptoms within 2 weeks prior to IVMP; 5) no spontaneous improvement prior to IVMP; 6) relapse involvement of gait with deterioration of at least 1 step in the relevant functional system (FS) (pyramidal, sensory or cerebellar) or an increase in EDSS of 1 point or more; and 7) ability to perform walking tests. Exclusion criteria were: 1) treatment with corticosteroids in the previous 3 months; 2) cognitive impairment; 3) vision impairment; 4) orthopedic disease; and 5) cardiac disease. Pre-attack EDSS and FS were well known because the patients were routinely seen in our outpatient facility every 3 to 6 months by a trained neurologist with experience in multicentre clinical trials (M.M.) [19].

OUTCOME MEASURES:

The following walking-based measures were administered before and after IVMP: the Expanded Disability Status Scale (EDSS) [18]; the Multiple Sclerosis Walking Scale-12 (MSWS-12) [20]; the 25-foot walk test (25FWT) [21]; the Six Spot Step Test (SSST) [22] and the StepWatch Activity Monitor (SAM, OrthoCare Innovations, Washington DC, USA). The 2-minute timed walk (2-minTW) was administered as a shorter (reduced) version of the 6-minute timed walk (6-minTW) [23]. The 2-minTW was chosen as a more appropriate walk test for patients suffering from MS relapse than the 6-minTW. The 25FWT and the SSST were repeated twice, and the average number of seconds was used in the analysis. Since test-retest reliability of the 2-minTW for the first 10 participants in both tests (before and after IVMP) was high (Cronbach α=0.981 and 0.992, respectively), subsequent participants were tested once. Comparison was undertaken using the first 2-minTW in those who had repeated testing. The translation of the MSWS-12 into Croatian was done according to the international standards. All patients had worn the SAM 1 week prior to IVMP was applied and wore it again the fourth week upon the corticosteroid therapy was completed. They were asked to wear the SAM continually except when engaged in water activities. The SAM was fitted according to the proprietary instructions and was individually programmed for cadence and sensitivity. Calibration was undertaken for the first 100 steps. Evaluation of the output data confirmed compliance with wearing the accelerometer for all patients except for 1. The analysis utilized the average daily step count, the percentage of time spent inactive, the percentage of time spent in low step activity (1–15 steps/min), the percentage of time spent in medium step activity (16–30 steps/min), the percentage of time spent in high step activity (>31 steps/min), the average peak activity index (average steps/min of the highest 30 minutes of the day regardless of when they occurred) and the average steps in 1, 5, 30 and 60 minutes (average steps/min of the highest continuous period of 1-min, 5-min, 30-min and 60-min periods). No significant difference was found in SAM output data between the first and the seventh day of monitoring before and after IVMP. Other tests were administered between 1 and 3 p.m. before IVMP and 31 day later at the same time of day. All walk tests, (25FWT, SSST and 2-minTW) participants performed in the same type of footwear and clothes. To reduce patients’ bias, the walking-based tests were conducted without fixed order, with the exception of continuous long-term walk monitoring. To reduce the clinicians’ bias, the indication for IVMP was established independently of the study and different parts of research were also performed independently by different authors (clinical examination – M.M.; MSWS-12 and walk tests – P.F.G.; SAM instructions and analyses of outputs – I.L.; statistics – V.Č.).

STATISTICAL ANALYSIS:

All statistical analyses were done using Statistica 7.0 software. Wilcoxon signed rank test was used to assess the significance of changes of walking-based measures after IVMP. Differences were considered significant at P<0.05. Responsiveness was determined from Time 1 and 2 data by calculating both effect size (ES: mean change score divided by SD of admission scores) [24] and standardized response means (SRM: mean change score divided by SD of change scores) [25], since they can produce different values [26]. They were interpreted using Cohen’s criteria (values of 0.2–0.49 were defined as small, those of 0.5–0.79 as moderate and that of 0.8 and greater as large) [27]. The relative responsiveness of competing walking-based measures (EDSS, MSWS-12, 2-minTW, SSST, 25FWT and SAM parameters) was determined by computing their relative efficiency (RE) [28]. RE estimates the extent to which 1 measure is more or less efficient at detecting change relative to another measure. We computed RE as pair-wise squared Z values from Wilcoxon signed ranks test. The walking-based measure with the largest Z value was chosen as the denominator for the pair-wise calculation. This measure has a measurement precision of 100% and the other values are estimated as a percentage of the most responsive measure. The effect of different walking-based measure responsiveness on sample size estimates was evaluated by computing the number of patients required for each measure to detect the same effect of IVMP. The computed values are relative to 100 patients using the most responsive walking-based measure [100×{(Z value of walking measure with largest Z value/Z value of other walking measure)2}] (11).

Results

The demographic and clinical characteristics of patients are presented in Table 1. A total of 49 patients were studied. The group consisted of 39 women and 10 men, with a median age of 35 years (range 18–56 years), median disease duration of 8 years (range 1.3–27), median EDSS of 3.0 (range 1.5–6.0) before treatment, and with a median Body Mass index of 22.7 (range 16.7–32.3). Most patients are well-educated (92% had a high school or college degree) and not actively employed (61% were unemployed or retired).

Table 2 presents statistically significant differences of all walking-based measures applied, indicating significant improvement of walking ability 1 month after relapses following steroid pulses.

Different responsiveness between applied walking-based measures is showed in Table 3. Both the patient-rated (MSWS-12) and clinician-rated (EDSS) scales showed large degrees of responsiveness as determined by SRM (1.05 and 1.29, respectively) and large or moderate responsiveness as determined by ES (1.02 and 0.69, respectively). Walk tests (2-minTW, SSST and 25-FTW) showed large and moderate responsiveness as determined by SRM (0.89, 0.69 or 0.55, respectively) and moderate or small responsiveness as determined by ES (0.54, 0.31 and 0.27, respectively). The real-world ambulation measured by SAM had small responsiveness of all parameters except for average steps in 1 minute and average peak activity index, which had moderate responsiveness as determined by SRM (0.70 and 0.53, respectively).

Table 4 presents slightly different results regarding RE. The EDSS had the largest Z value and was chosen as the denominator for the pair-wise calculation. This scale had measurement precision of 100%. The 2-minTW, MSWS-12, SSST and 25FWT had measurement precision of 95.1%, 82.4%, 75% and 68.3%, respectively. SAM parameters had RE between 60% for average steps in 1 minute and 15.5% for percentage of time spent in low step activity.

Table 5 displays RE of SAM parameters, which were obtained by proprietary software. The average steps in 1 minute had the largest Z value; this was chosen as the denominator for pair-wise calculation and had measurement precision of 100%. Average steps in 5 minutes, average peak activity index and average daily step count had measurement precision of 77.5%, 70% and 68.4%, respectively. The worst RE had percentage of time spent in medium step activity and percentage of time spent in low step activity, with RE of 36.8% and 25.8%, respectively.

Different walking-based measures as sample size estimates required for each measure to detect the same effect of IVMP are shown in Table 6. The number of patients required to detect the improvement detected by the EDSS ranged from 105 for 2-minTW to 453 and 647 patients for percentage of time spent in medium and low step activity and percentage of time spent in low step activity. .

Discussion

Walking difficulties are very common for MS patients [2,3]; therefore the selection of appropriate walking-based outcome measures is essential to evaluate response to treatment, as well as disease progression sensitivity. The aim of this study was to examine the effects of IVMP on the recovery of walking ability in patients experiencing MS relapses and to compare the responsiveness of walking-based measures that might be used in MS clinical trails. As the importance of responsiveness lies in the balance between statistical power and sample size [28], the next step taken was to examine the potential implications for clinical trials of using walking-based measures with different levels of responsiveness.

In this study all applied methods of walking assessment indicated significant improvement of walking ability in patients with walking difficulties caused by MS relapse 1 month after IVMP. Two previous randomized, double-blind and placebo-controlled studies [29,30] convincingly demonstrated, by EDSS scoring, that IVMP accelerates clinical recovery from relapse of RRMS 1 month after treatment. As the EDSS is strongly biased toward walking, our finding is not surprising.

However, the responsiveness of different walking-based measures varied markedly in terms of effect sizes, relative efficiency and implication for sample size estimation. The highest responsiveness was obtained by MSWS-12, EDSS and 2-minTW. Regarding the RE, the most successful measures were EDSS, 2-minTW and MSWS-12. The potential impact of different responsiveness on sample size estimation showed that the number of patients required to detect the same improvement of walking ability obtained by IVMP ranged from 1 (EDDS) to 6.5 (percentage of time spent in low step activity).

The results confirmed the high responsiveness of MSWS-12. The MSWS-12 is the newest multi-item rating scale of walking ability in MS patients that combines patients’ perspectives with psychometric methods. MSWS-12, as the other patient-rated scale, has limitations in self-report biases and recall. Our study scales was unblinded, and it is possible that the high responsiveness results reflect the fact that patients expected a change in walking ability to occur with therapy. Furthermore, a relatively short period between the 2 tests using walking-based measures was set to minimize the impact of seasonality on continuous monitoring of long-term walking. Therefore, we suppose that the practice effects, particularly because of recall, familiarity and motivation, had an impact on our results. However, in this study and in the original publication of Hobart et al. [20], the responsiveness of MSWS-12 is very similar.

The EDSS is the best known and the most widely used clinician-rated scale. The EDSS has been criticized on several counts, including, among others, having a limited responsiveness as pointed in Hobart et al. [9]. Contrary to the results they obtained our study has yielded different ones. A possible explanation for this discrepancy is likely due to the differences between the 2 cohorts, as well as therapy and inclusion criteria. Our study exclusively included RRMS patients who were selected for steroid therapy (in Hobart’s study more than half of patients had primary or secondary progressive MS and they were treated with physical therapy), and were less disabled having single restriction of MS relapse with walking difficulties.

Numerous standardized walking tests of lower extremity function are used with MS patients. Regardless of their advantages (quickness and inexpensiveness), there are also some disadvantages (single activity execution within a limited time frame, non-familiar environment, dependence on exact instructions and an unquantifiable impact of the observer). The 25FWT reflects only the ability to walk a given distance and walking speed. In our study, the responsiveness of 25FWT is small and is very similar to the responsiveness of 25FWT in Hobart et al. [20]. The responsiveness of the SSST has never been studied, and we suppose it is higher than the responsiveness of the 25FWT. The SSST also reflects the ability to walk a given distance and walking speed, but it depends more on coordination, balance and ease of abduction in the hip than does the 25FTW. The range of measurement of SSST is much wider and its floor effect is less pronounced than that of the 25FTW [22]. We confirmed the higher responsiveness of the SSST according to the 25FWT in this sample of patients. However, the responsiveness of SSST determined by SRM is moderate and that determined by EF is small. Responsiveness of 2-minTW in MS patients with relapse has never been studied. The results reveal that out of all applied walking tests, the 2-minTW has the best responsiveness (large responsiveness determined by SRM and moderate responsiveness determined by EF).

In recent years, accelerometer-based technology has enabled reliable and valid data recording of frequency and intensity of walking over continuous time intervals [31,32]. Continuous walking monitoring provides a direct and objective measure of mobility in a community setting. In this study, the SAM outputs have small responsiveness. Exceptions are the 2 parameters reflecting burst walking activity (average steps in 1 minute and average peak activity index), which have moderate responsiveness determined by SRM. The obtained result indicating that SAM parameters show relatively small responsiveness was actually expected, because numerous personal and environmental factors affect everyday walking [33,34].

This study has some limitations concerning the generalizability and direct applicability of our results to clinical trials in MS. Firstly, the data were not collected within the context of a randomized controlled trail. Secondly, we compared several walking-based measures in a small sample from 1 clinical site. Thirdly, we examined walking-based measures only in a sample of MS patients who were selected for steroid therapy and with the single restriction of having MS relapse with walking difficulties. Fourthly, the relatively short period between the 2 tests involves the potential practice effects of all walking-based tests, not just the MSWS-12. Another limitation is that we studied only ”positive“ responsiveness of walking-based measures. Nevertheless, showing that the variable responsiveness of walking-based measures has substantial implications for clinical trials, we hope this study will contribute to creation of a body of knowledge for evidence-based selection of outcome measures [15].

Conclusions

Further evaluations of responsiveness of walking-based measures for a more modest treatment [5,35–37] of walking difficulties in MS patients are suggested. The “negative“ responsiveness of walking-based measures for evaluation of walking ability worsening that may occur over time might be the topic of another research study. Finally, the impact of practice effects on responsiveness of walking-based measures in MS patients should be quantified.