Advertisement

Development and validation of the self-reported disability status scale (SRDSS) to estimate EDSS-categories

Open AccessPublished:April 27, 2020DOI:https://doi.org/10.1016/j.msard.2020.102148

      Abstract

      Background

      Clinician-assessed Expanded Disease Status Scale (EDSS) is gold standard in clinical investigations but normally unavailable in population-based, patient-centred MS-studies. Our objective was to develop a self-reported gait measure reflecting EDSS-categories.

      Methods

      We developed the self-reported disability status scale (SRDSS) with three categories (≤3.5, 4–6.5, ≥7) based on three mobility-related questions. The SRDSS was determined for 173 persons with MS and validated against clinical EDSS to calculate sensitivity and specificity.

      Results

      Accuracy was 88.4% (153 correctly classified) and weighted kappa 0.73 (0.62–0.84). Sensitivity/specificity-pairs were 94.5%/77.8%, 69.0%/94.7% and 100%/98.2% for SRDSS ≤3.5, 4–6.5 and ≥7, respectively.

      Conclusions

      Self-reported SRDSS approximates EDSS-categories well and fosters comparability between clinical and population-based studies.

      Keywords

      1. Introduction

      The expanded disability status scale (EDSS) is the most commonly used outcome measure in clinical research in the field of MS. Despite acknowledged shortcomings (gait-focused, inaccurate picture of cognition, substantial interrater-variability, impairment-centred), the EDSS is still considered the gold standard for measuring disability (
      • Bovis F.
      • Signori A.
      • Carmisciano L.
      • Maietta I.
      • Steinerman J.R.
      • Li T.
      • Tansy A.P.
      • Sormani M.P.
      Expanded disability status scale progression assessment heterogeneity in multiple sclerosis according to geographical areas.
      ;
      • Meyer-Moock S.
      • Feng Y.S.
      • Maeurer M.
      • Dippel F.W.
      • Kohlmann T.
      Systematic literature review and validity evaluation of the Expanded Disability Status Scale (EDSS) and the Multiple Sclerosis Functional Composite (MSFC) in patients with multiple sclerosis.
      ;
      • Twork S.
      • Wiesmeth S.
      • Spindler M.
      • Wirtz M.
      • Schipper S.
      • Pöhlau D.
      • Klewer J.
      • Kugler J.
      Disability status and quality of life in multiple sclerosis: non-linearity of the Expanded Disability Status Scale (EDSS).
      ). By contrast, the EDSS bears little meaning to persons with MS (PwMS). In a questionnaire of the Swiss MS Registry (SMSR), only 11% knew their EDSS-score (unpublished data). This lack of knowledge amongst PwMS about the own EDSS poses substantial challenges for observational studies relying on patient self-reports such as the SMSR or the UK MS Registry, because this hinders interstudy comparability (
      • Ford D.V.
      • Jones K.H.
      • Middleton R.M.
      • Lockhart-Jones H.
      • Maramba I.D.C.
      • Noble G.J.
      • Osborne L.A.
      • Lyons R.A.
      The feasibility of collecting information from people with Multiple Sclerosis for the UK MS Register via a web portal: characterising a cohort of people with MS.
      ;

      Puhan, M.A., Steinemann, N., Kamm, C.P., Müller, S., Kuhle, J., Kurmann, R., Calabrese, P., Kesselring, J., von Wyl, V., 2018. A digitally facilitated citizen-science driven approach accelerates participant recruitment and increases study population diversity. Swiss Med Wkly. 148, w14623. doi:10.4414/smw.2018.14623.

      ;
      • Steinemann N.
      • Kuhle J.
      • Calabrese P.
      • Kesselring J.
      • Disanto G.
      • Merkler D.
      • Pot C.
      • Ajdacic-Gross V.
      • Rodgers S.
      • Puhan M.A.
      • von Wyl V.
      The Swiss Multiple Sclerosis Registry (SMSR): study protocol of a participatory, nationwide registry to promote epidemiological and patient-centered MS research.
      ).
      Although self-reported EDSS-proxy measures have been proposed, no instrument has emerged as a standard (
      • Collins C.D.E.
      • Ivry B.
      • Bowen J.D.
      • Cheng E.M.
      • Dobson R.
      • Goodin D.S.
      • Lechner-Scott J.
      • Kappos L.
      • Galea I.
      A comparative analysis of Patient-Reported Expanded Disability Status Scale tools.
      ). Therefore, there is still a need for short, reliable, and robust instruments for self-assessments of disability status that offer comparability with the ubiquitous EDSS. Our aim was 1) to develop a patient-based proxy-measure of easy-to-assess information which reflects predefined, clinical EDSS-categories and 2) to validate the measure with clinical EDSS-values.

      2. Methods

      The self-reported information was available from the SMSR, which is a prospective, patient-centred, ongoing study of adult PwMS in Switzerland (n = 2159, as of 25 November 2019) (

      Puhan, M.A., Steinemann, N., Kamm, C.P., Müller, S., Kuhle, J., Kurmann, R., Calabrese, P., Kesselring, J., von Wyl, V., 2018. A digitally facilitated citizen-science driven approach accelerates participant recruitment and increases study population diversity. Swiss Med Wkly. 148, w14623. doi:10.4414/smw.2018.14623.

      ;
      • Steinemann N.
      • Kuhle J.
      • Calabrese P.
      • Kesselring J.
      • Disanto G.
      • Merkler D.
      • Pot C.
      • Ajdacic-Gross V.
      • Rodgers S.
      • Puhan M.A.
      • von Wyl V.
      The Swiss Multiple Sclerosis Registry (SMSR): study protocol of a participatory, nationwide registry to promote epidemiological and patient-centered MS research.
      ). The clinical EDSS-values were available from two sources: on the one hand, key information such as diagnostic dates, disease severity, symptoms and EDSS were abstracted from medical records on site in clinics and practices for a 10% subpopulation of the SMSR (
      • Steinemann N.
      • Kuhle J.
      • Calabrese P.
      • Kesselring J.
      • Disanto G.
      • Merkler D.
      • Pot C.
      • Ajdacic-Gross V.
      • Rodgers S.
      • Puhan M.A.
      • von Wyl V.
      The Swiss Multiple Sclerosis Registry (SMSR): study protocol of a participatory, nationwide registry to promote epidemiological and patient-centered MS research.
      ). On the other hand, double participation in the SMSR and the clinic-based Swiss MS cohort study (SMSC), which includes EDSS-scores, allowed the linkage of records (
      • Disanto G.
      • Benkert P.
      • Lorscheider J.
      • Mueller S.
      • Vehoff J.
      • Zecca C.
      • Ramseier S.
      • Achtnichts L.
      • Findling O.
      • Nedeltchev K.
      • Radue E.
      • Sprenger T.
      • Stippich C.
      • Derfuss T.
      • Louvion J.-.F.
      • Kamm C.P.
      • Mattle H.P.
      • Lotter C.
      • Du Pasquier R.
      • Schluep M.
      • Pot C.
      • Lalive P.H.
      • Yaldizli Ö.
      • Gobbi C.
      • Kappos L.
      • Kuhle J.
      The Swiss Multiple Sclerosis Cohort-Study (SMSC): A Prospective Swiss Wide Investigation of Key Phases in Disease Evolution and New Treatment Options.
      ). Where both data sources were available, the SMSC data were included (SMSR: 71%, SMSC: 29%). The study was approved by the ethics committee of the canton of Zurich (PB-2016–00894, BASEC 2019–01027) and written informed consent was obtained from all participants (
      • Steinemann N.
      • Kuhle J.
      • Calabrese P.
      • Kesselring J.
      • Disanto G.
      • Merkler D.
      • Pot C.
      • Ajdacic-Gross V.
      • Rodgers S.
      • Puhan M.A.
      • von Wyl V.
      The Swiss Multiple Sclerosis Registry (SMSR): study protocol of a participatory, nationwide registry to promote epidemiological and patient-centered MS research.
      ).
      The self-reported disability status scale (SRDSS) defined in this study was constructed to represent mobility-centred descriptions of predefined EDSS-categories (
      • Wallin M.T.
      • Culpepper W.J.
      • Nichols E.
      • Bhutta Z.A.
      • Gebrehiwot T.T.
      • Hay S.I.
      • Khalil I.A.
      • Krohn K.J.
      • Liang X.
      • Naghavi M.
      • Mokdad A.H.
      • Nixon M.R.
      • Reiner R.C.
      • Sartorius B.
      • Smith M.
      • Topor-Madry R.
      • Werdecker A.
      • Vos T.
      • Feigin V.L.
      • Murray C.J.L.
      Global, regional, and national burden of multiple sclerosis 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016.
      ). It was based on three self-reported questions with no missing values allowed. The first question concerned the distance PwMS could walk in flat terrain (<10 m, 10 to 500 m, ≥500 m), the second whether PwMS used any walking aids (cane or rollator), and the third if PwMS used a wheelchair. Following the pre-defined decision tree, the branches resulted in three outcomes (SRDSS ≤3.5, 4–6.5, ≥7) (supplementary material Table A1). Branches resulting in the same outcome were collapsed (Fig. 1).
      Fig 1:
      Fig. 1Simplified decision tree to construct the self-reported disability status scale (SRDSS).
      The comparison was based on 173 PwMS, for which clinical EDSS-assessments were available within ±365 days of the SMSR survey data (median difference: 65 days, inter-quartile range (IQR): 19–122). The agreement between the reported EDSS-category (≤3.5, 4–6.5, ≥7) and the constructed SRDSS (≤3.5, 4–6.5, ≥7) was analysed by calculating the overall accuracy, the linear weights weighted kappa and the group-specific sensitivity and specificity. As a sensitivity analysis, the two EDSS-value sources were analysed separately. All statistical analyses were performed using STATA, version 13.

      3. Results

      The study population consisted of 173 PwMS with a median age of 48 (IQR: 38–55) and 74% women. The predominant disease course was relapsing-remitting MS (73%), followed by secondary-progressive MS (15%), primary-progressive MS (7%) and clinically-isolated syndrome (4%). The median disease duration was 7 years (IQR: 3–15).
      Of the 173 PwMS, the EDSS- and SRDSS-categories matched for 153, resulting in an accuracy of 88.4% and a weighted kappa of 0.73 (0.62–0.84). The sensitivity/specificity pairs were 94.5%/77.8%, 69.0%/94.7% and 100%/98.2% for SRDSS ≤3.5, 4–6.5 and ≥7, respectively (Table 1). The results of the sensitivity analysis, limiting the source of the clinical EDSS to either the medical record abstraction (71%) or the SMSC (29%), were similar (data not shown).
      Table 1Contingency table of the SRDSS and the corresponding EDSS-values.
      SRDSSEDSSSensitivitySpecificity
      ≤3.54–6.5≥7
      ≤3.512110094.5%77.8%
      4–6.5729069.0%94.7%
      ≥7033100%98.2%
      Accuracy88.4%Kappa0.73
      The sensitivity and specificity refer to SRDSS-specific subgroups (≤3.5, 4–6.5, ≥7). EDSS represents “Expanded Disability Status Scale” and SRDSS “Self-Reported Disability Status Scale”. Kappa stands for linear weights weighed Kappa.

      4. Discussion

      We developed the SRDSS, a proxy-measure to estimate EDSS-categories based on easy-to-assess self-reported data. The accuracy of the SRDSS was good (88.4%, weighted kappa 0.73) and only the middle SRDSS-category showed misclassification problems (sensitivity of 69.0%).
      The SRDSS is a convenient tool to estimate EDSS-categories based on three simple questions. These questions can be assessed quickly, and self-reported answers can be given orally, online or on paper. This brevity and flexibility can potentially reduce the under-representation of elderly and severely disabled persons. Furthermore, concerns regarding the self-report of walking distances play only a minor role due to the reduced granularity of the answer options (
      • Skjerbæk A.G.
      • Boesen F.
      • Petersen T.
      • Rasmussen P.V.
      • Stenager E.
      • Nørgaard M.
      • Feys P.
      • Kjeldgaard-Jørgensen M.L.
      • Hvid L.G.
      • Dalgas U.
      Can we trust self-reported walking distance when determining EDSS scores in patients with multiple sclerosis? The Danish MS hospitals rehabilitation study.
      ).
      It is important to note that the purpose of the proxy-measure is not mistaken. The SRDSS is primarily designed to offer an estimate of EDSS-scores in settings where EDSS-scores are not routinely captured or cannot be obtained, e.g. in large, patient-centred registries or regions with poor access to neurologists. In such settings, the SRDSS could help to contextualise results from observational studies, embedding findings in a rough estimate of the neurological status. The SRDSS neither replaces the EDSS nor is it meant as a monitoring tool. For these purposes, it lacks sensitivity and the level of detail needed.
      Further limitations of the SRDSS are related to its respective purpose. An outcome measure, which is heavily weighted towards a pathological extent of the neuroaxis, is unlikely to suffice ecologically relevant MS-studies. Hence, in the context of the latter, it might be more important to document changes that occur frequently instead of infrequently, as the additional noise may compromise power. For these purposes, a self-reported tool might serve as a complementary approach. However, the complexity and duration of some existing tools limit their scope and our newly developed SRDSS might help to close the gap (
      • Leddy S.
      • Hadavi S.
      • McCarren A.
      • Giovannoni G.
      • Dobson R.
      Validating a novel web-based method to capture disease progression outcomes in multiple sclerosis.
      ). Nonetheless, the SRDSS, also due to its focus on mobility, lacks sensitivity for the fine-grained differences at the transitions of categories and therefore is prone to certain misclassifications. However, this can also be partially attributed to a temporal difference between SRDSS- and EDSS-measurements, potentially allowing for relapses and disease progression. Lastly, the SRDSS was compared in 173 PwMS and a wider validation of the tool is encouraged.
      In summary, our study shows that the SRDSS can estimate EDSS-categories based on easy-to-assess self-reported information. It can be easily implemented and has thus the potential to be used in various settings, fosters interstudy comparability and increases the value of PwMS-reported information.

      Financial support

      This work was supported by the Swiss Multiple Sclerosis Society.

      CRediT authorship contribution statement

      Marco Kaufmann: Conceptualization, Methodology, Software, Validation, Formal analysis, Resources, Data curation, Writing – original draft, Visualization, Project administration. Anke Salmen: Investigation, Resources, Writing – review & editing. Laura Barin: Conceptualization, Methodology, Software, Resources, Data curation, Writing – review & editing. Milo Alan Puhan: Conceptualization, Methodology, Resources, Writing – review & editing, Supervision, Project administration, Funding acquisition. Pasquale Calabrese: Investigation, Resources, Writing – review & editing. Christian Philipp Kamm: Investigation, Resources, Writing – review & editing. Claudio Gobbi: Investigation, Resources, Data curation, Writing – review & editing. Jens Kuhle: Investigation, Resources, Data curation, Writing – review & editing. Zina-Mary Manjaly: Investigation, Resources, Writing – review & editing. Vladeta Ajdacic-Gross: Investigation, Resources, Writing – review & editing. Sandra Schafroth: Validation, Investigation, Resources, Data curation, Writing – review & editing. Britta Bottignole: Validation, Investigation, Resources, Data curation, Writing – review & editing. Sabin Ammann: Validation, Investigation, Resources, Data curation, Writing – review & editing. Chiara Zecca: Investigation, Resources, Writing – review & editing. Marcus D'Souza: Validation, Investigation, Resources, Data curation, Writing – review & editing. Viktor von Wyl: Conceptualization, Methodology, Software, Validation, Formal analysis, Resources, Data curation, Writing – original draft, Supervision, Project administration, Funding acquisition.

      Declaration of Conflict of Interests

      AS received speaker honoraria and/or travel compensation for activities with Almirall Hermal GmbH, Biogen, Merck, Novartis, Roche, and Sanofi Genzyme, none related to this work.
      CG: The Department of Neurology, Regional Hospital Lugano (EOC), Lugano, Switzerland, receives financial support from Teva, Merck Serono, Biogen, Genzyme, Roche, Celgene, Bayer, and Novartis. The submitted work is not related to these agreements.
      CPK has received honoraria for lectures as well as research support from Biogen, Novartis, Almirall, Bayer Schweiz AG, Teva, Merck, Sanofi Genzyme, Roche, Eli Lilly, Celgene and the Swiss MS Society (SMSG).
      CZ has received compensation for consulting services and for speaking activities from Biogen, Merck, Mylan, Novartis, Teva, Roche, and Sanofi Genzyme.
      JK's institution (University Hospital Basel) received and used exclusively for research support consulting fees from Biogen, Novartis, Protagen AG, Roche, and Teva; speaker fees from Biogen, Genzyme, Novartis, Roche, and the Swiss Multiple Sclerosis Society; travel expenses from Merck Serono, Novartis, and Roche; grants from Bayer AG, Biogen, the ECTRIMS Research Fellowship Programme, Genzyme, Merck, Novartis, Roche, the Swiss Multiple Sclerosis Society, the Swiss National Research Foundation (320,030_160,221), and the University of Basel.
      MDS has received travel support from Bayer AG, Biogen, Teva Pharmaceuticals and Sanofi Genzyme and research support from the University Hospital Basel.
      PC has received honoraria for speaking at scientific meetings, serving at scientific advisory boards and consulting activities from: Abbvie, Actelion, Almirall, Bayer-Schering, Biogen Idec, EISAI, Disease burden of MS Genzyme, Lundbeck, Merck Serono, Novartis, Pfizer, Teva, and Sanofi-Aventis; his research is also supported by the Swiss Multiple Sclerosis Society, the Swiss National Research Foundation and the SOFIA Foundation.
      BB, LB, MAP, MK, SAS, SA, VA, VvW and ZMM declare that there is no conflict of interest.

      Acknowledgements

      The authors wish to thank the Swiss Multiple Sclerosis Society for funding the Swiss MS Registry (SMSR). Moreover, we thank the study participants who not only contributed data but who are also absolutely instrumental in all aspects of study design and conduct of the SMSR. We further thank the members of the SMSR Data Centre at the Epidemiology, Biostatistics & Prevention Institute of the University of Zurich and the Swiss Multiple Sclerosis Cohort Study for the excellent collaboration.
      Members of the Swiss Multiple Sclerosis Registry are: Bernd Anderseck, Pasquale Calabrese, Andrew Chan, Britta Engelhardt, Claudio Gobbi, Roger Häussler, Christian P. Kamm, Susanne Kägi, Jürg Kesselring (President), Jens Kuhle (Chair of Clinical and Laboratory Research Committee), Roland Kurmann, Christoph Lotter, Marc Lutz, Kurt Luyckx, Doron Merkler, Patricia Monin, Stefanie Müller, Krassen Nedeltchev, Caroline Pot, Milo A. Puhan, Irene Rapold, Anke Salmen, Sven Schippling, Claude Vaney (Chair of Patient- and Population Research Committee), Viktor von Wyl (Chair of IT and Data Committee), Chiara Zecca.
      The Swiss MS Registry is supported by the scientific advisory board of the Swiss MS Society.

      Appendix. Supplementary materials

      References

        • Bovis F.
        • Signori A.
        • Carmisciano L.
        • Maietta I.
        • Steinerman J.R.
        • Li T.
        • Tansy A.P.
        • Sormani M.P.
        Expanded disability status scale progression assessment heterogeneity in multiple sclerosis according to geographical areas.
        Ann. Neurol. 2018; 84: 621-625https://doi.org/10.1002/ana.25323
        • Collins C.D.E.
        • Ivry B.
        • Bowen J.D.
        • Cheng E.M.
        • Dobson R.
        • Goodin D.S.
        • Lechner-Scott J.
        • Kappos L.
        • Galea I.
        A comparative analysis of Patient-Reported Expanded Disability Status Scale tools.
        Mult. Scler. J. 2016; 22: 1349-1358https://doi.org/10.1177/1352458515616205
        • Disanto G.
        • Benkert P.
        • Lorscheider J.
        • Mueller S.
        • Vehoff J.
        • Zecca C.
        • Ramseier S.
        • Achtnichts L.
        • Findling O.
        • Nedeltchev K.
        • Radue E.
        • Sprenger T.
        • Stippich C.
        • Derfuss T.
        • Louvion J.-.F.
        • Kamm C.P.
        • Mattle H.P.
        • Lotter C.
        • Du Pasquier R.
        • Schluep M.
        • Pot C.
        • Lalive P.H.
        • Yaldizli Ö.
        • Gobbi C.
        • Kappos L.
        • Kuhle J.
        The Swiss Multiple Sclerosis Cohort-Study (SMSC): A Prospective Swiss Wide Investigation of Key Phases in Disease Evolution and New Treatment Options.
        PLoS ONE. 2016; 11e0152347https://doi.org/10.1371/journal.pone.0152347
        • Ford D.V.
        • Jones K.H.
        • Middleton R.M.
        • Lockhart-Jones H.
        • Maramba I.D.C.
        • Noble G.J.
        • Osborne L.A.
        • Lyons R.A.
        The feasibility of collecting information from people with Multiple Sclerosis for the UK MS Register via a web portal: characterising a cohort of people with MS.
        BMC Med. Inform. Decis. Mak. 2012; 12https://doi.org/10.1186/1472-6947-12-73
        • Leddy S.
        • Hadavi S.
        • McCarren A.
        • Giovannoni G.
        • Dobson R.
        Validating a novel web-based method to capture disease progression outcomes in multiple sclerosis.
        J. Neurol. 2013; 260: 2505-2510https://doi.org/10.1007/s00415-013-7004-1
        • Meyer-Moock S.
        • Feng Y.S.
        • Maeurer M.
        • Dippel F.W.
        • Kohlmann T.
        Systematic literature review and validity evaluation of the Expanded Disability Status Scale (EDSS) and the Multiple Sclerosis Functional Composite (MSFC) in patients with multiple sclerosis.
        BMC Neurol. 2014; 14: 1-10https://doi.org/10.1186/1471-2377-14-58
        • Skjerbæk A.G.
        • Boesen F.
        • Petersen T.
        • Rasmussen P.V.
        • Stenager E.
        • Nørgaard M.
        • Feys P.
        • Kjeldgaard-Jørgensen M.L.
        • Hvid L.G.
        • Dalgas U.
        Can we trust self-reported walking distance when determining EDSS scores in patients with multiple sclerosis? The Danish MS hospitals rehabilitation study.
        Mult. Scler. J. 2019; 25: 1653-1660https://doi.org/10.1177/1352458518795416
        • Steinemann N.
        • Kuhle J.
        • Calabrese P.
        • Kesselring J.
        • Disanto G.
        • Merkler D.
        • Pot C.
        • Ajdacic-Gross V.
        • Rodgers S.
        • Puhan M.A.
        • von Wyl V.
        The Swiss Multiple Sclerosis Registry (SMSR): study protocol of a participatory, nationwide registry to promote epidemiological and patient-centered MS research.
        BMC Neurol. 2018; 18: 1-10https://doi.org/10.1186/s12883-018-1118-0
        • Twork S.
        • Wiesmeth S.
        • Spindler M.
        • Wirtz M.
        • Schipper S.
        • Pöhlau D.
        • Klewer J.
        • Kugler J.
        Disability status and quality of life in multiple sclerosis: non-linearity of the Expanded Disability Status Scale (EDSS).
        Health Qual. Life Outcomes. 2010; 8: 8-13https://doi.org/10.1186/1477-7525-8-55
        • Wallin M.T.
        • Culpepper W.J.
        • Nichols E.
        • Bhutta Z.A.
        • Gebrehiwot T.T.
        • Hay S.I.
        • Khalil I.A.
        • Krohn K.J.
        • Liang X.
        • Naghavi M.
        • Mokdad A.H.
        • Nixon M.R.
        • Reiner R.C.
        • Sartorius B.
        • Smith M.
        • Topor-Madry R.
        • Werdecker A.
        • Vos T.
        • Feigin V.L.
        • Murray C.J.L.
        Global, regional, and national burden of multiple sclerosis 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016.
        Lancet Neurol. 2019; 18: 269-285https://doi.org/10.1016/S1474-4422(18)30443-5
      1. Puhan, M.A., Steinemann, N., Kamm, C.P., Müller, S., Kuhle, J., Kurmann, R., Calabrese, P., Kesselring, J., von Wyl, V., 2018. A digitally facilitated citizen-science driven approach accelerates participant recruitment and increases study population diversity. Swiss Med Wkly. 148, w14623. doi:10.4414/smw.2018.14623.