Advertisement
Review article| Volume 73, 104628, May 2023

Download started.

Ok

Mobile apps used for people living with multiple sclerosis: A scoping review

  • Zahli Howard
    Affiliations
    School of Indigenous, Medical and Health Sciences, Faculty of Science, Medicine and Health, University of Wollongong, Wollongong, New South Wales, Australia
    Search for articles by this author
  • Khin Than Win
    Affiliations
    School of Computing and Information Technology, Faculty of Engineering and Information Sciences, University of Wollongong, Wollongong, New South Wales, Australia
    Search for articles by this author
  • Vivienne Guan
    Correspondence
    Corresponding author at: School of Indigenous, Medical and Health Sciences, Faculty of Science, Medicine and Health, University of Wollongong, Wollongong, New South Wales, Australia.
    Affiliations
    School of Indigenous, Medical and Health Sciences, Faculty of Science, Medicine and Health, University of Wollongong, Wollongong, New South Wales, Australia

    Illawarra Health and Medical Research Institute, Wollongong, New South Wales, Australia
    Search for articles by this author
Open AccessPublished:March 21, 2023DOI:https://doi.org/10.1016/j.msard.2023.104628

      Highlights

      • People living with multiple sclerosis tend to be involved in app development.
      • Videos/photos, gamification and embedded sensors were incorporated in the apps.
      • Behavior change theories were applied in the apps for fatigue management and physical activity.
      • Limited applied elements of dialogue and social support implied a lack of device-mediated interaction and engagement in the apps.

      Abstract

      Background

      Multiple Sclerosis (MS) is a chronic neurodegenerative disorder. People living with MS (plwMS) require long-term, multidisciplinary care in both clinical and community settings. MS-specific mHealth interventions have advanced in the form of clinical treatments, rehabilitation, disease monitoring and self-management of disease. However, mHealth interventions for plwMS appear to have limited proof of clinical efficacy. As native mobile apps target specific mobile operating systems, they tend to have better interactive designs leveraging platform-specific guidelines. Thus, to improve such efficacy, it is pivotal to explore the design characteristics of native mobile apps used for plwMS.

      Objectives

      This study aimed to explore the design characteristics of native mobile apps used for adults living with MS in academic settings.

      Methods

      A scoping review of studies was conducted. A literature search was performed through PubMed, CINAHL, MEDLINE and Cochrane Library. Per native mobile apps, characteristics, persuasive technology elements and evaluations were summarized.

      Results

      A total of 14 native mobile apps were identified and 43% of the identified apps were used for data collection (n=6). Approximately 70% of the included apps involved users (plwMS) whilst developing (n=10). A total of three apps utilized embedded sensors. Videos or photos were used for physical activity interventions (n=2) and gamification principles were applied for cognitive and/or motor rehabilitation interventions (n=3). Behavior change theories were integrated into the design of the apps for fatigue management and physical activity. Regarding persuasive technology, the design principles of primary support were applied across all identified apps. The elements of dialogue support and social support were the least applied. The methods for evaluating the identified apps were varied.

      Conclusion

      The findings suggest that the identified apps were in the early stages of development and had a user-centered design. By applying the persuasive systems design model, interaction design qualities and features of the identified mobile apps in academic settings were systematically evaluated at a deeper level. Identifying the digital functionality and interface design of mobile apps for plwMS will help researchers to better understand interactive design and how to incorporate these concepts in mHealth interventions for improvement of clinical efficacy.

      Keywords

      1. Introduction

      Multiple sclerosis (MS) affected 2.8 million people worldwide in 2020, which is 30% higher than in 2013 (
      • Walton C.
      • et al.
      Rising prevalence of multiple sclerosis worldwide: Insights from the Atlas of MS, third edition.
      ). MS is an autoimmune and neurodegenerative disease affecting the central nervous system, which is often diagnosed in young adults (
      • Reich D.S.
      • Lucchinetti C.F.
      • Calabresi P.A.
      Multiple sclerosis.
      ). Physical disability has been used as a primary indicator of MS health clinically (
      • Reich D.S.
      • Lucchinetti C.F.
      • Calabresi P.A.
      Multiple sclerosis.
      ). Symptoms (e.g., fatigue, cognitive impairment and pain) are major contributors to health related quality of life amongst people living with MS (plwMS) (
      • Thompson A.J.
      • et al.
      Multiple sclerosis.
      ). The current treatment for MS involves the use of disease-modifying treatments and symptom management (
      • Reich D.S.
      • Lucchinetti C.F.
      • Calabresi P.A.
      Multiple sclerosis.
      ;
      • Thompson A.J.
      • et al.
      Multiple sclerosis.
      ). Thus, plwMS require long-term, multidisciplinary care in both clinical and community settings (
      • Marziniak M.
      • et al.
      The use of digital and remote communication technologies as a tool for multiple sclerosis management: narrative review.
      ). For example, the use of disease-modifying treatments aims at reducing the risk of relapses and halting disability progression, along with symptomatic treatments and rehabilitation in community settings (
      • Reich D.S.
      • Lucchinetti C.F.
      • Calabresi P.A.
      Multiple sclerosis.
      ;
      • Thompson A.J.
      • et al.
      Multiple sclerosis.
      ).
      With just over 4 billion smartphone users worldwide (

      Mobile for Development. The mobile economy 2022. 2022 [cited 2022 20 July]; Available from: https://www.gsma.com/r/somic/.

      ), mHealth offers an obvious opportunity for increasing the reach of disease management and healthcare delivery. With the advances in digital technology, these devices integrate sensors that can obtain signal related to different aspects of medical purposes in different environments. mHealth has the potential for bridging time and distance barriers to health and healthcare delivery that is traditionally delivered face-to-face (
      • De Angelis M.
      • et al.
      Digital technology in clinical trials for multiple sclerosis: systematic review.
      ;
      • Demiris G.
      • et al.
      Patient-centered applications: use of information technology to promote disease management and wellness. A white paper by the AMIA knowledge in motion working group.
      ).
      MS specific mHealth interventions have advanced in the form of clinical treatments, rehabilitation, disease monitoring and self-management of disease. A 2018 analysis in the North America revealed that 2,556 (46.2%) of 5,408 participants with MS reported using mobile apps (
      • Marrie R.A.
      • et al.
      Use of eHealth and mHealth technology by persons with multiple sclerosis.
      ). Among plwMS who reported use of the mobile apps, 98.7% reported use of the apps was helpful (
      • Marrie R.A.
      • et al.
      Use of eHealth and mHealth technology by persons with multiple sclerosis.
      ). Mobile apps have been used to measure treatment adherence and side effects, as well as deliver psychotherapy and motor rehabilitation (
      • De Angelis M.
      • et al.
      Digital technology in clinical trials for multiple sclerosis: systematic review.
      ). However, mHealth interventions for plwMS appear to have limited proof of clinical efficacy. In a controlled environment (e.g., testing a mobile app), good adherence and usability were established in plwMS (
      • Midaglia L.
      • et al.
      Adherence and satisfaction of smartphone- and smartwatch-based remote active testing and passive monitoring in people with multiple sclerosis: nonrandomized interventional feasibility study.
      ). However, adherence to the interventions in real-world settings was low (
      • van der Walt A.
      • et al.
      Developing a digital solution for remote assessment in multiple sclerosis: from concept to software as a medical device.
      ).
      Interaction design of mobile apps plays a critical role in clinical efficacy. Adherence to digital interventions suggests relying heavily on interactive design of digital tools (
      • Bevens W.
      • et al.
      Attrition within digital health interventions for people with multiple sclerosis: systematic review and meta-analysis.
      ;
      • Kelders S.M.
      • et al.
      Persuasive system design does matter: a systematic review of adherence to web-based interventions.
      ). Interactive mobile apps support the way how users communicate and interact with the interventions on the daily basis (
      • Sharp H.
      • Rogers Y.
      • Preece J.
      Interaction Design: Beyond Human-Computer Interaction.
      ). Commercial apps tend to be developed by specialized companies in digital technology. Mobile device-mediate engagement is a key focus of commercial apps (
      • van der Walt A.
      • et al.
      Developing a digital solution for remote assessment in multiple sclerosis: from concept to software as a medical device.
      ). These apps are service-oriented, typically containing an appealing user interface and experience (
      • Khazen W.
      • et al.
      Rethinking the use of mobile apps for dietary assessment in medical research.
      ). However, there are limitation and potential risk to use commercial apps in clinical settings. The evidence reveals that commercial apps tend to have no basis in evidence and lack documentation regarding data privacy and confidentiality (
      • Khazen W.
      • et al.
      Rethinking the use of mobile apps for dietary assessment in medical research.
      ;
      • Arigo D.
      • et al.
      The history and future of digital health in the field of behavioral medicine.
      ). On the other hand, academic apps, developed by experts in the fields provide reliable and scientifically validated digital tools (
      • Khazen W.
      • et al.
      Rethinking the use of mobile apps for dietary assessment in medical research.
      ). However, evidence suggests that there is little support for mobile device-mediate engagement integrated into mobile apps in academic settings (
      • Bevens W.
      • et al.
      Attrition within digital health interventions for people with multiple sclerosis: systematic review and meta-analysis.
      ;
      • Kelders S.M.
      • et al.
      Persuasive system design does matter: a systematic review of adherence to web-based interventions.
      ). The Persuasive Systems Design (PSD) model focuses on features and quality of interaction design at the interface level (
      • Oinas-Kukkonen H.
      • Harjumaa M.
      Persuasive systems design: key issues, process model, and system features.
      ). The model classifies features of interaction design as primary task support, dialogue support, social support, and credibility support (
      • Oinas-Kukkonen H.
      • Harjumaa M.
      Persuasive systems design: key issues, process model, and system features.
      ). The definitions and examples for design principles of the PSD model are shown in Table 1. By applying this model, the interaction design qualities and features of mobile apps can be systematically evaluated, further exploring their possible influence on device-mediated communication at the interface level.
      Table 1Definitions and examples of design principles proposed in persuasive systems design model*.
      Persuasive systems designDefinitionExample
      Primary task support: Carrying out of user's primary task
      ReductionSystem should reduce effort that users expend with regard to performing their target behaviorA mHealth intervention for fatigue management includes a diary for recording energy levels in the morning and afternoon to divide the task of fatigue management into simple steps
      TunnellingSystem should guide users in the attitude change process by providing means for action that brings them closer to the target behaviorA mHealth intervention for the prevention of cigarette smoking that delivers the content in sequential lessons that can only be accessed when the previous lesson is completed
      TailoringSystem should provide tailored information for its user groupsA mHealth intervention for offering different information content for different user groups, e.g., information for beginners
      PersonalizedSystem should offer personalized content and services for its usersA mHealth intervention for providing information content based on individual's needs.
      Self-monitoringSystem should provide means for users to track their performance or statusA mHealth intervention for improving physical activity to provide a diary to track and view daily step count
      SimulationSystem should provide means for observing the link between the cause and effect with regard to users’ behaviorA mHealth intervention for weight loss includes before-and-after pictures of people
      RehearsalSystem should provide means for rehearsing a target behaviorA mHealth intervention for psychotherapy starts each session with the same important exercise
      Dialogue support: Providing feedback and implementing app-human dialogue support in a manner that helps users keep moving towards their goal or target behavior
      PraiseSystem should use praise via words, images, symbols, or sounds as a way to provide user feedback information based on his/her behaviorA mHealth intervention for healthy eating compliments users by sending automated text- messages for reaching individual goals
      RewardsSystem should provide virtual rewards for users in order to give credit for performing the target behaviorA mHealth intervention for improving physical activity gives users a virtual trophy if they follow their program
      RemindersSystem should remind users of their target behavior during use of the systemA mHealth intervention for weight management sends notifications to its users as daily reminders
      SuggestionSystem should suggest that users carry out behaviors during the system use processA mHealth intervention for healthier eating habits suggests that users eats fruit instead of candy at snack time.
      SimilaritySystem should imitate its users in some specific wayA mHealth intervention for increasing intake of vegetables provides food advice based on food preference
      LikingSystem should have a look and feel that appeals to its usersA mHealth intervention for encouraging intake of fruit using pictures of fruit
      Social roleSystem should adopt a social roleA mHealth intervention to support motor rehabilitation incorporates a virtual specialist to guide users through the exercise
      System credibility support: more credible and thus more persuasive
      TrustworthinessSystem should provide information that is truthful, fair and unbiasedA mHealth intervention for healthy eating provides information related to benefit of eating fruit and vegetables rather than simply providing biased advertising or marketing information on superfoods
      ExpertiseSystem should provide information showing knowledge, experience, and competenceA mHealth intervention for monitoring symptoms in people with multiple sclerosis is developed by neurologists and healthcare professionals
      Surface credibilitySystem should have competent look and feelA mHealth intervention for motor rehabilitation contains a limited and well-justified number of sessions
      Real-world feelSystem should provide information of the organization and/or actual people behind its content and servicesA mHealth intervention for medical data collection in people with multiple sclerosis provides possibilities to contact healthcare professionals through sending feedback or asking questions
      AuthoritySystem should refer to people in the role of authorityA mHealth intervention quotes an authority, such as a statement by government health office
      Third-party endorsementsSystem should provide endorsements from respected sourcesA mHealth intervention for healthy eating shows a certificate logo from the national professional organization of dietetics practice
      VerifiabilitySystem should provide means to verify the accuracy of site content via outside sourcesA mHealth intervention for psychotherapy is supported by offering links to evidence of the exercises
      Social support: motivates users by leveraging social influence
      Social learningSystem should provide means to observe other users who are performing their target behavior and to see the outcomes of their behaviorsAn mHealth intervention for weight management provides the option to share the self-monitoring journal of physical activity on the discussion board
      Social comparisonSystem should provide means for comparing performance with the performance of other usersA mHealth intervention for improving physical activity, users can share and compare information related to their daily step count via instant messaging application
      Normative influenceSystem should provide means for gathering together users who have the same goal and make them feel normsAn mHealth intervention for improving intake of vegetables provides feedback on intake by comparing it to the top three users who achieved the goals
      Social facilitationSystem should provide means for discerning other users who are performing the behaviorAn mHealth intervention for improving physical activity can recommend how many users are tracking their step count at the same time as them
      CooperationSystem should provide means for co-operationAn mHealth intervention for motor rehabilitation forms user groups to achieve a group goal each week
      CompetitionSystem should provide means for competing with other usersA mHealth intervention for medication tracking includes a leader board in which users who enter intake of medications at the right times in a month period receive the highest place
      RecognitionSystem should provide public recognition for users who perform their target behaviorA mHealth intervention for monitoring symptoms in multiple sclerosis shares personal stories of users who have succeeded in their goal behavior are published on the application
      * Adapted from Oinas-Kukkonen, H. and M. Harjumaa, Persuasive systems design: Key issues, process model, and system features. Communications of the association for Information Systems, 2009. 24(1): p. 28Identifying relevant studies.
      Previous literature reviews have been published surrounding mHealth interventions designed for plwMS (
      • Marziniak M.
      • et al.
      The use of digital and remote communication technologies as a tool for multiple sclerosis management: narrative review.
      ;
      • De Angelis M.
      • et al.
      Digital technology in clinical trials for multiple sclerosis: systematic review.
      ;
      • Marrie R.A.
      • et al.
      Use of eHealth and mHealth technology by persons with multiple sclerosis.
      ;
      • Bevens W.
      • et al.
      Attrition within digital health interventions for people with multiple sclerosis: systematic review and meta-analysis.
      ;
      • Lavorgna L.
      • et al.
      e-Health and multiple sclerosis: An update.
      ). However, the literature does not distinguish between web-based and native mobile apps. There are a number of strategies to develop an app using mobile devices, such as native mobile apps and web-based mobile apps (e.g., mobile web apps, web-based hybrid mobile apps and progressive web apps) (
      • Malavolta I.
      Beyond native apps: web technologies to the rescue! (keynote).
      ). As mobile operating systems are dominated by iOS and Android, native mobile apps appear to target these two platforms. Native mobile apps are downloaded directly to the user's device and stored locally from app stores. Platform-specific programming languages, tools and guidelines are used to develop native mobile apps. For example, to develop a native mobile app for iOS operating system, developers are required to follow the Development Guidelines, Design Guidelines and Brand and Marketing Guidelines (

      Apple Developer. App store review guidelines. 2023 [cited 2023 07 March]; Available from: https://developer.apple.com/app-store/review/guidelines/.

      ). It allows to create interactive products carrying rich user experiences, heavy advanced graphics with high performance (
      • Malavolta I.
      Beyond native apps: web technologies to the rescue! (keynote).
      ). Therefore, the present study focused on native mobile apps developed for iOS and/or Android operating systems. The aim of the study was to explore the current state of literature surrounding native mobile apps developed for adults living with MS in academic settings.

      2. Methods

      A scoping review is conducted to determine the scope of the research or to map the available literature on a phenomenon (
      • Munn Z.
      • et al.
      Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach.
      ). To better understand the scope and coverage of evidence on the design characteristics of native mobile apps for plwMS in academic settings, a scoping review was conducted based on the framework proposed by Arksey and O'Malley (
      • Arksey H.
      • O'Malley L.
      Scoping studies: towards a methodological framework.
      ). The review followed five steps, including identifying the research question, identifying relevant studies stage, study selection, charting the data and collating, summarizing and reporting the results (
      • Arksey H.
      • O'Malley L.
      Scoping studies: towards a methodological framework.
      ). This framework provides a systematic method for conducting a scoping review (
      • Munn Z.
      • et al.
      Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach.
      ;
      • Arksey H.
      • O'Malley L.
      Scoping studies: towards a methodological framework.
      ). It ensures the review is transparent and comprehensive (
      • Munn Z.
      • et al.
      Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach.
      ;
      • Arksey H.
      • O'Malley L.
      Scoping studies: towards a methodological framework.
      ). Identifying relevant studies, study selection and charting the data were conducted independently by two authors (ZH and VG.), with any disagreements resolved via consensus. Where consensus could not be reached, a third author was consulted (KTW).

      2.1 Identifying the research question

      Based on the framework, a scoping review employs both broad and more focused questions to provide a direction for the initial identification and eventual selection of relevant studies (
      • Arksey H.
      • O'Malley L.
      Scoping studies: towards a methodological framework.
      ). The broad directive in this review was to explore the features of mobile apps developed for plwMS (aged 18 years) in academic settings.
      The specific research questions that guided this review were as follows:
      • What are the key characteristics of mobile apps developed for adults living with MS in academic settings?
      • Are there any differences in characteristics between the purposes of mobile apps aimed at MS in terms of technology and interaction?
      • How are the mobile apps evaluated?
      A literature search was performed through the databases PubMed, MEDLINE (EBSCO), Cumulative Index to Nursing and Allied Health Literature (EBSCO) and Cochrane Library. Both MEDLINE and PubMed were searched to ensure that recent studies were involved (
      • Rosen L.
      • Suhami R.
      The art and science of study identification: a comparative analysis of two systematic reviews.
      ). The search covered the period from 1st April 2012 to 31st March 2022. The articles within the past 10 years were included in the present review, as mHealth described in articles before 10 years ago suggests to be less comparable with technologies described in newer articles (
      • Lentferink A.J.
      • et al.
      Key components in eHealth interventions combining self-tracking and persuasive ecoaching to promote a healthier lifestyle: a scoping review.
      ). The search strategy was based on two concepts, which were combined with ‘AND’: mobile apps and MS. Where possible Medical Subject Headings in addition to free-text search terms were used in the search (
      • Lentferink A.J.
      • et al.
      Key components in eHealth interventions combining self-tracking and persuasive ecoaching to promote a healthier lifestyle: a scoping review.
      ). An example of the search strategy of PubMed is available in Table 2. Articles were initially processed using Endnote 20 (2022, Endnote 20.3 for macOS) including removal of duplicates and screening, before being transferred into Microsoft Excel (2022, Microsoft Excel version 16.62 for macOS) for full-text review.
      Table 2PubMed search strategy.
      Concept category (combined with AND)Search terms
      Mobile Applications“Mobile Applications" [MeSH Terms] OR "smartphone" [MeSH Terms] OR "telemedicine" [MeSH Terms] OR "mobile*" [Title/Abstract] OR "app" [Title/Abstract] OR "smartphone*" [Title/Abstract] OR "mhealth" [Title/Abstract] OR "telemedicine" [Title/Abstract])
      Multiple sclerosis"Multiple Sclerosis" [MeSH Terms] OR "myelitis, transverse" [MeSH Terms] OR "Optic Neuritis" [MeSH Terms] OR "clinically isolated syndrome" [Title/Abstract] OR "disseminated sclerosis" [Title/Abstract] OR "Multiple Sclerosis" [Title/Abstract] OR "Optic Neuritis" [Title/Abstract] OR "transverse myelitis" [Title/Abstract]
      To be eligible for inclusion in this review, studies were required to meet the following inclusion criteria:
      • Mobile app was developed for iOS and/or Android operating systems.
      • An app was a stand-alone product without external components (e.g., an app without a web-based portal).
      • An academic app was developed by experts in the fields or for research purposes (
        • Khazen W.
        • et al.
        Rethinking the use of mobile apps for dietary assessment in medical research.
        ).
      • The design of apps can be determined in terms of technology and interaction.
      • Adults ( 18 years) diagnosed with any form of MS.
      • Intended for use by plwMS, not health care professionals.
      • Included evaluation(s) (e.g., user-based, expert-based, or using predictive model).
      • The study was published in a peer-reviewed academic journal or report from a scientific conference.
      • Full texts and apps published in English.

      2.2 Study selection

      Articles were screened based on the title and abstract. Any articles with titles or abstracts that did not fit the eligibility criteria were excluded. In the case that an abstract was not available or did not provide sufficient information to draw a conclusion regarding eligibility, the full-text articles were retrieved for further review. Following screening, full-text articles were reviewed against the eligibility criteria. When multiple articles describe the same mobile app, only the latest relevant full article which described the design of the app was included. The reference lists of all systematic reviews that were identified in the original search were checked to identify additional articles that met the inclusion criteria.

      2.3 Charting the data

      The following information was extracted from each included article by using Microsoft Excel (2022, Microsoft Excel version 16.62 for macOS): citation, country, name of the mobile app, purpose of the app, operating system(s), method of development, persuasive technology elements and evaluation (including sample characteristics, duration, used mobile device, setting, method and measured outcomes). The applied elements of persuasive technology within the apps were coded according to the PSD framework of Oinas-Kukkonen and Harjumaa (
      • Babbage D.R.
      • et al.
      MS energize: field trial of an app for self-management of fatigue for people with multiple sclerosis.
      ).
      Owing to focus on the design characteristics of apps, the included studies were assessed for quality using the tool by
      • Hawker S.
      • et al.
      Appraising the evidence: reviewing disparate data systematically.
      . Applying this tool, the abstract and title, introduction and aims, method and data, sampling, data analysis, ethics and bias, results, transferability, and implications and usefulness were evaluated by allocating a score between 1 and 4 described in
      • Hawker S.
      • et al.
      Appraising the evidence: reviewing disparate data systematically.
      . Higher scoring studies indicated higher quality.

      2.4 Collating, summarizing and reporting the results

      Descriptive summaries were applied for data analysis, resulting in an approach of a “narrative review” (
      • Arksey H.
      • O'Malley L.
      Scoping studies: towards a methodological framework.
      ). First, a descriptive summary was used to create a numerical overview of the characteristics of mobile apps developed for plwMS in academic settings. Differences in characteristics between apps were explored in terms of technology (e.g., using sensors), interaction (e.g., behavior change theory/techniques, design principles of the PSD model (
      • Oinas-Kukkonen H.
      • Harjumaa M.
      Persuasive systems design: key issues, process model, and system features.
      )) and evaluations of apps.
      Second, thematic analysis was applied to obtain more insight into the various components’ specific designs based on the purposes of mobile apps. When patterns were observed linking technology and interaction, these components were then identified as key components. Thematic analysis was used to create the data-charting form.

      3. Results

      The search yielded an initial 15,816 articles after the removal of duplicates. A total of 15,710 articles were excluded by screening titles and abstracts, and 82 articles were assessed for eligibility. A total 14 articles were identified for the review. The process of study selection is shown in Fig. 1.

      3.1 Study characteristics

      The characteristics of included mobile apps are reported in Table 3. A total of six identified apps were aimed for data collection (43%) (
      • Lang M.
      • et al.
      PatientConcept app: key characteristics, implementation, and its potential benefit.
      ;
      • Mokhberdezfuli M.
      • Ayatollahi H.
      • Naser Moghadasi A.
      A smartphone-based application for self-management in multiple sclerosis.
      ;
      • Palotai M.
      • et al.
      Usability of a mobile app for real-time assessment of fatigue and related symptoms in patients with multiple sclerosis: observational study.
      ;
      • Pratap A.
      • et al.
      Evaluating the utility of smartphone-based sensor assessments in persons with multiple sclerosis in the real-world using an App (elevateMS): observational, prospective pilot digital health study.
      ;
      • Rudick R.A.
      • et al.
      The Multiple Sclerosis Performance Test (MSPT): an iPad-based disability assessment tool.
      ;
      • Van Hecke W.
      • et al.
      A novel digital care management platform to monitor clinical and subclinical disease activity in multiple sclerosis.
      ), two apps for fatigue management (14%) (
      • Babbage D.R.
      • et al.
      MS energize: field trial of an app for self-management of fatigue for people with multiple sclerosis.
      ;
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ) and two apps for physical activity (14%) (
      • Geurts E.
      • et al.
      WalkWithMe: personalized goal setting and coaching for walking in people with multiple sclerosis.
      ;
      • Thirumalai M.
      • et al.
      TEAMS (Tele-Exercise and Multiple Sclerosis), a tailored telerehabilitation mhealth app: participant-centered development and usability study.
      ). Approximately one third of identified apps were used for multiple purposes (n=4) (
      • Lang M.
      • et al.
      PatientConcept app: key characteristics, implementation, and its potential benefit.
      ;
      • Mokhberdezfuli M.
      • Ayatollahi H.
      • Naser Moghadasi A.
      A smartphone-based application for self-management in multiple sclerosis.
      ;
      • Van Hecke W.
      • et al.
      A novel digital care management platform to monitor clinical and subclinical disease activity in multiple sclerosis.
      ;
      • Tacchino A.
      • et al.
      Design, development, and testing of an app for dual-task assessment and training regarding cognitive-motor interference (CMI-APP) in people with multiple sclerosis: multicenter pilot study.
      ). A total of 64% of identified apps (n=9) were developed for Android operating system (
      • Mokhberdezfuli M.
      • Ayatollahi H.
      • Naser Moghadasi A.
      A smartphone-based application for self-management in multiple sclerosis.
      ;
      • Palotai M.
      • et al.
      Usability of a mobile app for real-time assessment of fatigue and related symptoms in patients with multiple sclerosis: observational study.
      ;
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ;
      • Geurts E.
      • et al.
      WalkWithMe: personalized goal setting and coaching for walking in people with multiple sclerosis.
      ;
      • Thirumalai M.
      • et al.
      TEAMS (Tele-Exercise and Multiple Sclerosis), a tailored telerehabilitation mhealth app: participant-centered development and usability study.
      ;
      • Tacchino A.
      • et al.
      Design, development, and testing of an app for dual-task assessment and training regarding cognitive-motor interference (CMI-APP) in people with multiple sclerosis: multicenter pilot study.
      ;
      • Boukhvalova A.K.
      • Fan O.
      • Weideman A.M.
      • Harris T.
      • Kowalczyk E.
      • Pham L.
      • Kosa P.
      • Bielekova B.
      Smartphone level test measures disability in several neurological domains for patients with multiple sclerosis.
      ;
      • Hsieh K.
      • et al.
      Usability of a fall risk mHealth app for people with multiple sclerosis: mixed methods study.
      ;
      • Tacchino A.
      • et al.
      A new app for at-home cognitive training: description and pilot testing on patients with multiple sclerosis.
      ) and 21% of the apps (n=2) were developed for both Android and iOS operating systems (
      • Lang M.
      • et al.
      PatientConcept app: key characteristics, implementation, and its potential benefit.
      ;
      • Van Hecke W.
      • et al.
      A novel digital care management platform to monitor clinical and subclinical disease activity in multiple sclerosis.
      ). Although five out of 14 of the apps (36%) reported applying a user-centered design (
      • Pratap A.
      • et al.
      Evaluating the utility of smartphone-based sensor assessments in persons with multiple sclerosis in the real-world using an App (elevateMS): observational, prospective pilot digital health study.
      ;
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ;
      • Thirumalai M.
      • et al.
      TEAMS (Tele-Exercise and Multiple Sclerosis), a tailored telerehabilitation mhealth app: participant-centered development and usability study.
      ;
      • Tacchino A.
      • et al.
      Design, development, and testing of an app for dual-task assessment and training regarding cognitive-motor interference (CMI-APP) in people with multiple sclerosis: multicenter pilot study.
      ;
      • Hsieh K.
      • et al.
      Usability of a fall risk mHealth app for people with multiple sclerosis: mixed methods study.
      ), involving users (plwMS) during the development of the apps was reported in additional four apps (29%) (
      • Mokhberdezfuli M.
      • Ayatollahi H.
      • Naser Moghadasi A.
      A smartphone-based application for self-management in multiple sclerosis.
      ;
      • Van Hecke W.
      • et al.
      A novel digital care management platform to monitor clinical and subclinical disease activity in multiple sclerosis.
      ;
      • Geurts E.
      • et al.
      WalkWithMe: personalized goal setting and coaching for walking in people with multiple sclerosis.
      ;
      • Tacchino A.
      • et al.
      A new app for at-home cognitive training: description and pilot testing on patients with multiple sclerosis.
      ). One included app was designed for fall risk assessment and not only followed a user-centered design but also considered the common MS symptoms including fatigue, vision and cognitive impairment whilst developing (
      • Hsieh K.
      • et al.
      Usability of a fall risk mHealth app for people with multiple sclerosis: mixed methods study.
      ). One of the apps developed for fatigue management reported following the specific interface design guidelines of the operating system (
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ). A total of 13 apps had plwMS as a target population (
      • Lang M.
      • et al.
      PatientConcept app: key characteristics, implementation, and its potential benefit.
      ;
      • Mokhberdezfuli M.
      • Ayatollahi H.
      • Naser Moghadasi A.
      A smartphone-based application for self-management in multiple sclerosis.
      ;
      • Palotai M.
      • et al.
      Usability of a mobile app for real-time assessment of fatigue and related symptoms in patients with multiple sclerosis: observational study.
      ;
      • Pratap A.
      • et al.
      Evaluating the utility of smartphone-based sensor assessments in persons with multiple sclerosis in the real-world using an App (elevateMS): observational, prospective pilot digital health study.
      ;
      • Rudick R.A.
      • et al.
      The Multiple Sclerosis Performance Test (MSPT): an iPad-based disability assessment tool.
      ;
      • Van Hecke W.
      • et al.
      A novel digital care management platform to monitor clinical and subclinical disease activity in multiple sclerosis.
      ;
      • Babbage D.R.
      • et al.
      MS energize: field trial of an app for self-management of fatigue for people with multiple sclerosis.
      ;
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ;
      • Geurts E.
      • et al.
      WalkWithMe: personalized goal setting and coaching for walking in people with multiple sclerosis.
      ;
      • Thirumalai M.
      • et al.
      TEAMS (Tele-Exercise and Multiple Sclerosis), a tailored telerehabilitation mhealth app: participant-centered development and usability study.
      ;
      • Tacchino A.
      • et al.
      Design, development, and testing of an app for dual-task assessment and training regarding cognitive-motor interference (CMI-APP) in people with multiple sclerosis: multicenter pilot study.
      ;
      • Boukhvalova A.K.
      • Fan O.
      • Weideman A.M.
      • Harris T.
      • Kowalczyk E.
      • Pham L.
      • Kosa P.
      • Bielekova B.
      Smartphone level test measures disability in several neurological domains for patients with multiple sclerosis.
      ;
      • Hsieh K.
      • et al.
      Usability of a fall risk mHealth app for people with multiple sclerosis: mixed methods study.
      ) and one app was developed for use by cognitive-impaired plwMS (
      • Tacchino A.
      • et al.
      A new app for at-home cognitive training: description and pilot testing on patients with multiple sclerosis.
      ). Most articles scored very good on quality with a median score of 36 out of 40 (interquartile range 34.25, 36.00).
      Table 3Characteristics of included mobile apps.
      Citation and countryNamePurposeOperating system(s)Method of developmentUser involved for development
      • Babbage D.R.
      • et al.
      MS energize: field trial of an app for self-management of fatigue for people with multiple sclerosis.
      ; NZ and UK
      MS EnergizeFatigue managementiOSNot reportedNot reported
      • Boukhvalova A.K.
      • Fan O.
      • Weideman A.M.
      • Harris T.
      • Kowalczyk E.
      • Pham L.
      • Kosa P.
      • Bielekova B.
      Smartphone level test measures disability in several neurological domains for patients with multiple sclerosis.
      ; US
      Level TestMotor rehabilitationAndroidNot reportedNot reported
      • Geurts E.
      • et al.
      WalkWithMe: personalized goal setting and coaching for walking in people with multiple sclerosis.
      ; Belgium
      WalkWithMePhysical activityAndroidNot reported
      • Hsieh K.
      • et al.
      Usability of a fall risk mHealth app for people with multiple sclerosis: mixed methods study.
      ; US
      Steady-MSFall risk assessmentAndroidUser-centered design

      Symptoms including fatigue, vision impairment and cognitive impairment
      • Lang M.
      • et al.
      PatientConcept app: key characteristics, implementation, and its potential benefit.
      ; Germany
      PatientConceptData collection

      Self-management of disease
      Android and iOSNot reportedNot reported
      • Mokhberdezfuli M.
      • Ayatollahi H.
      • Naser Moghadasi A.
      A smartphone-based application for self-management in multiple sclerosis.
      ; Iran
      Not reportedData collection

      Self-management of disease
      AndroidNot reported
      • Palotai M.
      • et al.
      Usability of a mobile app for real-time assessment of fatigue and related symptoms in patients with multiple sclerosis: observational study.
      ; US
      Not reportedData collectionAndroidNot reportedNot reported
      • Pratap A.
      • et al.
      Evaluating the utility of smartphone-based sensor assessments in persons with multiple sclerosis in the real-world using an App (elevateMS): observational, prospective pilot digital health study.
      ; US
      elevateMSData collectioniOSUser-centered design
      • Rudick R.A.
      • et al.
      The Multiple Sclerosis Performance Test (MSPT): an iPad-based disability assessment tool.
      ; US
      MSPTData collectioniOSNot reportedNot reported
      • Tacchino A.
      • et al.
      A new app for at-home cognitive training: description and pilot testing on patients with multiple sclerosis.
      ; Italy
      COGNI-TRAcKCognitive rehabilitationAndroidNot reported
      • Tacchino A.
      • et al.
      Design, development, and testing of an app for dual-task assessment and training regarding cognitive-motor interference (CMI-APP) in people with multiple sclerosis: multicenter pilot study.
      ; Belgium, Italy, Finland and Israel
      CMI-APPCognitive and motor rehabilitationAndroidUser-centered design
      • Thirumalai M.
      • et al.
      TEAMS (Tele-Exercise and Multiple Sclerosis), a tailored telerehabilitation mhealth app: participant-centered development and usability study.
      ; US
      TEAMSPhysical activityAndroidUser-centered design

      Parallel-iterative design
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ; UK
      FACETSFatigue managementAndroidUser-centered design

      Google's Material Design guidelines

      Google user interface guidelines
      • Van Hecke W.
      • et al.
      A novel digital care management platform to monitor clinical and subclinical disease activity in multiple sclerosis.
      ; Belgium
      icompanionData collection

      Disease education

      Treatment reminder

      Consultation pre-visit checklist
      Android and iOSNot reported

      3.2 Digital functionality and behavior change theory/technique

      A total of three apps utilized embedded sensors, including two apps used for data collection (
      • Pratap A.
      • et al.
      Evaluating the utility of smartphone-based sensor assessments in persons with multiple sclerosis in the real-world using an App (elevateMS): observational, prospective pilot digital health study.
      ;
      • Rudick R.A.
      • et al.
      The Multiple Sclerosis Performance Test (MSPT): an iPad-based disability assessment tool.
      ) and one used for motor rehabilitation (
      • Boukhvalova A.K.
      • Fan O.
      • Weideman A.M.
      • Harris T.
      • Kowalczyk E.
      • Pham L.
      • Kosa P.
      • Bielekova B.
      Smartphone level test measures disability in several neurological domains for patients with multiple sclerosis.
      ). The apps used for physical activity intervention utilized videos or photos to present the intervention (
      • Geurts E.
      • et al.
      WalkWithMe: personalized goal setting and coaching for walking in people with multiple sclerosis.
      ;
      • Thirumalai M.
      • et al.
      TEAMS (Tele-Exercise and Multiple Sclerosis), a tailored telerehabilitation mhealth app: participant-centered development and usability study.
      ). The apps used for cognitive and/or motor rehabilitation were reported being gamified (
      • Tacchino A.
      • et al.
      Design, development, and testing of an app for dual-task assessment and training regarding cognitive-motor interference (CMI-APP) in people with multiple sclerosis: multicenter pilot study.
      ;
      • Boukhvalova A.K.
      • Fan O.
      • Weideman A.M.
      • Harris T.
      • Kowalczyk E.
      • Pham L.
      • Kosa P.
      • Bielekova B.
      Smartphone level test measures disability in several neurological domains for patients with multiple sclerosis.
      ;
      • Tacchino A.
      • et al.
      A new app for at-home cognitive training: description and pilot testing on patients with multiple sclerosis.
      ). Behavior change theories were integrated into the design of the apps for fatigue management, which were Cognitive Behavioral Therapy (
      • Babbage D.R.
      • et al.
      MS energize: field trial of an app for self-management of fatigue for people with multiple sclerosis.
      ) and Cognitive Behavioral and Energy Effectiveness Techniques (
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ). Behavior change techniques were integrated into the apps design for physical activity, which were goal setting and coaching for WalkWithMe (
      • Geurts E.
      • et al.
      WalkWithMe: personalized goal setting and coaching for walking in people with multiple sclerosis.
      ) and goal-setting, seeking social support, and overcoming barriers for TEAMS (
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ).

      3.3 Persuasive technology

      In the primary support, the design principles of reduction, tunnelling and tailoring were utilized across all apps, shown in Table 4. Approximately 70% of the apps applied the design principles of personalization and self-monitoring (n=10). For example, the fatigue management apps track sleep and energy level based on users’ needs (
      • Babbage D.R.
      • et al.
      MS energize: field trial of an app for self-management of fatigue for people with multiple sclerosis.
      ;
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ). A total of 36% of apps utilized rehearsal technique (n=5). For example, psychotherapy interventions started each session with the same exercise (
      • Tacchino A.
      • et al.
      Design, development, and testing of an app for dual-task assessment and training regarding cognitive-motor interference (CMI-APP) in people with multiple sclerosis: multicenter pilot study.
      ;
      • Tacchino A.
      • et al.
      A new app for at-home cognitive training: description and pilot testing on patients with multiple sclerosis.
      ). For dialogue support, 93% of the apps applied the design principle of liking (n=13). A total of nine out of 14 apps integrated the design principle of suggestion (64%), and six for reminders (43%). As for credibility, the design principles of trustworthiness, expertise and surface credibility were applied across all apps. The design principles of social support were rarely used in the identified apps.
      Table 4Design principles of persuasive system design by purposes of the mobile apps.
      Data collection (n=6), nFatigue management (n=2), nPhysical activity (n=2), nCognitive rehabilitation (n=1), nMotor rehabilitation (n=1), nCognitive and motor rehabilitation (n=1), nFall risk assessment (n=1), nTotal (n=14), n (%)
      Primary task support
      Reduction622111114 (100)
      Tunnelling622111114 (100)
      Tailoring622111114 (100)
      Personalized421111010 (71)
      Self-monitoring421111010 (71)
      Simulation01000001 (7)
      Rehearsal02110105 (36)
      Dialogue support
      Praise01100002 (14)
      Rewards01000001 (7)
      Reminders51000006 (43)
      Suggestion32101119 (64)
      Liking622011113 (93)
      Social role00100012 (14)
      Credibility support
      Trustworthiness622111114 (100)
      Expertise622111114 (100)
      Surface credibility622111114 (100)
      Real-world feel10000001 (7)
      Social support
      Social learning00100001 (7)
      Social comparison00100001 (7)
      Social facilitation00100001 (7)

      3.4 Evaluations

      There were five of 14 studies applied a cross-sectional study design (36%) for evaluation, presented in Table 5. A total of six studies reported the durations of evaluations were greater than eight weeks (
      • Lang M.
      • et al.
      PatientConcept app: key characteristics, implementation, and its potential benefit.
      ;
      • Pratap A.
      • et al.
      Evaluating the utility of smartphone-based sensor assessments in persons with multiple sclerosis in the real-world using an App (elevateMS): observational, prospective pilot digital health study.
      ;
      • Geurts E.
      • et al.
      WalkWithMe: personalized goal setting and coaching for walking in people with multiple sclerosis.
      ;
      • Tacchino A.
      • et al.
      Design, development, and testing of an app for dual-task assessment and training regarding cognitive-motor interference (CMI-APP) in people with multiple sclerosis: multicenter pilot study.
      ;
      • Boukhvalova A.K.
      • Fan O.
      • Weideman A.M.
      • Harris T.
      • Kowalczyk E.
      • Pham L.
      • Kosa P.
      • Bielekova B.
      Smartphone level test measures disability in several neurological domains for patients with multiple sclerosis.
      ;
      • Tacchino A.
      • et al.
      A new app for at-home cognitive training: description and pilot testing on patients with multiple sclerosis.
      ). A total of nine of the identified apps were evaluated by using smartphones (64%). Tablets were used to evaluate the interventions of cognitive rehabilitation (
      • Tacchino A.
      • et al.
      Design, development, and testing of an app for dual-task assessment and training regarding cognitive-motor interference (CMI-APP) in people with multiple sclerosis: multicenter pilot study.
      ;
      • Tacchino A.
      • et al.
      A new app for at-home cognitive training: description and pilot testing on patients with multiple sclerosis.
      ), while smartphones were employed to evaluate data collection (
      • Lang M.
      • et al.
      PatientConcept app: key characteristics, implementation, and its potential benefit.
      ;
      • Mokhberdezfuli M.
      • Ayatollahi H.
      • Naser Moghadasi A.
      A smartphone-based application for self-management in multiple sclerosis.
      ;
      • Palotai M.
      • et al.
      Usability of a mobile app for real-time assessment of fatigue and related symptoms in patients with multiple sclerosis: observational study.
      ;
      • Pratap A.
      • et al.
      Evaluating the utility of smartphone-based sensor assessments in persons with multiple sclerosis in the real-world using an App (elevateMS): observational, prospective pilot digital health study.
      ). There were eight out of 14 apps evaluated in natural settings (57%). In the controlled settings, a qualitative study tended to be applied to evaluate the apps (n=4) (
      • Rudick R.A.
      • et al.
      The Multiple Sclerosis Performance Test (MSPT): an iPad-based disability assessment tool.
      ;
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ;
      • Thirumalai M.
      • et al.
      TEAMS (Tele-Exercise and Multiple Sclerosis), a tailored telerehabilitation mhealth app: participant-centered development and usability study.
      ;
      • Hsieh K.
      • et al.
      Usability of a fall risk mHealth app for people with multiple sclerosis: mixed methods study.
      ). Approximate 30% of the apps were evaluated by using MS related measurements such as disease stage, disease duration, Expanded Disability Status Scale and self-reported MS severity (n=4) (
      • Pratap A.
      • et al.
      Evaluating the utility of smartphone-based sensor assessments in persons with multiple sclerosis in the real-world using an App (elevateMS): observational, prospective pilot digital health study.
      ;
      • Van Hecke W.
      • et al.
      A novel digital care management platform to monitor clinical and subclinical disease activity in multiple sclerosis.
      ;
      • Tacchino A.
      • et al.
      Design, development, and testing of an app for dual-task assessment and training regarding cognitive-motor interference (CMI-APP) in people with multiple sclerosis: multicenter pilot study.
      ;
      • Boukhvalova A.K.
      • Fan O.
      • Weideman A.M.
      • Harris T.
      • Kowalczyk E.
      • Pham L.
      • Kosa P.
      • Bielekova B.
      Smartphone level test measures disability in several neurological domains for patients with multiple sclerosis.
      ). Approximate 30% of the evaluations (n=3) deployed the System Usability Scale (
      • Babbage D.R.
      • et al.
      MS energize: field trial of an app for self-management of fatigue for people with multiple sclerosis.
      ;
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ;
      • Hsieh K.
      • et al.
      Usability of a fall risk mHealth app for people with multiple sclerosis: mixed methods study.
      ) when using a survey to evaluate the apps (n=9, 64%) (
      • Lang M.
      • et al.
      PatientConcept app: key characteristics, implementation, and its potential benefit.
      ;
      • Mokhberdezfuli M.
      • Ayatollahi H.
      • Naser Moghadasi A.
      A smartphone-based application for self-management in multiple sclerosis.
      ;
      • Palotai M.
      • et al.
      Usability of a mobile app for real-time assessment of fatigue and related symptoms in patients with multiple sclerosis: observational study.
      ;
      • Babbage D.R.
      • et al.
      MS energize: field trial of an app for self-management of fatigue for people with multiple sclerosis.
      ;
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ;
      • Geurts E.
      • et al.
      WalkWithMe: personalized goal setting and coaching for walking in people with multiple sclerosis.
      ;
      • Tacchino A.
      • et al.
      Design, development, and testing of an app for dual-task assessment and training regarding cognitive-motor interference (CMI-APP) in people with multiple sclerosis: multicenter pilot study.
      ;
      • Hsieh K.
      • et al.
      Usability of a fall risk mHealth app for people with multiple sclerosis: mixed methods study.
      ;
      • Tacchino A.
      • et al.
      A new app for at-home cognitive training: description and pilot testing on patients with multiple sclerosis.
      ). A total of 43% of the evaluations focused on the usability of the apps (n=6) (
      • Lang M.
      • et al.
      PatientConcept app: key characteristics, implementation, and its potential benefit.
      ;
      • Mokhberdezfuli M.
      • Ayatollahi H.
      • Naser Moghadasi A.
      A smartphone-based application for self-management in multiple sclerosis.
      ;
      • Babbage D.R.
      • et al.
      MS energize: field trial of an app for self-management of fatigue for people with multiple sclerosis.
      ;
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ;
      • Thirumalai M.
      • et al.
      TEAMS (Tele-Exercise and Multiple Sclerosis), a tailored telerehabilitation mhealth app: participant-centered development and usability study.
      ;
      • Hsieh K.
      • et al.
      Usability of a fall risk mHealth app for people with multiple sclerosis: mixed methods study.
      ).
      Table 5Summary of evaluations of the mobile apps.
      Citation and countryPopulationNumber of participantsAge (years)DurationDeviceSettingMethodOutcome
      • Babbage D.R.
      • et al.
      MS energize: field trial of an app for self-management of fatigue for people with multiple sclerosis.
      ; NZ and UK
      Adults with MS1141 – 59 (range)5-6 weeksSmartphoneNaturalSurvey

      Qualitative study
      Usability

      Perceived usefulness
      • Boukhvalova A.K.
      • Fan O.
      • Weideman A.M.
      • Harris T.
      • Kowalczyk E.
      • Pham L.
      • Kosa P.
      • Bielekova B.
      Smartphone level test measures disability in several neurological domains for patients with multiple sclerosis.
      ; US
      Adults with MSCross-sectional cohort (112 MS patients and 15 healthy volunteers)

      Longitudinal cohort (32 MS patients, 15 healthy volunteers)

      Cross-sectional validation cohort (n=29 MS patients)
      56.4 ± 9.5 (Mean ± SD)4 months for longitudinal cohortSmartphoneNaturalMS related measurementsValidity
      • Geurts E.
      • et al.
      WalkWithMe: personalized goal setting and coaching for walking in people with multiple sclerosis.
      ; Belgium
      Adults with RRMS745 ± 1010 weeksSmartphoneNaturalSurvey

      Qualitative study
      Appreciation

      Effectiveness
      • Hsieh K.
      • et al.
      Usability of a fall risk mHealth app for people with multiple sclerosis: mixed methods study.
      ; US
      Adults with MS10 (5 each X2 iterations)Iteration 1: 53.2 ± 13.1

      Iteration 2: 54.6 ± 8.7
      Cross-sectionalSmartphoneControlledSurvey

      Qualitative study
      Usability
      • Lang M.
      • et al.
      PatientConcept app: key characteristics, implementation, and its potential benefit.
      ; Germany
      Adults with MS95Not reported8 weeksSmartphoneNot reportedSurveyUsefulness

      Usability
      • Mokhberdezfuli M.
      • Ayatollahi H.
      • Naser Moghadasi A.
      A smartphone-based application for self-management in multiple sclerosis.
      ; Iran
      Adults with MS60 MS patientsOnly reported age range of 31–40 years old (MS patients: 36.8 ± 10.3, n=21, 35%)Cross-sectionalSmartphoneControlledSurveyUsability
      • Palotai M.
      • et al.
      Usability of a mobile app for real-time assessment of fatigue and related symptoms in patients with multiple sclerosis: observational study.
      ; US
      Adults with MS5652 ± 92 weeksSmartphoneNaturalSurveyPatient Compliance
      • Pratap A.
      • et al.
      Evaluating the utility of smartphone-based sensor assessments in persons with multiple sclerosis in the real-world using an App (elevateMS): observational, prospective pilot digital health study.
      ; US
      3 groups: Self-referred participants who reported an MS diagnosis, clinic-referred participants with neurologist-confirmed MS, and participants without MS (controls)495 had MS (n=359 self-referred, n=136 clinic-referred), and 134 controls.Self-referred (45.20 ± 11.64), clinic-referred (48.93±11.20) and control (39.34 ± 11.41)12 weeksSmartphoneNaturalMS related measurementsRetention analysis

      Associations between self-reported MS severity and sensor-based active functional tests measurements, and the impact of local weather conditions on disease burden
      • Rudick R.A.
      • et al.
      The Multiple Sclerosis Performance Test (MSPT): an iPad-based disability assessment tool.
      ; US
      Adults with MS and Health controls51 MS patients

      49 healthy controls
      MS patients 46.2 ± 10.1

      Healthy controls 45.7 ± 10.7
      Cross-sectionalTabletControlledQualitative study

      MS related measurements
      Test-retest reliability

      Associations to EDSS, disease stage, disease duration and patient-reported outcomes

      Associations to technician-administered neurological and neuropsychological testing
      • Tacchino A.
      • et al.
      A new app for at-home cognitive training: description and pilot testing on patients with multiple sclerosis.
      ; Italy
      Cognitive-impaired patients with MS1649.06 ± 9.108 weeksTabletNaturalSurveyDisposability-to-use (usability, motivation to use, compliance to treatment)
      • Tacchino A.
      • et al.
      Design, development, and testing of an app for dual-task assessment and training regarding cognitive-motor interference (CMI-APP) in people with multiple sclerosis: multicenter pilot study.
      ; Belgium, Italy, Finland and Israel
      Adults with MS (EDSS 2-5, no relapse past 30 days, MMSE score >26)1552.6 ± 8.68 weeksTabletNaturalSurvey

      MS related measurements
      Feasibility

      Adherence to the program

      Perceived exertion during the training

      Subjective experience regarding the training and the training activities
      • Thirumalai M.
      • et al.
      TEAMS (Tele-Exercise and Multiple Sclerosis), a tailored telerehabilitation mhealth app: participant-centered development and usability study.
      ; US
      Adults with MS with mobility limitationUsability test 1: 3

      Usability test 2: 5
      Not reportedCross-sectionalTabletControlledQualitative studyUsability
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ; UK
      Adults with MS1149 ± 8.41Cross-sectionalSmartphoneControlledSurvey

      Qualitative study
      Usability
      • Van Hecke W.
      • et al.
      A novel digital care management platform to monitor clinical and subclinical disease activity in multiple sclerosis.
      ; Belgium
      Adults with MS1301Not reportedNot reportedNot reportedNaturalMS related measurementsSensitivity to clinical differences between MS Types
      *Abbreviations: MS: Multiple Sclerosis; RRMS: Relapsing-Remitting Multiple Sclerosis; EDSS: Expanded Disability Status Scale; MMSE: Mini Mental Status Exam
      ^ Range

      4. Discussion

      This scoping review identified and explored the current native mobile apps in academic settings designed for plwMS. The majority of the included studies involved users (plwMS) whilst developing. The findings revealed that the digital functionality and interface design of the identified mobile apps were determined by the purpose of the apps. The finding demonstrated that there was an overall underutilization of dialogue support and social support within the identified mobile apps. The methods used for evaluating the mobile apps for plwMS were varied in academic settings.
      The present exploratory results reveal that the design characteristics of the native mobile apps developed for plwMS were selected based on the purposes of the apps. The identified apps utilized smartphone embedded sensors which ensure precision in the collection of data (
      • Pratap A.
      • et al.
      Evaluating the utility of smartphone-based sensor assessments in persons with multiple sclerosis in the real-world using an App (elevateMS): observational, prospective pilot digital health study.
      ;
      • Rudick R.A.
      • et al.
      The Multiple Sclerosis Performance Test (MSPT): an iPad-based disability assessment tool.
      ;
      • Boukhvalova A.K.
      • Fan O.
      • Weideman A.M.
      • Harris T.
      • Kowalczyk E.
      • Pham L.
      • Kosa P.
      • Bielekova B.
      Smartphone level test measures disability in several neurological domains for patients with multiple sclerosis.
      ). Due to the symptoms of MS such as fatigue, pain and cognitive decline, using built-in sensors of devices allows real-time reporting of symptoms and minimizes the user burden. This can be a key tool in monitoring and mitigating symptoms of MS. The apps designed for fatigue management also had defining characteristics. Fatigue management apps for plwMS typically consisted of educational content as well as sleep and/or energy tracking tools (
      • Babbage D.R.
      • et al.
      MS energize: field trial of an app for self-management of fatigue for people with multiple sclerosis.
      ;
      • Thomas S.
      • et al.
      Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
      ). Fatigue management affects up to 80% of plwMS (
      • Lerdal A.
      • et al.
      A prospective study of patterns of fatigue in multiple sclerosis.
      ). The evidence suggests that fatigue plays a role in finding and keeping a job, engaging in home management, child care, and leisure in plwMS (
      • Lisak D.
      Overview of symptomatic management of multiple sclerosis.
      ). Thus, the goals of the identified apps appear to support self-management of fatigue. Furthermore, the identified apps designed for physical activity interventions utilize photo and video demonstrations (
      • Geurts E.
      • et al.
      WalkWithMe: personalized goal setting and coaching for walking in people with multiple sclerosis.
      ;
      • Thirumalai M.
      • et al.
      TEAMS (Tele-Exercise and Multiple Sclerosis), a tailored telerehabilitation mhealth app: participant-centered development and usability study.
      ). Video-based demonstration was proven to be superior to a standard paper-based home exercise program in exercise adherence for plwMS (
      • Motl R.W.
      • et al.
      Randomized controlled trial of an e-learning designed behavioral intervention for increasing physical activity behavior in multiple sclerosis.
      ). This may be important for physical activity as it mirrors what a person would experience in clinic from home (
      • Motl R.W.
      • et al.
      Randomized controlled trial of an e-learning designed behavioral intervention for increasing physical activity behavior in multiple sclerosis.
      ). The apps for cognitive and motor rehabilitation were most likely to be gamified. Gamification is seen as the use of game design elements in non-game contexts to increase motivation to continue use (
      • Deterding S.
      • et al.
      From game design elements to gamefulness: defining "gamification".
      ). They are either designed for plwMS to improve working memory or to assess their cognitive function and how these affected other aspects of their lives (
      • Tacchino A.
      • et al.
      Design, development, and testing of an app for dual-task assessment and training regarding cognitive-motor interference (CMI-APP) in people with multiple sclerosis: multicenter pilot study.
      ;
      • Tacchino A.
      • et al.
      A new app for at-home cognitive training: description and pilot testing on patients with multiple sclerosis.
      ). Cognitive rehabilitation is a key as cognitive impairment affects approximately 65% of plwMS and can have an impact on all areas of a person's life (
      • Longley W.
      • Honan C.
      Cognitive impairment in multiple sclerosis: the role of the general practitioner in cognitive screening and care coordination.
      ). Moreover, using sensors in this type of intervention can be an advancement in the realm of digital health interventions as they can automatically and continuously record high-density data from users (
      • Boukhvalova A.K.
      • Fan O.
      • Weideman A.M.
      • Harris T.
      • Kowalczyk E.
      • Pham L.
      • Kosa P.
      • Bielekova B.
      Smartphone level test measures disability in several neurological domains for patients with multiple sclerosis.
      ).
      Regarding the design features of persuasive technology, the findings suggest that the selected design principles tend to be determined by the purpose of the identified mobile apps. The goals of using mHealth in health and healthcare delivery appear to automate and advance current evidence-based practice leveraging the fast development and maturation of digital tools (
      • Ting D.S.W.
      • et al.
      Digital technology and COVID-19.
      ;
      • Budd J.
      • et al.
      Digital technologies in the public-health response to COVID-19.
      ). The increasing popularity of smartphones offers new opportunities to implement innovative, scalable tools to improve the reach and effectiveness of health and healthcare delivery (
      • Arigo D.
      • et al.
      The history and future of digital health in the field of behavioral medicine.
      ). Thus, primary task support is the most utilized across the apps to act as a supportive tool. Another important element of primary task support is rehearsal (
      • Abraham C.
      • Michie S.
      A taxonomy of behavior change techniques used in interventions.
      ). However, the literature suggests that the utilization of rehearsal appears to be reported as lower than reality as authors may see rehearsal as an obvious element of the intervention and fail to report on it (
      • Kelders S.M.
      • et al.
      Persuasive system design does matter: a systematic review of adherence to web-based interventions.
      ). Dialogue support was employed less across all the apps. The present finding reported that liking was the most employed design principle in dialogue support. This indicated that the apps overall were well laid out, but a lack of device-mediated interaction and engagement. Furthermore, over 55% of the included apps did not utilize reminders. Studies have shown the importance of reminders as effective tools in encouraging behavior change in health interventions (
      • Webb T.L.
      • et al.
      Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy.
      ). This may be due to the nature of apps, with many being data collection tools. Dialogue support elements such as praise and rewards are seldom used, which may be a weakness as recent literature surrounding the gamification of apps show praise and rewards are expected to have positive effects on the outcomes of health interventions (
      • Krath J.
      • Schürmann L.
      • von Korflesch H.F.O.
      Revealing the theoretical basis of gamification: a systematic review and analysis of theory in research on gamification, serious games and game-based learning.
      ). As for credibility, almost all the apps employed the features of trustworthiness, expertise, and surface credibility. This reinforces the nature of mobile apps in academic settings. Social support was the least utilized category of the PSD model. Social support plays an important role in behavior change, which has been explored in available literature (
      • van Dam H.A.
      • et al.
      Social support in diabetes: a systematic review of controlled intervention studies.
      ;
      • DiMatteo M.R.
      Social support and patient adherence to medical treatment: a meta-analysis.
      ). This may be due to the nature of the apps to provide support for interventions of self-management. As mobile health apps are to support consumer's self-management behavior and to assist patient engagement, providing relevant design features from the PSD will mimic or assist the communication of patient provider communication (
      • Win K.T.
      • Roberts M.R.H.
      • Oinas-Kukkonen H.
      Persuasive system features in computer-mediated lifestyle modification interventions for physical activity.
      ).
      While the primary task support and the credibility support were the most frequently noted features of mobile app for plwMS in this study, more features from dialogue support and social support could be included by applying the design principles of the PSD model. However, the functionalities do not by any means cover all of the design principle of the PSD model (
      • Oinas-Kukkonen H.
      • Harjumaa M.
      Persuasive systems design: key issues, process model, and system features.
      ). The use of design principles are determined by the goals and requirements of information systems (
      • Oinas-Kukkonen H.
      • Harjumaa M.
      Persuasive systems design: key issues, process model, and system features.
      ). For example, a recent qualitative study on app requirements suggests that an app for promoting the physical activity guidelines for plwMS should be user-friendly and track personal goals, as well as incorporate motivational, functional, and personalization strategies in its design (
      • Neal W.N.
      • Richardson E.V.
      • Motl R.W.
      Informing the development of a mobile application for the physical activity guidelines in multiple sclerosis: a qualitative, pluralistic approach.
      ). By applying the design principles related to rewards of dialogue support, an app can include features that encourage human-app interaction to help users keep moving towards their goals, leveraging gamification principles. One of examples is to create a challenge for tracking physical activity on a personal basis, where users are awarded a virtual trophy if they successfully complete the challenge. This feature provides a more engaging and enjoyable way for users to gain experience in tracking physical activity. In addition to individual challenges, the app can also offer an option to establish team challenges with a shared goal. This feature applies the design principle of cooperation in social support, with the aim to work together to achieve a shared goal. Furthermore, design principles of the PSD model are applied to the interface design of digital tools with the aim of guiding the planned persuasive effects of the tools (
      • Fogg B.J.
      • Fogg B.J.
      Persuasive Technology: Using Computers to Change What We Think and Do.
      ). The designers intend to promote specific attitude and behavior changes through the use of the tools (
      • Fogg B.J.
      • Fogg B.J.
      Persuasive Technology: Using Computers to Change What We Think and Do.
      ). Therefore, the nature, purpose, procedures, potential risks, and benefits of the study can be clearly communicated to participants, who are able to provide explicit and informed consent before any data is collected and analyzed.
      The finding of the present study suggests that most identified apps designed for use by plwMS applied a user-centered design principle. This is especially important as the previous evidence suggests digital health tools not meeting the needs of users (
      • Hone T.
      • Palladino R.
      • Filippidis F.T.
      Association of searching for health-related information online with self-rated health in the European Union.
      ). The goal of applying a user-centered design approach is to create solutions specific to the intended users and ensure the tool is easy to use, easy to understand and acceptable (
      • McCurdie T.
      • et al.
      mHealth consumer apps: the case for user-centered design.
      ). The target users and relevant stakeholders are involved in the different stages of product development. However, there is a gap in translating the specific needs for digital health tools in plwMS into technology features (
      • Bevens W.
      • et al.
      Attrition within digital health interventions for people with multiple sclerosis: systematic review and meta-analysis.
      ). The use of digital health tools requires a specific set of literacy (
      • Lupton D.
      M-health and health promotion: the digital cyborg and surveillance society.
      ). For example, the competencies and skills are needed to engage and navigate with mobile devices (
      • Lin T.T.C.
      • Bautista J.R.
      Understanding the relationships between mhealth apps’ characteristics, trialability, and mHealth literacy.
      ). There is a patient concern that digital interventions tend to be too complicated when it comes to health and healthcare delivery (
      • Illiger K.
      • et al.
      Mobile technologies: expectancy, usage, and acceptance of clinical staff and patients at a University Medical Center.
      ). Therefore, a supportive environment within apps may be required to guide users to gain positive experience.
      The present results suggest that the methods used to evaluate the mobile apps in academic settings are varied. Evaluation and design of a mobile app tend to be closely integrated (
      • Sharp H.
      • Rogers Y.
      • Preece J.
      Interaction Design: Beyond Human-Computer Interaction.
      ). For example, usability testing may be integrated in the varied stages of product development to establish how well an app functions and serves its intended purpose for a target population. The outcomes of evaluation tend to be integrated into the digital health tool via iterations. In the present study, approximately half of the identified apps focused on evaluating usability and one third of such studies used the System Usability Scale (
      • Brooke J.
      SUS-A quick and dirty usability scale.
      ) to evaluate usability. The System Usability Scale is used to assess user perceptions of the usability of a software system (
      • Brooke J.
      SUS-A quick and dirty usability scale.
      ). Although the literature suggests that the System Usability Scale is the most widely used standardized survey measure (
      • Lewis J.R.
      The system usability scale: past, present, and future.
      ), new tools developed to assess the quality of health mobile apps (e.g., Mobile App Rating Scale (
      • Stoyanov S.R.
      • et al.
      Mobile app rating scale: a new tool for assessing the quality of health mobile apps.
      )) were also used to assess user experience (
      • Geurts E.
      • et al.
      WalkWithMe: personalized goal setting and coaching for walking in people with multiple sclerosis.
      ). In addition, MS-related outcomes were evaluated to establish preliminary evidence on clinical efficacy of the identified app. This is likely due to the early stages of development and nature of the academic setting, which is for research purposes.
      However, there are several limitations for the present study. The most significant limitation of this scoping review was that the inclusion criteria necessitated that each included studies clearly described the details on the interface design of mobile apps. This decision was made to align with the scope of this review. Further, as there was no standard reporting system for digital technology tools, when assessing for the apps, we relied on the description and screenshots of the app to make the decision on digital functionality, applied behavior change theory/technique, and persuasive technology. Although an extensive scope search was conducted in scientific databases, some relevant literature in other domains might have been excluded. The results of this scoping review are only inclusive of studies written in English; therefore, findings may not be generalizable internationally. In addition, industry-developed apps were excluded in this analysis. These apps tend to undertake the path of designing and developing software as a medical device (
      • van der Walt A.
      • et al.
      Developing a digital solution for remote assessment in multiple sclerosis: from concept to software as a medical device.
      ). Software as a medical device is a type of software that is capable of performing medical functions without requiring any specific medical device hardware (

      IMDRF Software as a Medical Device (SaMD) Working Group. “Software as a medical device": possible framework for risk categorization and corresponding considerations 2014 [cited 2023 07 March]; Available from: https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-140918-samd-framework-risk-categorization-141013.pdf.

      ). Software as a medical device must meet the standards of device regulatory agencies on design control, cybersecurity and data privacy, risk analysis, and clinical evaluation (

      IMDRF Software as a Medical Device (SaMD) Working Group. “Software as a medical device": possible framework for risk categorization and corresponding considerations 2014 [cited 2023 07 March]; Available from: https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-140918-samd-framework-risk-categorization-141013.pdf.

      ). All these elements are considered in the certification of software as medical device (

      IMDRF Software as a Medical Device (SaMD) Working Group. “Software as a medical device": possible framework for risk categorization and corresponding considerations 2014 [cited 2023 07 March]; Available from: https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-140918-samd-framework-risk-categorization-141013.pdf.

      ). Therefore, developing apps for use as medical devices requires maintaining a balance between clinical, technical, and regulatory processes to achieve the required level of rigor. Moreover, industry-developed apps tend to be owned by a company or individual. The design tends to be kept confidential and not publicly disclosed for gaining a competitive advantage in a specific market or industry. However, development for software as medical device appears to commence after establishing proof of concept of digital solutions in a research context (
      • van der Walt A.
      • et al.
      Developing a digital solution for remote assessment in multiple sclerosis: from concept to software as a medical device.
      ). Proof of concept in app developing refers to the process of creating and testing a new idea to determine its feasibility (
      • Sharp H.
      • Rogers Y.
      • Preece J.
      Interaction Design: Beyond Human-Computer Interaction.
      ). The goals of proof of concept are to provide technical, functional, and/or user experience issues that may arise related to the new idea (
      • Sharp H.
      • Rogers Y.
      • Preece J.
      Interaction Design: Beyond Human-Computer Interaction.
      ). The outcome of proof of concept offers feedback for further improving the app in clinical effectiveness, technical capabilities and user experience (
      • van der Walt A.
      • et al.
      Developing a digital solution for remote assessment in multiple sclerosis: from concept to software as a medical device.
      ). By embracing Industry 4.0 or digital technologies, government agencies have initiated and invested substantially in innovation and research commercialization. A growing number of universities have developed programs and resources to facilitate the flow of research from universities to industry (
      • Harman G.
      Australian university research commercialisation: perceptions of technology transfer specialists and science and technology academics.
      ;
      • Battaglia D.
      • Paolucci E.
      • Ughetto E.
      The role of Proof-of-Concept programs in facilitating the commercialization of research-based inventions.
      ). The outcome of the studies on apps in academic settings provides evidence to determine whether the digital solution has commercial viability for attracting funding and investors (
      • Battaglia D.
      • Paolucci E.
      • Ughetto E.
      The role of Proof-of-Concept programs in facilitating the commercialization of research-based inventions.
      ). It not only provides ongoing support for maintenance of these apps but also offer opportunities to further refine and improve the app for clinical effectiveness. Thus, conducting a scoping review on native mobile apps in academic setting may determine the scope on key characteristics of apps for plwMS.

      5. Conclusion

      This scoping review identified and explored the current mobile apps available for plwMS. The present findings suggest that the content and digital technologies of these apps were chosen differently based on the purposes of the apps. By applying design principles of the PSD model, the interaction design qualities and features of the identified mobile apps in academic settings were systematically evaluated at a deeper level. Despite the limitations, the majority of included studies were in the early stages of development and had a user-centered design, highlighting an upcoming realm of digital health technology. Further studies are needed in this field to expand on mobile digital tools available for plwMS.

      Research involving human and animal participants

      The present study was a review. Ethics approval is not required for reviews.

      Role of funding source

      The funder had no role in the design of the study; in the development, analyses or interpretation of data; in the writing of the manuscript or in the decision to publish the results.

      Funding

      VG was supported by Multiple Sclerosis Australia Postdoctoral Fellowship (Grant number 20-223).

      CRediT authorship contribution statement

      Zahli Howard: Formal analysis, Investigation, Data curation, Writing – original draft. Khin Than Win: Methodology, Writing – review & editing. Vivienne Guan: Conceptualization, Methodology, Formal analysis, Investigation, Data curation, Writing – original draft, Writing – review & editing, Supervision.

      Declaration of Competing Interest

      The authors declare no conflict of interest.

      References

        • Abraham C.
        • Michie S.
        A taxonomy of behavior change techniques used in interventions.
        Health Psychol. 2008; 27: 379-387
      1. Apple Developer. App store review guidelines. 2023 [cited 2023 07 March]; Available from: https://developer.apple.com/app-store/review/guidelines/.

        • Arigo D.
        • et al.
        The history and future of digital health in the field of behavioral medicine.
        J. Behav. Med. 2019; 42: 67-83
        • Arksey H.
        • O'Malley L.
        Scoping studies: towards a methodological framework.
        Int. J. Soc. Res. Methodol. 2005; 8: 19-32
        • Babbage D.R.
        • et al.
        MS energize: field trial of an app for self-management of fatigue for people with multiple sclerosis.
        Internet Interv. 2019; 18100291
        • Battaglia D.
        • Paolucci E.
        • Ughetto E.
        The role of Proof-of-Concept programs in facilitating the commercialization of research-based inventions.
        Res. Policy. 2021; 50104268
        • Bevens W.
        • et al.
        Attrition within digital health interventions for people with multiple sclerosis: systematic review and meta-analysis.
        J. Med. Internet Res. 2022; 24: e27735
        • Boukhvalova A.K.
        • Fan O.
        • Weideman A.M.
        • Harris T.
        • Kowalczyk E.
        • Pham L.
        • Kosa P.
        • Bielekova B.
        Smartphone level test measures disability in several neurological domains for patients with multiple sclerosis.
        Front. Neurol. 2019; 10: 358
        • Brooke J.
        SUS-A quick and dirty usability scale.
        Usability Eval. Ind. 1996; 189: 4-7
        • Budd J.
        • et al.
        Digital technologies in the public-health response to COVID-19.
        Nat. Med. 2020; 26: 1183-1192
        • De Angelis M.
        • et al.
        Digital technology in clinical trials for multiple sclerosis: systematic review.
        J. Clin. Med. 2021; 10: 2328
        • Demiris G.
        • et al.
        Patient-centered applications: use of information technology to promote disease management and wellness. A white paper by the AMIA knowledge in motion working group.
        J. Am. Med. Inform. Assoc. 2008; 15: 8-13
        • Deterding S.
        • et al.
        From game design elements to gamefulness: defining "gamification".
        in: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, Tampere, Finland Association for Computing Machinery, 2011: 9-15
        • DiMatteo M.R.
        Social support and patient adherence to medical treatment: a meta-analysis.
        Health Psychol. 2004; 23: 207-218
        • Fogg B.J.
        • Fogg B.J.
        Persuasive Technology: Using Computers to Change What We Think and Do.
        Elsevier Science & Technology, San Francisco, United States2003
        • Geurts E.
        • et al.
        WalkWithMe: personalized goal setting and coaching for walking in people with multiple sclerosis.
        in: Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization. Association for Computing Machinery, 2019: 51-60
        • Harman G.
        Australian university research commercialisation: perceptions of technology transfer specialists and science and technology academics.
        J. Higher Educ. Policy Manag. 2010; 32: 69-83
        • Hawker S.
        • et al.
        Appraising the evidence: reviewing disparate data systematically.
        Qual. Health Res. 2002; 12: 1284-1299
        • Hone T.
        • Palladino R.
        • Filippidis F.T.
        Association of searching for health-related information online with self-rated health in the European Union.
        Eur. J. Public Health. 2016; 26: 748-753
        • Hsieh K.
        • et al.
        Usability of a fall risk mHealth app for people with multiple sclerosis: mixed methods study.
        JMIR Hum. Factors. 2021; 8: e25604
        • Illiger K.
        • et al.
        Mobile technologies: expectancy, usage, and acceptance of clinical staff and patients at a University Medical Center.
        JMIR mHealth uHealth. 2014; 2: e42
      2. IMDRF Software as a Medical Device (SaMD) Working Group. “Software as a medical device": possible framework for risk categorization and corresponding considerations 2014 [cited 2023 07 March]; Available from: https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-140918-samd-framework-risk-categorization-141013.pdf.

        • Kelders S.M.
        • et al.
        Persuasive system design does matter: a systematic review of adherence to web-based interventions.
        J. Med. Internet Res. 2012; 14: e152
        • Khazen W.
        • et al.
        Rethinking the use of mobile apps for dietary assessment in medical research.
        J. Med. Internet Res. 2020; 22: e15619
        • Krath J.
        • Schürmann L.
        • von Korflesch H.F.O.
        Revealing the theoretical basis of gamification: a systematic review and analysis of theory in research on gamification, serious games and game-based learning.
        Comput. Hum. Behav. 2021; 125106963
        • Lang M.
        • et al.
        PatientConcept app: key characteristics, implementation, and its potential benefit.
        Neurol. Ther. 2019; 8: 147-154
        • Lavorgna L.
        • et al.
        e-Health and multiple sclerosis: An update.
        Mult. Scler. J. 2018; 24: 1657-1664
        • Lentferink A.J.
        • et al.
        Key components in eHealth interventions combining self-tracking and persuasive ecoaching to promote a healthier lifestyle: a scoping review.
        J. Med. Internet Res. 2017; 19: e277
        • Lerdal A.
        • et al.
        A prospective study of patterns of fatigue in multiple sclerosis.
        Eur. J. Neurol. 2007; 14: 1338-1343
        • Lewis J.R.
        The system usability scale: past, present, and future.
        Int. J. Hum. Comput. Interact. 2018; 34: 577-590
        • Lin T.T.C.
        • Bautista J.R.
        Understanding the relationships between mhealth apps’ characteristics, trialability, and mHealth literacy.
        J. Health Commun. 2017; 22: 346-354
        • Lisak D.
        Overview of symptomatic management of multiple sclerosis.
        J. Neurosci. Nurs. 2001; 33: 224-230
        • Longley W.
        • Honan C.
        Cognitive impairment in multiple sclerosis: the role of the general practitioner in cognitive screening and care coordination.
        Aust. J. Gen. Pract. 2022; 51: 225-231
        • Lupton D.
        M-health and health promotion: the digital cyborg and surveillance society.
        Soc. Theory Health. 2012; 10: 229-244
        • Malavolta I.
        Beyond native apps: web technologies to the rescue! (keynote).
        in: Proceedings of the 1st International Workshop on Mobile Development, Amsterdam, Netherlands Association for Computing Machinery, 2016: 1-2
        • Marrie R.A.
        • et al.
        Use of eHealth and mHealth technology by persons with multiple sclerosis.
        Mult. Scler. Relat. Disord. 2019; 27: 13-19
        • Marziniak M.
        • et al.
        The use of digital and remote communication technologies as a tool for multiple sclerosis management: narrative review.
        JMIR Rehabil. Assist. Technol. 2018; 5: e5
        • McCurdie T.
        • et al.
        mHealth consumer apps: the case for user-centered design.
        Biomed. Instrum. Technol. 2012; 46: 49-56
        • Midaglia L.
        • et al.
        Adherence and satisfaction of smartphone- and smartwatch-based remote active testing and passive monitoring in people with multiple sclerosis: nonrandomized interventional feasibility study.
        J. Med. Internet Res. 2019; 21: e14863
      3. Mobile for Development. The mobile economy 2022. 2022 [cited 2022 20 July]; Available from: https://www.gsma.com/r/somic/.

        • Mokhberdezfuli M.
        • Ayatollahi H.
        • Naser Moghadasi A.
        A smartphone-based application for self-management in multiple sclerosis.
        J. Healthc. Eng. 2021; 20216749951
        • Motl R.W.
        • et al.
        Randomized controlled trial of an e-learning designed behavioral intervention for increasing physical activity behavior in multiple sclerosis.
        Mult. Scler. J. Exp. Transl. Clin. 2017; 32055217317734886
        • Munn Z.
        • et al.
        Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach.
        BMC Med. Res. Methodol. 2018; 18: 143
        • Neal W.N.
        • Richardson E.V.
        • Motl R.W.
        Informing the development of a mobile application for the physical activity guidelines in multiple sclerosis: a qualitative, pluralistic approach.
        Disabil. Rehabil. Assist. Technol. 2022; : 1-9https://doi.org/10.1080/17483107.2022.2153937
        • Oinas-Kukkonen H.
        • Harjumaa M.
        Persuasive systems design: key issues, process model, and system features.
        Commun. Assoc. Inf. Syst. 2009; 24: 28
        • Palotai M.
        • et al.
        Usability of a mobile app for real-time assessment of fatigue and related symptoms in patients with multiple sclerosis: observational study.
        JMIR Mhealth Uhealth. 2021; 9: e19564
        • Pratap A.
        • et al.
        Evaluating the utility of smartphone-based sensor assessments in persons with multiple sclerosis in the real-world using an App (elevateMS): observational, prospective pilot digital health study.
        JMIR Mhealth Uhealth. 2020; 8: e22108
        • Reich D.S.
        • Lucchinetti C.F.
        • Calabresi P.A.
        Multiple sclerosis.
        N. Engl. J. Med. 2018; 378: 169-180
        • Rosen L.
        • Suhami R.
        The art and science of study identification: a comparative analysis of two systematic reviews.
        BMC Med. Res. Methodol. 2016; 16: 24
        • Rudick R.A.
        • et al.
        The Multiple Sclerosis Performance Test (MSPT): an iPad-based disability assessment tool.
        J. Vis. Exp. 2014; 88: e51318
        • Sharp H.
        • Rogers Y.
        • Preece J.
        Interaction Design: Beyond Human-Computer Interaction.
        5th ed. Wiley, Indianapolis2019
        • Stoyanov S.R.
        • et al.
        Mobile app rating scale: a new tool for assessing the quality of health mobile apps.
        JMIR mHealth uHealth. 2015; 3: e27
        • Tacchino A.
        • et al.
        A new app for at-home cognitive training: description and pilot testing on patients with multiple sclerosis.
        JMIR mHealth uHealth. 2015; 3: e85
        • Tacchino A.
        • et al.
        Design, development, and testing of an app for dual-task assessment and training regarding cognitive-motor interference (CMI-APP) in people with multiple sclerosis: multicenter pilot study.
        JMIR Mhealth Uhealth. 2020; 8: e15344
        • Thirumalai M.
        • et al.
        TEAMS (Tele-Exercise and Multiple Sclerosis), a tailored telerehabilitation mhealth app: participant-centered development and usability study.
        JMIR Mhealth Uhealth. 2018; 6: e10181
        • Thomas S.
        • et al.
        Creating a digital toolkit to reduce fatigue and promote quality of life in multiple sclerosis: participatory design and usability study.
        JMIR Form. Res. 2021; 5: e19230
        • Thompson A.J.
        • et al.
        Multiple sclerosis.
        Lancet N. Am. Ed. 2018; 391: 1622-1636
        • Ting D.S.W.
        • et al.
        Digital technology and COVID-19.
        Nat. Med. 2020; 26: 459-461
        • van Dam H.A.
        • et al.
        Social support in diabetes: a systematic review of controlled intervention studies.
        Patient Educ. Couns. 2005; 59: 1-12
        • van der Walt A.
        • et al.
        Developing a digital solution for remote assessment in multiple sclerosis: from concept to software as a medical device.
        Brain Sci. 2021; 11: 1247
        • Van Hecke W.
        • et al.
        A novel digital care management platform to monitor clinical and subclinical disease activity in multiple sclerosis.
        Brain Sci. 2021; 11
        • Walton C.
        • et al.
        Rising prevalence of multiple sclerosis worldwide: Insights from the Atlas of MS, third edition.
        Mult. Scler. J. 2020; 26: 1816-1821
        • Webb T.L.
        • et al.
        Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy.
        J. Med. Internet Res. 2010; 12: e4
        • Win K.T.
        • Roberts M.R.H.
        • Oinas-Kukkonen H.
        Persuasive system features in computer-mediated lifestyle modification interventions for physical activity.
        Inform. Health Soc. Care. 2019; 44: 376-404