Advertisement

An agent-based simulation model for informed shared decision making in multiple sclerosis

  • Author Footnotes
    1 Present address: ARN—Anestesia, Reanimação e Neurologia, Lda, Campo Grande 14-6A, 1700-092 Lisboa, Portugal.
    Mário Veloso
    Correspondence
    Tel.: +351 964020950 (Mob); fax: +351 217993913.
    Footnotes
    1 Present address: ARN—Anestesia, Reanimação e Neurologia, Lda, Campo Grande 14-6A, 1700-092 Lisboa, Portugal.
    Affiliations
    Hospital Egas Moniz (Centro Hospitalar de Lisboa Ocidental, EPE), Rua da Junqueira, 126, 1349-019 Lisboa, Portugal
    Search for articles by this author
  • Author Footnotes
    1 Present address: ARN—Anestesia, Reanimação e Neurologia, Lda, Campo Grande 14-6A, 1700-092 Lisboa, Portugal.
Open AccessPublished:May 07, 2013DOI:https://doi.org/10.1016/j.msard.2013.04.001

      Highlights

      • A major challenge in MS is to make a prediction of the disease evolution.
      • Studies identify predictive factors but do not focus individual prognosis.
      • New ways for presenting probabilistic information to decision-making are required.
      • Proposed model provide an easy to use simulation tool concerning individual long term disability prediction.

      Abstract

      Shared decision making (SDM) is concerned with patient involvement into medical decisions and chronic conditions such as Multiple sclerosis (MS), with only partially effective treatments leading to potential severe side effects, conflicting evidence, and uncertain evidence on outcomes, constitute a typical condition for SDM. As treatment options increase and patients participate more intensively in decisions, the need for evidence-based information (EBI) becomes clear.
      Natural history (NH) studies of MS represent the basic sources for required EBI and are especially useful to contribute to the practical exercise of prognosis formulation and to enable the evaluation of effectiveness in the context of treatment. Several of these identify early clinical factors predictive of the course of MS but there is no consensus method for determining the long term progression of disability and evolution of individual patients on the basis of observations on the early stages of the disease, which constitutes a major challenge for the practicing neurologist.
      Aiming at delivering more reliable prognosis estimation, this study combines the distribution of patients reaching specific levels of disability within defined time periods as determined in NH studies, with disability curves and severity scores as a function of time, in terms of percentiles and deciles respectively, derived from longitudinal data analysis studies. A computer agent-based simulation model was implemented as a comprehensive and easy to utilize tool able to predict and monitor progression of disability in MS patients, and to support the neurologist discussing prognosis scenarios with the individual patient for effective SDM.

      Keywords

      1. Introduction

      The goal of both diagnosis and therapeutic decisions is to improve the prognosis for the patient, since prognosis refers to all medical outcomes that may occur during the patient's disease process. This also implies that patient management should be driven by two major prognosis related topics: the natural history (NH), which is the prognosis of the disease without medical interventions, and the prognosis changes as resulting from medical interventions. Additionally, prognosis is a major concern to the patient who wants to be informed about his prospects and prognosis related information is a mandatory requirement for an informed and active participation of the patient on self-clinical decision-making.
      A major challenge in multiple sclerosis (MS) for the practicing neurologist is to make a prediction of the long term evolution of individual patients on the basis of observations on the early stages of the disease. An immediate effect of such difficulty is to inhibit the communication to the patient of a realistic estimation of his/her evolution, particularly in a long term basis.
      Prognosis' estimates or prediction can be made in several ways. As opposed to an informal way (e.g. intuitively or using expert opinions), modern patient management requires that appropriateness of medical interventions is supported by scientific evidence, integrating clinical expertise, patient values, and the best research evidence into the decision making process for patient care, the basis for evidence-based medicine (
      • Sackett D.L.
      • Rosenberg W.M.C.
      • Gray J.A.M.
      • Haynes R.B.
      • Richardson W.S.
      Evidence-based medicine: what it is and what it isn′t.
      ). Including patient values and preferences in the clinical decision-making is an ethical issue (
      • Hughes T.E.
      • Larson L.N.
      Patient involvement in health care. A procedural justice viewpoint.
      ) and can contribute to improve patient care (
      • O'Connor A.M.
      • Wennberg J.E.
      • Legare F.
      • Llewellyn-Thomas H.A.
      • Moulton B.W.
      • Sepucha K.R.
      • et al.
      Toward the ‘tipping point’: decision aids and informed patient choice.
      ,
      • Sepucha K.R.
      • Fowler Jr., F.J.
      • Mulley Jr., A.G.
      Policy support for patient-centered care: the need for measurable improvements in decision quality.
      ). Shared decision-making (SDM) is a major component of such patient-centered care, and has been defined as a process that “allows both physicians and patients to honor the values and preferences of the patient, while also permitting the physician to provide medical expertise to promote the patient′s health” (
      • O'Connor A.M.
      • Wennberg J.E.
      • Legare F.
      • Llewellyn-Thomas H.A.
      • Moulton B.W.
      • Sepucha K.R.
      • et al.
      Toward the ‘tipping point’: decision aids and informed patient choice.
      ). Hence, SDM takes informed consent a step further in the process of communication between a patient and a physician. It is much more than obtaining the patient′s authorization or agreement to undergo a specific medical intervention; SDM means to involve the patient actively on the medical decisions and such patient empowerment also means giving the patient more responsibility.
      Chronic conditions such as MS, with only partially effective treatments with potential severe side effects, conflicting evidence, and uncertain evidence on outcomes, where the benefit–harm ratio is short and/or doubtful, or when available options have different benefit–harm profiles that patients value differently, constitute a typical condition for this SDM and enforces its need (
      • Heesen C.
      • Solari A.
      • Giordano A.
      • Kasper J.
      • Köpke S.
      Decisions on multiple sclerosis immunotherapy: new treatment complexities urge patient engagement.
      ,
      • Wennberg J.E.
      • Fisher E.S.
      • Skinner J.S.
      Geography and the debate over Medicare reform.
      ).
      As treatment options increase and patients participate more intensively in decisions, the need for evidence-based information becomes clear. There is an evidence that patients may, in exchange for therapeutic benefits, be willing to accept greater levels of risk than are actually posed by some therapies (

      Calfee, J.E. A representative survey of MS patients on attitudes toward the benefits and risks of drug therapy. AEI-Brookings Joint Center for Regulatory Studies; 2006.

      ,
      • Johnson F.R.
      • Özdemir S.
      • Hauber A.B.
      • Kauf T.L.
      Women's stated willingness to accept perceived risk for vasomotor symptom relief.
      ,
      • Johnson F.R.
      • Özdemir S.
      • Mansfield C.A.
      • Hass S.
      • Miller D.W.
      • Siegel C.A.
      • et al.
      Crohn's disease patients' benefit-risk preferences: serious adverse event risks versus treatment efficacy.
      ). Drug therapies for MS offer a range of potential benefits, but they may also involve life-threatening risks, including liver failure, leukemia, and progressive multifocal leukoencephalopathy (
      • Brassat D.
      • Recher C.
      • Waubant E.
      • Le Page E.
      • Rigal-Huguet F.
      • Laurent G.
      • et al.
      Therapy-related acute myeloblastic leukemia after mitoxantrone treatment in a patient with MS.
      ,
      • Francis G.
      • Grumser Y.
      • Alteri E.
      • Micaleff A.
      • O'Brien F.
      • Alsop J.
      • et al.
      Hepatic reactions during treatment of multiple sclerosis with Interferon β-1a, incidence and clinical significance.
      ,
      • Yousry T.A.
      • Habil D.M.
      • Major E.O.
      • Ryschkewitsch C.
      • Fahle G.
      • Fischer S.
      • et al.
      Evaluation of patients treated with natalizumab for progressive multifocal leukoencephalopathy.
      ). Yet patients may be misinformed particularly by means of internet information, misinterpret the results of scientific research (
      • Jadad A.R.
      • Haynes R.B.
      • Hunt D.
      • Browman G.P.
      The internet and evidence-based decision-making: a needed synergy for efficient knowledge management in health care.
      ,
      • Kaplan B.
      • Brennan P.F.
      Consumer informatics supporting patients as co-producers of quality.
      ), have unrealistic expectations of treatment benefits and harms, and clinicians may be poor judges of a patient′s values (
      • O'Connor A.M.
      • Wennberg J.E.
      • Legare F.
      • Llewellyn-Thomas H.A.
      • Moulton B.W.
      • Sepucha K.R.
      • et al.
      Toward the ‘tipping point’: decision aids and informed patient choice.
      ).
      There also is evidence that patient decision aids are better than usual care in improving patients' knowledge and expectations about interventions, as well as improving agreement between values and choice (
      • Elwyn G.
      • O'Connor A.
      • Stacey D.
      • Volk R.
      • Edwards A.
      • Coulter A.
      • et al.
      International Patient Decision Aids Standards (IPDAS) Collaboration. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process.
      ).
      A diversity of methods has emerged for decision support from different scientific fields such as Statistics, Decision Analysis and Artificial Intelligence. Cox models, recursive partitioning analysis, Weibull models, decision trees, Markov models, Partially Observable Markov Decision Process, Bayesian networks and influence diagrams. No matter the approach, the first step is to build up a prognostic model able to predict the probability of some outcome as optimally as possible.
      Several studies have been performed to identify early clinical factors predictive of the MS course (
      • Levic Z.M.
      • Dujmovic I.
      • Pekmezovic T.
      • Jareninski M.
      • Marinkovic J.
      • Stojsavljevic N.
      • et al.
      Prognostic factors for survival in multiple sclerosis.
      ,
      • Runmarker B.
      • Andersen O.
      Prognostic factors in a multiple sclerosis incidence cohort with twenty five years of follow-up.
      ,
      • Weinshenker B.G.
      • Bass B.
      • Rice G.P.A.
      • Noseworthy J.H.
      • Carriere W.
      • Basekerville J.
      • et al.
      The natural history of multiple sclerosis: a geographically based study: 2. Predictive value of the early clinical course.
      ,
      • Weinshenker B.G.
      • Rice G.P.A.
      • Noseworthy J.H.
      • Carriere W.
      • Basekerville J.
      • Ebers G.C.
      The natural history of multiple sclerosis: a geographically based study: 4. Applications to planning and interpretation of clinical therapeutic trials.
      ,
      • Kremenchutzky M.
      • Rice G.P.A.
      • Baskerville J.
      • Wingerchuk M.
      • Ebers G.C.
      The natural history of multiple sclerosis: a geographically based study. 9: observations on the progressive phase of rhe disease.
      ,
      • Confavreux C.
      • Vukusic S.
      Natural history of multiple sclerosis: a unifying concept.
      ,
      • Ebers G.C.
      Prognostic factors for multiple sclerosis: the importance of natural history studies.
      ,
      • Vukusic S.
      • Confavreux C.
      Natural history of multiple sclerosis: risk factors and prognostic indicators.
      ,
      • Scalfari A.
      • Neuhaus A.
      • Muraro P.A.
      • Daumer M.
      • DeLuca G.C.
      • Muraro P.A.
      • et al.
      Early relapses, onset of progression, and late outcome in multiple sclerosis.
      ), such as: gender, disease course, age at onset of disease, initial symptoms, number of functional systems involved, first interval attack, attack frequency, and incomplete remission after the first episode. However, the majority of work in this area is not focused on the individual prognosis to the patient; neither does address treatment effects on the NH of the disease.
      • Daumer M.
      • Neuhaus A.
      • Lederer C.
      • Scholz M.
      • Wolinsky J.S.
      • Heiderhoff M.
      Prognosis of the individual course of disease-steps in developing a decision support tool for multiple sclerosis.
      , describe an online analytical processing tool that matches the characteristics of a given patient with the most similar patients of the Sylvia Lawry Centre for Multiple Sclerosis Research database. An “individual risk profile” in terms of the disease course of all similar patients in the database is displayed, hence enabling to project a hypothesized outcome for that patient (
      • Daumer M.
      • Neuhaus A.
      • Lederer C.
      • Scholz M.
      • Wolinsky J.S.
      • Heiderhoff M.
      Prognosis of the individual course of disease-steps in developing a decision support tool for multiple sclerosis.
      ). The main limitations of this tool are related to the characteristics of the patients included in the database. The clinical data are derived only from the placebo groups of randomized clinical trials, and the respective observation period is limited to a maximum of three years (
      • Daumer M.
      • Neuhaus A.
      • Lederer C.
      • Scholz M.
      • Wolinsky J.S.
      • Heiderhoff M.
      Prognosis of the individual course of disease-steps in developing a decision support tool for multiple sclerosis.
      ).
      In the study of
      • Wolfson C.
      • Confavreux C.A.
      Markov model of the natural history of multiple sclerosis.
      ,
      • Wolfson C.
      • Confavreux C.
      A improvements to a simple Markov model of the natural history of multiple sclerosis: I. Short term prognosis.
      , a Markov model is proposed to represent the disease course by means of transitions between disease states, as to evaluate the effect of prognostic factors on those transitions. Because Markov processes are memory less, once a state is known, the future evolution of the disease is independent of the past evolution. This limitation is handled in the work of
      • Bergamaschi R.
      • Quaglini S.
      • Trojano M.
      • Amato M.P.
      • Tavazzi E.
      • Paolicelli D.
      • et al.
      Early prediction of the long term evolution of multiple sclerosis: the Bayesian Risk Estimate for Multiple Sclerosis (BREMS) score.
      ,
      • Bergamaschi R.
      • Berzuini C.
      • Romani A.
      • Cosi V.
      Predicting secondary progression in relapsing-remitting multiple sclerosis: a Bayesian analysis.
      , by proposing a Bayesian model specifying the full joint probability distribution for a set of random variables that characterize the entire course of the disease. The risk of reaching secondary progression was significantly related to specific clinical factors presented during the first year of the disease, all of them associated with a specific statistical weight, the Bayesian local relative risk, used to calculate the Bayesian Risk Estimate for MS (BREMS) score for any given patient (
      • Bergamaschi R.
      • Quaglini S.
      • Trojano M.
      • Amato M.P.
      • Tavazzi E.
      • Paolicelli D.
      • et al.
      Early prediction of the long term evolution of multiple sclerosis: the Bayesian Risk Estimate for Multiple Sclerosis (BREMS) score.
      ,
      • Bergamaschi R.
      • Berzuini C.
      • Romani A.
      • Cosi V.
      Predicting secondary progression in relapsing-remitting multiple sclerosis: a Bayesian analysis.
      ). However, no other prognostic outcomes are provided.
      • Achiron A.
      • Barak Y.
      • Rotstein Z.
      Longitudinal disability curves for predicting the course of relapsing-remitting multiple sclerosis.
      use NH information from a large database to generate longitudinal disability curves for prediction of disease progression based on the mean Expanded Disability Status Scale (EDSS) scores from the first year of disease onset represented as a major percentile group. These curves represent cohort percentiles and enable to foresee the relative risk of disease progression, as well as to identify deviations in the curves (
      • Achiron A.
      • Barak Y.
      • Rotstein Z.
      Longitudinal disability curves for predicting the course of relapsing-remitting multiple sclerosis.
      ). Similarly, the global Multiple Sclerosis Severity Score (MSSS) (
      • Roxburgh R.H.
      • Seaman S.R.
      • Masterman T.
      • Hensiek A.E.
      • Sawcer S.J.
      • Vukusic S.
      • et al.
      Multiple Sclerosis Severity Score Using disability and disease duration to rate disease severity.
      ) has been proposed as a population disability assessment tool enabling comparisons of relative disease severity at all EDSS levels for a given disease duration. A more recent study (
      • Gray O.M.
      • Jolley D.
      • Zwanikken C.
      • Trojano M.
      • Grand'Maison F.
      • Duquette P.
      • et al.
      The Multiple Sclerosis Severity Score (MSSS) re-examined: EDSS rank stability in the MSBase dataset increases 5 years after onset of multiple sclerosis.
      ) confirmed the validity of the MSSS as a 5-year severity rank predictor in individual patients.
      In spite of the increasing number of prognostic models developed in Medicine, the fact is that such prognostic models are seldom used in the clinical practice (
      • Wyatt J.C.
      • Altman D.G.
      Commentary prognostic models: clinically useful or quickly forgotten.
      ,
      • Dong Y.
      • Chbat N.W.
      • Gupta A.
      • Hadzikadic M.
      • Gajic O.
      Systems modeling and simulation applications for critical care medicine.
      ). A major challenge, often underestimated, consists of the ability to provide the relevant information for decision-making in a comprehensive and easy to process format both for patients and physicians. Different patients have different information needs, and different ways of decision-making (
      • Mazur D.J.
      • Hickam D.H.
      Patients' interpretations of probability terms.
      ). Moreover, many people, including physicians, have difficulty in process critically quantitative information (
      • Woloshin S.
      • Schwartz L.M.
      • Moncur M.
      • Gabriel S.
      • Tosteson A.N.
      Assessing values for health: numeracy matters.
      ). Considering that most information relevant to decision-making is inherently probabilistic, new ways for presenting such information are required (
      • Edwards A.
      • Elwyn G.
      • Mulley A.
      Explaining risks: turning numerical data into meaningful pictures.
      ,
      • Heesen C.
      • Solari A.
      • Giordano A.
      • Kasper J.
      • Köpke S.
      Decisions on multiple sclerosis immunotherapy: new treatment complexities urge patient engagement.
      ).
      Building and using models is part of everybody′s life, being simplification and abstraction of the real system the key in modeling. Agent-based modeling (ABM) is a type of modeling in which the focus is on representing agents (such as people) and their interactions (
      • Miller J.H.
      • Page S.E.
      Complex adaptive systems: an introduction to computational models of social life (Princeton studies in complexity).
      ), enabling to effectively capture a very rich set of complex behaviors and interactions, hence highly suited to modeling complex phenomena. ABM capabilities explain its extensive use in a diversity of fields, including decision-support and Medicine (
      • Cook D.A.
      • Hatala R.
      • Brydges R.
      • Zendejas B.
      • Szostek J.H.
      • Wang A.T.
      • et al.
      Technology-enhanced simulation for health professions education: A systematic review and meta-analysis.
      ,
      • Small S.D.
      Simulation applications for human factors and systems evaluation.
      ).
      The main goal of the present work is to provide the clinician with an easy to use simulation tool concerning individual long term (30 years) disability prediction and treatment effect on the estimated individual prognosis, in order to enable a comprehensive interaction with the patient with MS for effective SDM.

      2. Methods

      2.1 The simulation model

      The simulation model was implemented using the Netlogo 5.0.2 programmable modeling environment (

      Wilensky U, NetLogo. Evanston IL: Center for connected learning and computer based modeling, Northwestern University; 1999. Avaibable from:.〈http://ccl.northwestern.edu/netlogo/〉.

      ). Netlogo is a programming language with predominantly agent-oriented attributes, with unique capabilities that make it extremely powerful for producing and visualizing simulations of multi-agents systems. Taking into account that available disease-modifying agents are only indicated for the relapsing-remitting form of MS, the proposed simulation model only addresses this form of disease.
      For simulation purposes the model uses virtual population of 100 agents (simulating persons with relapsing-remitting MS) and all calculations are reflected in this population by means of different sorts of graphics (Fig. 1). Basically, disease progression is derived from the number of years from disease onset to EDSS scores 4, 6 and 7, according to published data of the Lyon database (
      • Confavreux C.
      • Compston A.
      The natural history of multiple sclerosis.
      ,
      • Confavreux C.
      • Vukusic S.
      • Adeleine P.
      Early clinical predictors and progression of irreversible disability in multiple sclerosis: an amnesic process.
      ,
      • Confavreux C.
      • Vukusic S.
      • Moreau T.
      • Adeleine P
      Relapses and progression of disability in multiple sclerosis.
      ,
      • Confavreux C.
      • Vukusic S.
      Natural history of multiple sclerosis: a unifying concept.
      ). The 95% confidence interval of the time to assignment of the different EDSS scores was used as reference for building the state transition rules that guarantee a proper probability distribution of the 100 agents (virtual MS patients) along the time. Transitions intervals are anticipated or delayed in accordance to the individual patient characteristics as entered by the user. These characteristics are the clinical variables used to build up the BREMS score (
      • Bergamaschi R.
      • Quaglini S.
      • Trojano M.
      • Amato M.P.
      • Tavazzi E.
      • Paolicelli D.
      • et al.
      Early prediction of the long term evolution of multiple sclerosis: the Bayesian Risk Estimate for Multiple Sclerosis (BREMS) score.
      ,
      • Bergamaschi R.
      • Berzuini C.
      • Romani A.
      • Cosi V.
      Predicting secondary progression in relapsing-remitting multiple sclerosis: a Bayesian analysis.
      ), corresponding to known prognostic factors: age at onset of disease, gender, sphincter onset, pure motor onset, motor-sensory onset, sequel after onset, number of involved functional systems at onset, number of sphincter plus motor relapses, EDSS≥4.
      Figure thumbnail gr1
      Fig. 1Simulation model user interface.
      Dependent on selected patient characteristics, the system will display a 30 years prediction of the probabilities to be at EDSS scores <4, 4, 6 and 7 in two graphics of a group of 100 patients with similar clinical characteristics. One graphic, will display 100 “persons” with different color intensity (pink for females, blue for males) representing the different EDSS possibilities. The second graphic, will present the yearly distribution of the different EDSS states along the 30 years projection. The chance of being in a progressive phase of the disease at 10 years after onset, and the value of patient quality adjusted life years (QALYs) after 30 years will be also displayed numerically in two distinct boxes. Patient QALYs are calculated as a function of the number of years at the different EDSS states estimated for the individual patient.
      By providing the system with the EDSS in subsequent years, the prognosis estimation in terms of disability progression will be refined. The patient EDSS score will be graphically situated in relation with the 10 years disability progression curves representing the respective annually percentile figures (
      • Achiron A.
      • Barak Y.
      • Rotstein Z.
      Longitudinal disability curves for predicting the course of relapsing-remitting multiple sclerosis.
      ). On the other hand, the yearly introduced EDSSs will enable to calculate the MSSS for those years, hence to make a 30 years projection corresponding to the maximum EDSS score expected to the derived decile (
      • Roxburgh R.H.
      • Seaman S.R.
      • Masterman T.
      • Hensiek A.E.
      • Sawcer S.J.
      • Vukusic S.
      • et al.
      Multiple Sclerosis Severity Score Using disability and disease duration to rate disease severity.
      ). Besides the predictive facilities, these graphics will enable the user to monitor disability progression along the years, hence to early recognize any up-deviation which certainly correspond to a clinical deterioration.
      The simulation model enables three ways for treatment assessment. Firstly, pressing the “assess” button after selecting a treatment from the list, a new 30 years prediction is made by the system reflecting the treatment effect in terms of “Absolute Risk Reduction” on two variables in accordance to published data in the literature: relapse free and progression free at 2 years treatment (
      • Filippini G.
      • Munari L.
      • Incorvaia B.
      • Ebers G.C.
      • Polman C.
      • D′Amico R.
      • et al.
      Interferons in relapsing remitting multiple sclerosis: a systematic review.
      ,
      • Freedman M.S.
      • Hughes B.
      • Mikol D.D.
      • Bennett R.
      • Cuffel B.
      • Divan V.
      • et al.
      Efficacy of diasease-modifying therapies in relapsing remitting multiple sclerosis: a systematic comparison.
      ,
      • Jacobs L.D.
      • Cookfair D.L.
      • Rudick R.A.
      • Herndon R.M.
      • Richert J.R.
      • Salazar A.M.
      • et al.
      The Multiple Sclerosis Collaborative Research Group (MSCRG). Intramuscular interferon beta-1a for disease progression in relapsing multiple sclerosis.
      ,
      • Johnson K.P.
      • Brooks B.R.
      • Cohen J.A.
      • Ford C.C.
      • Goldstein J.
      • Lisak R.P.
      • et al.
      The Copolymer 1 Multiple Sclerosis Study Group. Copolymer 1 reduces relapse rate and improves disability in relapsing-remitting multiple sclerosis: results of a phase III multicenter, double-blind placebo-controlled trial.
      ,
      • Polman C.H.
      • O'Connor P.W.
      • Havrdova E.
      • Hutchinson M.
      • Kappos L.
      • Miller D.H.
      • et al.
      A randomized, placebo-controlled trial of natalizumab for relapsing multiple sclerosis.
      ). Treatment effect is calculated through the impact of those two factors upon the time to reach the different EDSS states, and this will be reflected in the estimation of the EDSS distribution along the 30 years that will be displayed, together a prediction of the QALYs gain at 30 years, estimated as described before. Secondly, treatment effect can be monitored through the early recognition of any deviation in the disability curves and MSSS progression graphics. Finally, by introducing yearly data in addition to the EDDS (number of mild and severe relapses, and number of new Magnetic Ressonance Imaging lesions), a textual recommendation is displayed in accordance to published consensus adopted criteria (
      • Jeffery D.
      • Bashir K.
      • Buchwald L.
      • Coyle P.
      • Freedman M.
      • Markowitz C.
      • et al.
      Optimizing immunomodulatory therapy for MS patients: an integrated management model.
      ).

      2.2 Model validation

      Data for the validation of the simulation model was retrospectively obtained from the hospital medical records of a group of 50 patients selected from the total of the 173 patients observed by the author between January and July of 2012 in the MS outpatient clinic of Hospital Egas Moniz. The following criteria were used for selecting the validation group: Patients with relapsing-remitting multiple sclerosis (RRMS) with at least 10 years of disease evolution and sufficient data in their medical records, namely the number of relapses at years 1, 2, 5 and 10, and the EDSS scores at years 2, 5 and 10.
      The study has been approved by the Hospital Ethics Committee and performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments.
      The validation procedure consisted of assessing the concordance degree between the simulation model predictions and the real value presented by the patients, for 10 and 20 years estimations, using patient EDSS at years 2 and 5 of disease evolution.

      3. Results

      Thirty seven patients among the 173 patients observed were discharged because of lack of data in their clinical record together with patient incapacity in describing the facts related to their initial clinical history.
      Data from the validation group of 50 patients is presented in Table 1, as compared to the data of the overall 136 patients.
      Table 1Validation group as compared to overall patient characteristics.
      RRMS
      RRMS: relapsing-remitting multiple sclerosis.
      (N=127)
      (N=50)PPMSm
      PPMS: primary progressive multiple sclerosis.
      (N=9)
      Gender
       Male—N (%)31 (24.4%)13 (26%)5 (55.6%)
       Female—N (%)96 (75.6%)37 (74%)4 (44.4%)
      Age at disease onset
       Mean (SD
      SD: standard deviation.
      )
      28.0 (± 17.4)28.42 (± 8.39)
       Median27.825.65
      Functional system affected—N (%)
       Motor33 (26.0%)12 (24.0%)
       Sensory37 (29.1%)18 (36.0%)
       Coordenation7 (5.5%)1 (2.0%)
       Visual34 (26.8%)14 (28.0%)
       Brainstem11 (8.7%)5 (10.0%)
       Undefined5 (3.9%)0
      Interval 1st–2nd relapse
       Mean (SD
      SD: standard deviation.
      )
      4.8 (±11.1)4.66 (±6.03)
       Median2.02.59
      a RRMS: relapsing-remitting multiple sclerosis.
      b PPMS: primary progressive multiple sclerosis.
      c SD: standard deviation.
      All the data and validation procedures described next focus on this subgroup of 50 patients. The mean age at onset was 28.4 years (± 8.4, median 25,6), mean disease evolution 17.1 (±7.8), and the mean EDSS at the last observation was 3 (± 2.2), median 2. The number of relapses and EDSS scores were recorded (Table 2); the mean time to reach EDSS 4 was 12.4 (±6.9) years and EDSS 6 was 20.9 (±8.8), median 11 and 18 respectively.
      Table 2Recorded relapses and EDSS
      EDSS: Expanded Disability Status Scale.
      scores.
      Year 1Year 2Year 5Year 10
      Mean number of relapses (SD
      SD: standard deviation.
      )
      0.47 (±0,7)0.80 (±1.1)1.60 (±1.4)2,76 (±2.0)
      EDSS mean score (SD
      SD: standard deviation.
      )
      0.76 (±1.0)1.27 (±1.0)2,02 (±2.0)
      a EDSS: Expanded Disability Status Scale.
      b SD: standard deviation.
      Table 3 presents the correlation between the number of relapses and EDSS at years 5 and 10, while Table 4 presents the correlation between EDSS scores at different years.
      Table 3Correlation between number of relapses and EDSS.
      EDSS
      EDSS: Expanded Disability Status Scale.
      at year 5
      EDSS
      EDSS: Expanded Disability Status Scale.
      at year 10
      Number of relapses at year 20.50 (p=0.0001)0.26 (p=0.031)
      Number of relapses at year 50.60 (p<0.0001)0.40 (p=0.002)
      a EDSS: Expanded Disability Status Scale.
      Table 4Correlation between EDSS
      EDSS: Expanded Disability Status Scale.
      at early stages with EDSS
      EDSS: Expanded Disability Status Scale.
      at 10 years.
      EDSS
      EDSS: Expanded Disability Status Scale.
      at year 5
      EDSS
      EDSS: Expanded Disability Status Scale.
      at year 10
      EDSS
      EDSS: Expanded Disability Status Scale.
      at year 2
      0.75 (p<0.0001)0.60 (p<0.0001)
      EDSS
      EDSS: Expanded Disability Status Scale.
      at year 5
      0.77 (p<0.0001)
      a EDSS: Expanded Disability Status Scale.
      As to the model 10 years prediction, a strong correlation was found both with the disability curves (Pearson coefficient=0.75; p<0.0001) and the MSSS projection (Pearson coefficient=0.67; p<0.0001). If one considers the minimum–maximum interval of MSSS estimation, in 76% of cases the system made an adequate prediction. As to the 20 years prediction of the MSSS, similar correlation degree (Pearson coefficient=0.67; p=0.0016) and probability of min–max correct estimation (76%—13 of 17 patients) were found.
      Because lack of data it was not possible to evaluate the model behavior concerning the patient characteristics at disease onset, neither treatment effect.

      4. Discussion

      NH studies of MS are especially useful to contribute to the practical exercise of prognosis formulation and to enable the evaluation of effectiveness in the context of treatment. Analysis of the MS' NH using different statistical approaches, proposed similar conclusions concerning the prognosis influence of some basic predictors. However, because of high variation amongst patients' disease course only rough predictions for the individual patient are possible.
      The present study combined the results of longitudinal studies (
      • Achiron A.
      • Barak Y.
      • Rotstein Z.
      Longitudinal disability curves for predicting the course of relapsing-remitting multiple sclerosis.
      ,
      • Bergamaschi R.
      • Quaglini S.
      • Trojano M.
      • Amato M.P.
      • Tavazzi E.
      • Paolicelli D.
      • et al.
      Early prediction of the long term evolution of multiple sclerosis: the Bayesian Risk Estimate for Multiple Sclerosis (BREMS) score.
      ,
      • Bergamaschi R.
      • Berzuini C.
      • Romani A.
      • Cosi V.
      Predicting secondary progression in relapsing-remitting multiple sclerosis: a Bayesian analysis.
      ,
      • Roxburgh R.H.
      • Seaman S.R.
      • Masterman T.
      • Hensiek A.E.
      • Sawcer S.J.
      • Vukusic S.
      • et al.
      Multiple Sclerosis Severity Score Using disability and disease duration to rate disease severity.
      ) with previous NH studies and disease prognostic factors, in order to provide clinicians with an easy to use tool concerning individual long term disability prediction and treatment effect on the estimated individual prognosis, enabling a comprehensive interaction with the patient with MS for effective SDM. For such purpose, a computer model able to simulate individual patient prognosis was implemented. Based on patient characteristics at disease onset, the model provides individual 30 years disability prediction derived from the time to reach EDSS scores 4, 6 and 7. It also enables to asses treatment effect on the estimated individual prognosis, as well as to monitor deviations in the disability curves (percentiles) and in the MSSS (deciles), as patient data in subsequent years is provided.
      Modeling and simulation have been used in a variety of scientific domains (
      • Law A
      • Kelton W.D.
      Simulation modeling and analysis.
      ), and significant improvement in decision-making, efficiency and quality reported (

      Kuljis J, Paul RJ, Stergioulas LK. Can health care benefit from modeling and simulation methods in the same way as business and manufacturing has? In: Winter simulation conference proceedings 2007; p. 1449–53.

      ). The main advantage of modeling is to facilitate to understand the behavior of a real system and then to test it through a variety of simulations of different scenarios.
      The simulation model now presented is drawn upon available scientific evidence relevant for the defined purpose. However, in general, available data are scarce and/or not adequate for the modeling requirements limiting model validation. Ultimately, the main limitations of the current model are the limitations of the data that the model proposes to represent and process. In order to ensure consistency, a single data source was used but a meta-analysis of several sources is also a valid option.
      Promising results were obtained through performed simulations with data from real patients when complementing model estimations based on patient characteristics at disease onset with initial defined percentiles and deciles. Using the initial EDSS it is already possible to predict progression along percentiles and deciles regardless of relapses, since relapses seem not significantly influence long term disability (
      • Confavreux C.
      • Vukusic S.
      • Moreau T.
      • Adeleine P
      Relapses and progression of disability in multiple sclerosis.
      ), as also documented in the present study. Initial estimations can be later refined by specifying EDSS scores in subsequent years.
      A major added value of the current model consists of its monitoring capabilities particularly by enabling the clinician to be aware early of any change to expected course of disease. Any up-deviation from assigned percentile or decile means unexpected disease deterioration, hence the need for intervention (e.g. treatment switch or escalation). This capability is further refined by means of the criteria for the identification of sub-optimal treatment response. The results obtained through the assessment of the different disease-modifying agents effect in terms of QALYs gain after 30 years deserve a comment. Whether the reduced gain obtained is a simulation problem or it represents actually a low impact of treatment into long term accumulated quality of life, remains a question to be solved. Nevertheless, lack of a major impact of disease-modifying agents upon disease severity as measured by the MSSS has been previously reported (
      • Pachner A.R.
      • Steiner I.
      The multiple sclerosis severity score (MSSS) predicts disease severity over time.
      ).
      Previewed future model developments are twofold. On one hand, improvement of the data related to NH of the disease supporting the model is an ongoing process. In particular, partnership with other research and/or MS centers is being envisaged. The overall idea is to validate the model with different data sets and to evaluate model simulations as to the impact of patient characteristics (e.g. prognostic factors) at disease onset upon the time to reach the different EDSS states. Similarly, treatment effect on state transition intervals also needs deeper testing. On the other hand, model integration in a patient record system seems to be a natural extension to current achievements. NetLogo, besides enabling to delivering models as java applets, hence easily incorporated into web pages, also provides a programming interface to other programming languages.

      5. Conclusions

      To be useful for clinicians, a MS prognosis model should provide a reliable prediction at the individual level and a way to monitor disability progression in MS patients. By combining two different approaches of modeling disability progression, the proposed model of prognosis prediction seems to be at least as good as the ones that can be extracted from available scientific evidence. Therefore, one can conclude that the current model is a valuable tool and, because of its conceptual simplicity and easy utilization, this type of simulation model can be used in everyday clinical practice both to monitor progression of disability, and to support the neurologist discussing prognosis scenarios with the individual patient for effective SDM.

      Conflict of interest

      The author has no conflicts of interest or disclosure to report. This work has received no financial support.

      References

        • Achiron A.
        • Barak Y.
        • Rotstein Z.
        Longitudinal disability curves for predicting the course of relapsing-remitting multiple sclerosis.
        Multiple Sclerosis. 2003; 9: 486-491
        • Bergamaschi R.
        • Quaglini S.
        • Trojano M.
        • Amato M.P.
        • Tavazzi E.
        • Paolicelli D.
        • et al.
        Early prediction of the long term evolution of multiple sclerosis: the Bayesian Risk Estimate for Multiple Sclerosis (BREMS) score.
        Journal of Neurology, Neurosurgery, and Psychiatry. 2007; 78: 757-759
        • Bergamaschi R.
        • Berzuini C.
        • Romani A.
        • Cosi V.
        Predicting secondary progression in relapsing-remitting multiple sclerosis: a Bayesian analysis.
        Journal of the Neurological Sciences. 2001; 189: 13-21
        • Brassat D.
        • Recher C.
        • Waubant E.
        • Le Page E.
        • Rigal-Huguet F.
        • Laurent G.
        • et al.
        Therapy-related acute myeloblastic leukemia after mitoxantrone treatment in a patient with MS.
        Neurology. 2002; 59: 954-955
      1. Calfee, J.E. A representative survey of MS patients on attitudes toward the benefits and risks of drug therapy. AEI-Brookings Joint Center for Regulatory Studies; 2006.

        • Confavreux C.
        • Compston A.
        The natural history of multiple sclerosis.
        in: Compston A. McAlpine's multiple sclerosis. 4th ed. Churchill Livingstone Elsevier, London2006 (p. 183–272)
        • Confavreux C.
        • Vukusic S.
        • Adeleine P.
        Early clinical predictors and progression of irreversible disability in multiple sclerosis: an amnesic process.
        Brain. 2003; 126: 770-782
        • Confavreux C.
        • Vukusic S.
        • Moreau T.
        • Adeleine P
        Relapses and progression of disability in multiple sclerosis.
        New England Journal of Medicine. 2000; 343: 1430-1438
        • Confavreux C.
        • Vukusic S.
        Natural history of multiple sclerosis: a unifying concept.
        Brain. 2006; 129: 606-616
        • Cook D.A.
        • Hatala R.
        • Brydges R.
        • Zendejas B.
        • Szostek J.H.
        • Wang A.T.
        • et al.
        Technology-enhanced simulation for health professions education: A systematic review and meta-analysis.
        Journal of the American Medical Association. 2011; 306: 978-988
        • Daumer M.
        • Neuhaus A.
        • Lederer C.
        • Scholz M.
        • Wolinsky J.S.
        • Heiderhoff M.
        Prognosis of the individual course of disease-steps in developing a decision support tool for multiple sclerosis.
        BMC Medical Informatics and Decision Making. 2007; 7: 11
        • Dong Y.
        • Chbat N.W.
        • Gupta A.
        • Hadzikadic M.
        • Gajic O.
        Systems modeling and simulation applications for critical care medicine.
        Annals of Intensice Care. 2012; 2: 18
        • Ebers G.C.
        Prognostic factors for multiple sclerosis: the importance of natural history studies.
        Journal of Neurology. 2005; 252: iii15-iii20
        • Edwards A.
        • Elwyn G.
        • Mulley A.
        Explaining risks: turning numerical data into meaningful pictures.
        British Medical Journal. 2002; 324: 827-830
        • Elwyn G.
        • O'Connor A.
        • Stacey D.
        • Volk R.
        • Edwards A.
        • Coulter A.
        • et al.
        International Patient Decision Aids Standards (IPDAS) Collaboration. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process.
        British Medical Journal. 2006; 333: 417
        • Filippini G.
        • Munari L.
        • Incorvaia B.
        • Ebers G.C.
        • Polman C.
        • D′Amico R.
        • et al.
        Interferons in relapsing remitting multiple sclerosis: a systematic review.
        Lancet. 2003; 361: 545-552
        • Francis G.
        • Grumser Y.
        • Alteri E.
        • Micaleff A.
        • O'Brien F.
        • Alsop J.
        • et al.
        Hepatic reactions during treatment of multiple sclerosis with Interferon β-1a, incidence and clinical significance.
        Drug Safety. 2003; 26: 815-827
        • Freedman M.S.
        • Hughes B.
        • Mikol D.D.
        • Bennett R.
        • Cuffel B.
        • Divan V.
        • et al.
        Efficacy of diasease-modifying therapies in relapsing remitting multiple sclerosis: a systematic comparison.
        European Neurology. 2008; 60: 1-11
        • Gray O.M.
        • Jolley D.
        • Zwanikken C.
        • Trojano M.
        • Grand'Maison F.
        • Duquette P.
        • et al.
        The Multiple Sclerosis Severity Score (MSSS) re-examined: EDSS rank stability in the MSBase dataset increases 5 years after onset of multiple sclerosis.
        Neurology Asia. 2008; 13: 217-219
        • Heesen C.
        • Solari A.
        • Giordano A.
        • Kasper J.
        • Köpke S.
        Decisions on multiple sclerosis immunotherapy: new treatment complexities urge patient engagement.
        Journal of the Neurological Sciences. 2011; 306: 192-197
        • Hughes T.E.
        • Larson L.N.
        Patient involvement in health care. A procedural justice viewpoint.
        Medical Care. 1991; 29: 297-303
        • Jacobs L.D.
        • Cookfair D.L.
        • Rudick R.A.
        • Herndon R.M.
        • Richert J.R.
        • Salazar A.M.
        • et al.
        The Multiple Sclerosis Collaborative Research Group (MSCRG). Intramuscular interferon beta-1a for disease progression in relapsing multiple sclerosis.
        Annals of Neurology. 1996; 39: 285-294
        • Jadad A.R.
        • Haynes R.B.
        • Hunt D.
        • Browman G.P.
        The internet and evidence-based decision-making: a needed synergy for efficient knowledge management in health care.
        Canadian Medical Association Journal. 2000; 162: 362-365
        • Jeffery D.
        • Bashir K.
        • Buchwald L.
        • Coyle P.
        • Freedman M.
        • Markowitz C.
        • et al.
        Optimizing immunomodulatory therapy for MS patients: an integrated management model.
        Journal of the Neurological Sciences. 2002; 201: 89-90
        • Johnson F.R.
        • Özdemir S.
        • Hauber A.B.
        • Kauf T.L.
        Women's stated willingness to accept perceived risk for vasomotor symptom relief.
        Journal of Women's Health. 2007; 16: 1028-1040
        • Johnson F.R.
        • Özdemir S.
        • Mansfield C.A.
        • Hass S.
        • Miller D.W.
        • Siegel C.A.
        • et al.
        Crohn's disease patients' benefit-risk preferences: serious adverse event risks versus treatment efficacy.
        Gastroenterology. 2007; 133: 769-779
        • Johnson K.P.
        • Brooks B.R.
        • Cohen J.A.
        • Ford C.C.
        • Goldstein J.
        • Lisak R.P.
        • et al.
        The Copolymer 1 Multiple Sclerosis Study Group. Copolymer 1 reduces relapse rate and improves disability in relapsing-remitting multiple sclerosis: results of a phase III multicenter, double-blind placebo-controlled trial.
        Neurology. 1995; 45: 1268-1276
        • Kaplan B.
        • Brennan P.F.
        Consumer informatics supporting patients as co-producers of quality.
        Journal of the American Medical Informatics Association. 2001; 8: 209-216
        • Kremenchutzky M.
        • Rice G.P.A.
        • Baskerville J.
        • Wingerchuk M.
        • Ebers G.C.
        The natural history of multiple sclerosis: a geographically based study. 9: observations on the progressive phase of rhe disease.
        Brain. 2006; 129: 584-594
      2. Kuljis J, Paul RJ, Stergioulas LK. Can health care benefit from modeling and simulation methods in the same way as business and manufacturing has? In: Winter simulation conference proceedings 2007; p. 1449–53.

        • Law A
        • Kelton W.D.
        Simulation modeling and analysis.
        McGrawHill Science/Engineering/Math, New York1999
        • Levic Z.M.
        • Dujmovic I.
        • Pekmezovic T.
        • Jareninski M.
        • Marinkovic J.
        • Stojsavljevic N.
        • et al.
        Prognostic factors for survival in multiple sclerosis.
        Multiple Sclerosis. 1999; : 171-178
        • Mazur D.J.
        • Hickam D.H.
        Patients' interpretations of probability terms.
        Journal of General Internal Medicine. 1991; 6: 237-240
        • Miller J.H.
        • Page S.E.
        Complex adaptive systems: an introduction to computational models of social life (Princeton studies in complexity).
        Princeton University Press, Princeton, New Jersey2007
        • O'Connor A.M.
        • Wennberg J.E.
        • Legare F.
        • Llewellyn-Thomas H.A.
        • Moulton B.W.
        • Sepucha K.R.
        • et al.
        Toward the ‘tipping point’: decision aids and informed patient choice.
        Health Affairs. 2007; 26: 716-725
        • Pachner A.R.
        • Steiner I.
        The multiple sclerosis severity score (MSSS) predicts disease severity over time.
        Journal of the Neurological Sciences. 2009; 278: 66-70
        • Polman C.H.
        • O'Connor P.W.
        • Havrdova E.
        • Hutchinson M.
        • Kappos L.
        • Miller D.H.
        • et al.
        A randomized, placebo-controlled trial of natalizumab for relapsing multiple sclerosis.
        New England Journal of Medicine. 2006; 354: 899-910
        • Roxburgh R.H.
        • Seaman S.R.
        • Masterman T.
        • Hensiek A.E.
        • Sawcer S.J.
        • Vukusic S.
        • et al.
        Multiple Sclerosis Severity Score Using disability and disease duration to rate disease severity.
        Neurology. 2005; 64: 1144-1151
        • Runmarker B.
        • Andersen O.
        Prognostic factors in a multiple sclerosis incidence cohort with twenty five years of follow-up.
        Brain. 1993; 116: 117-134
        • Sackett D.L.
        • Rosenberg W.M.C.
        • Gray J.A.M.
        • Haynes R.B.
        • Richardson W.S.
        Evidence-based medicine: what it is and what it isn′t.
        British Medical Journal. 1996; 312: 71-72
        • Scalfari A.
        • Neuhaus A.
        • Muraro P.A.
        • Daumer M.
        • DeLuca G.C.
        • Muraro P.A.
        • et al.
        Early relapses, onset of progression, and late outcome in multiple sclerosis.
        Archives of Neurology. 2012; (Publisehed online November 19)
        • Sepucha K.R.
        • Fowler Jr., F.J.
        • Mulley Jr., A.G.
        Policy support for patient-centered care: the need for measurable improvements in decision quality.
        Health Affairs (Millwood). 2004; : VAR54-VAR62
        • Small S.D.
        Simulation applications for human factors and systems evaluation.
        Anesthesiology Clinics. 2007; 25: 237-259
        • Vukusic S.
        • Confavreux C.
        Natural history of multiple sclerosis: risk factors and prognostic indicators.
        Current Opinion in Neurology. 2007; 20: 269-274
        • Weinshenker B.G.
        • Bass B.
        • Rice G.P.A.
        • Noseworthy J.H.
        • Carriere W.
        • Basekerville J.
        • et al.
        The natural history of multiple sclerosis: a geographically based study: 2. Predictive value of the early clinical course.
        Brain. 1989; 112: 133-146
        • Weinshenker B.G.
        • Rice G.P.A.
        • Noseworthy J.H.
        • Carriere W.
        • Basekerville J.
        • Ebers G.C.
        The natural history of multiple sclerosis: a geographically based study: 4. Applications to planning and interpretation of clinical therapeutic trials.
        Brain. 1991; 114: 1057-1067
        • Wennberg J.E.
        • Fisher E.S.
        • Skinner J.S.
        Geography and the debate over Medicare reform.
        Health Affairs (Millwood). 2002; : W96-W114
      3. Wilensky U, NetLogo. Evanston IL: Center for connected learning and computer based modeling, Northwestern University; 1999. Avaibable from:.〈http://ccl.northwestern.edu/netlogo/〉.

        • Wolfson C.
        • Confavreux C.
        A improvements to a simple Markov model of the natural history of multiple sclerosis: I. Short term prognosis.
        Neuroepidemiology. 1987; 6: 101-115
        • Wolfson C.
        • Confavreux C.A.
        Markov model of the natural history of multiple sclerosis.
        Neuroepidemiology. 1985; 4: 227-239
        • Woloshin S.
        • Schwartz L.M.
        • Moncur M.
        • Gabriel S.
        • Tosteson A.N.
        Assessing values for health: numeracy matters.
        Medical Decision Making. 2001; 21: 382-390
        • Wyatt J.C.
        • Altman D.G.
        Commentary prognostic models: clinically useful or quickly forgotten.
        British Medical Journal. 1995; 311: 1539
        • Yousry T.A.
        • Habil D.M.
        • Major E.O.
        • Ryschkewitsch C.
        • Fahle G.
        • Fischer S.
        • et al.
        Evaluation of patients treated with natalizumab for progressive multifocal leukoencephalopathy.
        New England Journal of Medicine. 2006; 354: 924-933