Research Article| Volume 38, 101513, February 2020

Who is not coming to clinic? A predictive model of excessive missed appointments in persons with multiple sclerosis

  • Elizabeth S. Gromisch
    Corresponding author at: 490 Blue Hills Avenue, Hartford, CT 06112.
    Mandell Center for Multiple Sclerosis, Mount Sinai Rehabilitation Hospital, Trinity Health Of New England, 490 Blue Hills Avenue, Hartford, CT 06112, United States

    Psychology Service, VA Connecticut Healthcare System, 950 Campbell Avenue, West Haven, CT 06516, United States

    Department of Neurology, University of Connecticut School of Medicine, 263 Farmington Avenue, Farmington, CT 06030, United States

    Department of Medical Sciences, Frank H. Netter MD School of Medicine at Quinnipiac University, 370 Bassett Road, North Haven, CT 06473, United States
    Search for articles by this author
  • Aaron P. Turner
    Multiple Sclerosis Center of Excellence West, Veterans Affairs, 1660 South Columbian Way, Seattle, WA 98108, United States

    Rehabilitation Care Service, VA Puget Sound Health Care System, 1660 South Columbian Way, Seattle, WA 98108, United States

    Department of Rehabilitation Medicine, University of Washington, 325 Ninth Avenue, Seattle, WA 98104, United States
    Search for articles by this author
  • Steven L. Leipertz
    Multiple Sclerosis Center of Excellence West, Veterans Affairs, 1660 South Columbian Way, Seattle, WA 98108, United States
    Search for articles by this author
  • John Beauvais
    Psychology Service, VA Connecticut Healthcare System, 950 Campbell Avenue, West Haven, CT 06516, United States

    Department of Psychiatry, Yale University School of Medicine, 333 Cedar Street, New Haven, CT 06510, United States
    Search for articles by this author
  • Jodie K. Haselkorn
    Multiple Sclerosis Center of Excellence West, Veterans Affairs, 1660 South Columbian Way, Seattle, WA 98108, United States

    Rehabilitation Care Service, VA Puget Sound Health Care System, 1660 South Columbian Way, Seattle, WA 98108, United States

    Department of Rehabilitation Medicine, University of Washington, 325 Ninth Avenue, Seattle, WA 98104, United States

    Department of Epidemiology, University of Washington, 325 Ninth Avenue, Seattle, WA 98104, United States
    Search for articles by this author
Published:November 09, 2019DOI:


      • More than 59% missed at least one appointment over a two-year period.
      • Only 4.28% of participants had excessive missed appointments.
      • Appointment and DMT adherence are associated, but the direction is still unclear.
      • This tool may help to identify those with more challenges for care coordination.


      Background: Missed appointments can have negative effects on several facets of healthcare, including disruption of services, worse patient health outcomes, and increased costs. The influence of demographic and clinical factors on missed appointments has been studied in a number of chronic conditions, but not yet in multiple sclerosis (MS). Engagement in healthcare services is a particular concern with this population, given the complexity of the condition. Furthermore, excessive missed appointments has emerged as a risk factor for suboptimal adherence to disease modifying therapies (DMTs), prompting further exploration into this issue and whether a tool could be developed to triage possible interventions for persons with MS on DMTs who are missing their appointments. As such, this study aimed to investigate the rate and factors associated with missed appointments among a large national sample of persons with MS and develop a predictive model of excessive missed appointments.
      Methods: Administrative data from 01/01/2013 to 12/31/2015 were extracted from the VA MS Center of Excellence Data Repository. Variables not related to excessive missed appointments, defined as missing more than 20% of scheduled appointments, in bivariate analyses (p > 0.20) were excluded. Remaining baseline co-occurring conditions, demographic, and healthcare utilization variables were entered into a logistic regression model, using a backward elimination criteria of p < 0.05. Calibration and discrimination of the model were assessed. An initial predictive score was generated based on the value of the variable and its β-value from the final model.
      Results: The number of missed appointments ranged from 0 to 84 over a two-year period. Over 59% missed at least one appointment, though only 4.28% had excessive missed appointments. Seven variables were retained in the model: adherence to DMTs, age, distance, histories of post-traumatic stress disorder, congestive heart failure, and chronic obstructive pulmonary disease, and emergency visits. Predictive scores ranged from -6.42 to 0.96 (M = -2.61, SD = 1.15). The final model had good discrimination, calibration, and fit.
      Conclusions: By using this model and accompanying score, clinicians could have a good chance of predicting individuals who will miss more than 20% of their appointments and triaging interventions.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Multiple Sclerosis and Related Disorders
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Bodenlos J.S.
        • Grothe K.B.
        • Whitehead D.
        • Konkle-Parker D.J.
        • Jones G.N.
        • Brantley P.J.
        Attitudes toward health care providers and appointment attendance in HIV/AIDS patients.
        J. Assoc. Nurses AIDS Care. 2007; 18: 65-73
        • Bofill L.
        • Waldrop-Valverde D.
        • Metsch L.
        • Pereyra M.
        • Kolber M.A.
        Demographic and psychosocial factors associated with appointment attendance among HIV-positive outpatients.
        AIDS Care. 2011; 23: 1219-1225
        • Boos E.M.
        • Bittner M.J.
        • Kramer M.R.
        A profile of patients who fail to keep appointments in a Veterans Affairs primary care clinic.
        WMJ. 2016; 115: 185-190
        • Burks J.
        • Marshall T.S.
        • Ye X.
        Adherence to disease-modifying therapies and its impact on relapse, health resource utilization, and costs among patients with multiple sclerosis.
        CEOR. 2017; 9: 251
        • Catz S.L.
        • McClure J.
        • Jones G.
        • Brantley P.
        Predictors of outpatient medical appointment attendance among persons with HIV.
        AIDS Care. 1999; 11: 361-373
        • Chiu C.
        • Bishop M.
        • Pionke J.
        • Strauser D.
        • Santens R.L.
        Barriers to the accessibility and continuity of health-care services in people with multiple sclerosis: a literature review.
        Int J MS Care. 2017; 19: 313-321
        • Ciechanowski P.
        • Russo J.
        • Katon W.
        • Simon G.
        • Ludman E.
        • Von Korff M.
        • Young B.
        • Lin E.
        Where is the patient? The association of psychosocial factors and missed primary care appointments in patients with diabetes.
        Gen Hosp Psychiatry. 2006; 28: 9-17
        • Compton M.T.
        • Rudisch B.E.
        • Craw J.
        • Thompson T.
        • Owens D.A.
        Predictors of missed first appointments at community mental health centers after psychiatric hospitalization.
        Psychiatr. Serv. 2006; 57: 531-537
        • Culpepper W.
        • Ehrmantrout M.
        • Wallin M.
        • Flannery K.
        • Bradham D.
        Identification of the VHA MS surveillance registry. The problem of case finding from administrative databases.
        J Rehabil Res Dev. 2006; 43: 17-24
        • Daggy J.
        • Lawley M.
        • Willis D.
        • Thayer D.
        • Suelzer C.
        • DeLaurentis P.-.C.
        • Turkcan A.
        • Chakraborty S.
        • Sands L.
        Using no-show modeling to improve clinic performance.
        Health Informatics J. 2010; 16: 246-259
        • Daniels K.
        • Loganathan M.
        • Wilson R.
        • Kasckow J.
        Appointment attendance in patients with schizophrenia.
        Clin. Pract. 2014; 11: 467
        • Dantas L.F.
        • Fleck J.L.
        • Oliveira F.L.C.
        • Hamacher S.
        No-shows in appointment scheduling–a systematic literature review.
        Health Policy (New York). 2018; 122: 412-421
        • Field A.
        Discovering Statistics Using SPSS.
        3rd edition. Sage Publications Ltd, London, UK2009
        • Freedman M.S.
        Disease-modifying drugs for multiple sclerosis: current and future aspects.
        Expert Opin Pharmacother. 2006; 7: S1-S9
        • George A.
        • Rubin G.
        Non-attendance in general practice: a systematic review and its implications for access to primary health care.
        Fam Pract. 2003; 20: 178-184
        • Gromisch E.S.
        • Turner A.P.
        • Leipertz S.L.
        • Beauvais J.
        • Haselkorn J.K.
        Risk factors for suboptimal medication adherence: development of an explanatory model for disease modifying therapy use.
        Arch Phys Med Rehabil. 2019;
        • Guy R.
        • Hocking J.
        • Wand H.
        • Stott S.
        • Ali H.
        • Kaldor J.
        How effective are short message service reminders at increasing clinic attendance? A meta‐analysis and systematic review.
        Health Serv Res. 2012; 47: 614-632
        • Halpern R.
        • Agarwal S.
        • Borton L.
        • Oneacre K.
        • Lopez-Bresnahan M.V.
        Adherence and persistence among multiple sclerosis patients after one immunomodulatory therapy failure: retrospective claims analysis.
        Adv Ther. 2011; 28: 761
        • Hartung D.M.
        • Bourdette D.N.
        • Ahmed S.M.
        • Whitham R.H.
        The cost of multiple sclerosis drugs in the US and the pharmaceutical industry: too big to fail?.
        Neurology. 2015; 84: 2185-2192
        • Hasvold P.E.
        • Wootton R.
        Use of telephone and SMS reminders to improve attendance at hospital appointments: a systematic review.
        J Telemed Telecare. 2011; 17: 358-364
        • Hussain-Gambles M.
        • Neal R.D.
        • Dempsey O.
        • Lawlor D.A.
        • Hodgson J.
        Missed appointments in primary care: questionnaire and focus group study of health professionals.
        Br J Gen Pract. 2004; 54: 108-113
        • Karter A.J.
        • Parker M.M.
        • Moffet H.H.
        • Ahmed A.T.
        • Ferrara A.
        • Liu J.Y.
        • Selby J.V.
        Missed appointments and poor glycemic control: an opportunity to identify high-risk diabetic patients.
        Med Care. 2004; : 110-115
        • Kheirkhah P.
        • Feng Q.
        • Travis L.M.
        • Tavakoli-Tabasi S.
        • Sharafkhaneh A.
        Prevalence, predictors and economic consequences of no-shows.
        BMC Health Serv Res. 2015; 16: 13
        • Lacy N.L.
        • Paulman A.
        • Reuter M.D.
        • Lovejoy B.
        Why we don't come: patient perceptions on no-shows.
        Ann. Family Med. 2004; 2: 541-545
        • Lorig K.R.
        • Holman H.R.
        Self-management education: history, definition, outcomes, and mechanisms.
        Ann. Behav. Med. 2003; 26: 1-7
        • Macharia W.M.
        • Leon G.
        • Rowe B.H.
        • Stephenson B.J.
        • Haynes R.B.
        An overview of interventions to improve compliance with appointment keeping for medical services.
        JAMA. 1992; 267: 1813-1817
        • Marrie R.A.
        • Hanwell H.
        General health issues in multiple sclerosis: comorbidities, secondary conditions, and health behaviors.
        Continuum (Minneap Minn). 2013; 19: 1046-1057
        • Marrie R.A.
        • Leung S.
        • Tyry T.
        • Cutter G.R.
        • Fox R.
        • Salter A.
        Use of eHealth and mHealth technology by persons with multiple sclerosis.
        Mult Scler Relat Disord. 2019; 27: 13-19
        • McLean S.M.
        • Booth A.
        • Gee M.
        • Salway S.
        • Cobb M.
        • Bhanbhro S.
        • Nancarrow S.A.
        Appointment reminder systems are effective but not optimal: results of a systematic review and evidence synthesis employing realist principles.
        Patient Prefer Adherence. 2016; 10: 479
        • McQueenie R.
        • Ellis D.A.
        • McConnachie A.
        • Wilson P.
        • Williamson A.E.
        Morbidity, mortality and missed appointments in healthcare: a national retrospective data linkage study.
        BMC Med. 2019; 17: 2
        • Mitchell A.J.
        • Selmes T.
        Why don't patients attend their appointments? Maintaining engagement with psychiatric services.
        Adv. Psychiatr. Treat. 2007; 13: 423-434
        • Moore C.G.
        • Wilson-Witherspoon P.
        • Probst J.C.
        Time and money: effects of no-shows at a family practice residency clinic.
        Family Med. 2001; 33: 522-527
      1. National Multiple Sclerosis Society, 2018. Disease-modifying therapies for MS. (Accessed September 17 2018,).

        • Neal R.D.
        • Hussain-Gambles M.
        • Allgar V.L.
        • Lawlor D.A.
        • Dempsey O.
        Reasons for and consequences of missed appointments in general practice in the UK: questionnaire survey and prospective review of medical records.
        BMC Fam Pract. 2005; 6: 47
        • Nguyen D.L.
        • DeJesus R.S.
        • Wieland M.L.
        Missed appointments in resident continuity clinic: patient characteristics and health care outcomes.
        J Grad Med Educ. 2011; 3: 350-355
        • Nwabuo C.C.
        • Dy S.M.
        • Weeks K.
        • Young J.H.
        Factors associated with appointment non-adherence among African-Americans with severe, poorly controlled hypertension.
        PLoS ONE. 2014; 9e103090
        • Ogedegbe G.
        • Schoenthaler A.
        • Fernandez S.
        Appointment-keeping behavior is not related to medication adherence in hypertensive African-Americans.
        J Gen Intern Med. 2007; 22: 1176-1179
        • Osterberg L.
        • Blaschke T.
        Adherence to medication.
        New Engl. J. Med. 2005; 353: 487-497
        • Rae-Grant A.D.
        • Turner A.P.
        • Sloan A.
        • Miller D.
        • Hunziker J.
        • Haselkorn J.K.
        Self-management in neurological disorders: systematic review of the literature and potential interventions in multiple sclerosis care.
        J Rehabil Res Dev. 2011; 48: 1087
        • Reynolds M.W.
        • Stephen R.
        • Seaman C.
        • Rajagopalan K.
        Persistence and adherence to disease modifying drugs among patients with multiple sclerosis.
        Curr Med Res Opin. 2010; 26: 663-674
        • Ritchie P.
        • Jenkins M.
        • Cameron P.
        A telephone call reminder to improve outpatient attendance in patients referred from the emergency department: a randomised controlled trial.
        Aust N Z J Med. 2000; 30: 585-592
        • Sainani K.L.
        Explanatory versus predictive modeling.
        PM&R. 2014; 6: 841-844
        • Shevil E.
        • Finlayson M.
        Process evaluation of a self-management cognitive program for persons with multiple sclerosis.
        Patient Educ Couns. 2009; 76: 77-83
        • Siegel S.D.
        • Turner A.P.
        • Haselkorn J.K.
        Adherence to disease-modifying therapies in multiple sclerosis: does caregiver social support matter?.
        Rehabil Psychol. 2008; 53: 73
        • Tsivgoulis G.
        • Katsanos A.H.
        • Grigoriadis N.
        • Hadjigeorgiou G.M.
        • Heliopoulos I.
        • Papathanasopoulos P.
        • Kilidireas C.
        • Voumvourakis K.
        • Dardiotis E.
        The effect of disease modifying therapies on disease progression in patients with relapsing-remitting multiple sclerosis: a systematic review and meta-analysis.
        PLoS ONE. 2015; 10e0144538
        • Turner A.
        • Kivlahan D.
        • Sloan A.
        • Haselkorn J.
        Predicting ongoing adherence to disease modifying therapies in multiple sclerosis: utility of the health beliefs model.
        MSJ. 2007; 13: 1146-1152
        • Turner A.P.
        • Chapko M.K.
        • Yanez D.
        • Leipertz S.L.
        • Sloan A.P.
        • Whitham R.H.
        • Haselkorn J.K.
        Access to multiple sclerosis specialty care.
        PM&R. 2013; 5: 1044-1050
        • Turner A.P.
        • Williams R.M.
        • Sloan A.P.
        • Haselkorn J.K.
        Injection anxiety remains a long-term barrier to medication adherence in multiple sclerosis.
        Rehabil Psychol. 2009; 54: 116
        • Wallin M.T.
        • Culpepper W.J.
        • Campbell J.D.
        • Nelson L.M.
        • Langer-Gould A.
        • Marrie R.A.
        • Cutter G.R.
        • Kaye W.E.
        • Wagner L.
        • Tremlett H.
        The prevalence of MS in the United States: a population-based estimate using health claims data.
        Neurology. 2019; (0000000000007035)
        • Youden W.J.
        Index for rating diagnostic tests.
        Cancer. 1950; 3: 32-35