Advertisement

Identification of cognitive impairment, depression, and fatigue among multiple sclerosis patients in a large comprehensive care center: A mixed-methods, qualitative study

Open AccessPublished:August 19, 2022DOI:https://doi.org/10.1016/j.msard.2022.104117

      Highlights

      • Multiple sclerosis (MS) patients commonly experience cognitive impairment, depression, and fatigue (CDF).
      • Providers recognize the importance of CDF to their MS patients, but standardized CDF screening tools are used infrequently in clinical practice.
      • In lieu of standardized tools, providers rely on subjective assessments of CDF.
      • Building standardized screening into institutional practices may increase screening with validated measures.
      • Providers need more access to support staff and electronic screening tools successfully implement standardized CDF assessments in clinical practice.

      Abstract

      Background

      Despite studies suggesting a high prevalence of cognitive impairment, depression, and fatigue (CDF) among patients with multiple sclerosis (MS), standardized CDF tools are used infrequently in clinical practice, potentially resulting in underdiagnosis. We documented the use of standardized tools to identify CDF in MS and sought to understand provider attitudes toward the tools and their use.

      Methods

      This mixed-methods study analyzed electronic health records (EHRs) from a large US urban MS center to determine the frequency and types of CDF screenings and numbers of MS treatment encounters (January 2018–December 2019). Participants included neurologists and nurse practitioners with ≥30 eligible patients and a convenience sample of adult MS patients (≥18 years) with at least outpatient encounters during the study period. Semistructured provider interviews (n = 6; the principal investigator and 1 provider were excluded) were conducted, transcribed, coded, and analyzed to characterize screening patterns. Assessments included proportions of encounters and patients who had standardized CDF screenings, positive screening results, and documentation of a treatment recommendation, as well as provider attitudes toward tools and reported barriers and facilitators for use. Bivariate analysis was used to evaluate the relationship between screening rates and patient and provider covariates for groups with sufficient sample size (n = 30).

      Results

      The final population included 260 unique patients, 489 outpatient encounters, and 8 providers. Of 260 patients (75% female, 83% aged <65 years), 24% (n = 63) were screened with a depression tool. Only 2% (n = 4) were screened with a tool measuring cognitive impairment, and none were screened with a tool measuring fatigue. Screening rates varied little by provider type. Higher depression screening rates were associated with white race (difference: 13.2%; 95% CI: 2.8−23.5%; P = .01), ≤2 visits during the study period (difference: 7.6%; 95% CI: 0.6−14.5%; P = .03), and provider experience >10 years (difference: 14.6%; 95% CI: 3.5−25.8%; P = .01). Lack of support staff and perception of limited treatment options were commonly cited barriers to standardized screening in provider interviews. The higher rate of depression screening is likely driven by institutional culture and priorities.

      Conclusion

      Providers recognize the importance of CDF to patients, despite infrequent use of standardized screening. Integrating screening into institutional practices may enable ongoing tracking of assessment scores and provide a more comprehensive and longitudinal picture of symptom progression.

      Keywords

      1. Introduction

      Multiple sclerosis (MS) is an immune-mediated neurologic disorder. As disease-modifying therapies have become more effective for physical symptoms of MS, neuropsychologic symptoms have become increasingly important as therapeutic targets (
      • McNicholas N.
      • O'Connell K.
      • Yap S.M.
      Cognitive dysfunction in early multiple sclerosis: a review.
      ). The complex interrelationship between cognitive impairment, depression, and fatigue (CDF) in people with MS (PwMS) underscores the importance of objective screening for identifying and treating the neuropsychologic symptoms that affect quality of life (
      • Bradshaw J.
      Cognition, depression and fatigue in multiple sclerosis.
      ;
      • Marrie R.A.
      • Patten S.B.
      • Berrigan L.I.
      • Tremlett H.
      • Wolfson C.
      • Warren S.
      • Leung S.
      • Fiest K.M.
      • McKay K.A.
      • Fisk J.D.
      CIHR Team in the Epidemiology and Impact of Comorbidity on Multiple, S.E.
      Diagnoses of depression and anxiety versus current symptoms and quality of life in multiple sclerosis.
      ).
      Epidemiologic survey studies suggest a high prevalence of CDF based on self-report in various populations with MS (clinics, internet patient groups, and MS registries) using standardized screening tools and self-reported histories. The prevalences range from 60–65% for cognitive impairment, 36–54% for depression, and up to 92% for fatigue, which are seldom matched in empirical studies (

      Costello, K., Kalb, R., for the 2019 Revision-Writing Team, 2019. The use of disease-modifying therapies in multiple sclerosis. Principles and current evidence [a consensus paper]. https://www.nationalmssociety.org/NationalMSSociety/media/MSNationalFiles/Brochures/DMT_Consensus_MS_Coalition.pdf, (accessed March 23, 2021).

      ;
      • Marrie R.A.
      • Patten S.B.
      • Berrigan L.I.
      • Tremlett H.
      • Wolfson C.
      • Warren S.
      • Leung S.
      • Fiest K.M.
      • McKay K.A.
      • Fisk J.D.
      CIHR Team in the Epidemiology and Impact of Comorbidity on Multiple, S.E.
      Diagnoses of depression and anxiety versus current symptoms and quality of life in multiple sclerosis.
      ;

      Minden, S., Turner, A., Kalb, R., Burke, D., 2014. Emotional disorders in multiple sclerosis clinical bulletin; information for health professionals. http://www.nationalmssociety.org/NationalMSSociety/media/MSNationalFiles/Brochures/Clinical-Bulletin-Emotional-Disorders-5-5-14.pdf, (accessed May 3, 2021).

      ). The American Academy of Neurology (AAN) recommends annual screening for CDF among PwMS (

      Bever, C., Rae-Grant, A., for the American Academy of Neurology Multiple Sclerosis Work Group Members, 2014. Multiple sclerosis quality measurement set. https://www.aan.com/siteassets/home-page/policy-and-guidelines/quality/quality-measures/14msmeasureset_pg.pdf, (accessed March 22, 2021).

      ;
      • Rae-Grant A.
      • Bennett A.
      • Sanders A.E.
      • Phipps M.
      • Cheng E.
      • Bever C.
      Quality improvement in neurology: multiple sclerosis quality measures: executive summary.
      ). Although standardized tools are useful for tracking CDF over time, subjective reports contribute to a comprehensive clinical profile. For example, patient-reported cognitive complaints have been recognized as a preclinical stage of mild cognitive impairment (
      • Geerlings M.I.
      • Jonker C.
      • Bouter L.M.
      • Adèr H.J.
      • Schmand B.
      Association between memory complaints and incident Alzheimer's disease in elderly people with normal baseline cognition.
      ;
      • Kletenik I.
      • Alvarez E.
      • Honce J.M.
      • Valdez B.
      • Vollmer T.L.
      • Medina L.D.
      Subjective cognitive concern in multiple sclerosis is associated with reduced thalamic and cortical gray matter volumes.
      ). Reliance on subjective evaluation of CDF, however, can pose a risk to its accurate and timely identification. Romero et al. observe that signs of cognitive impairment in particular are more subtle in PwMS than in people with other neurological conditions; for example, clinical examination is often insensitive to the dysfunctions in information processing speed characteristic of MS (
      • Romero K.
      • Shammi P.
      • Feinstein A.
      Neurologists׳ accuracy in predicting cognitive impairment in multiple sclerosis.
      ). Patients with CDF present with overlapping symptoms, making it difficult to distinguish these conditions based on a subjective patient/caregiver report; thus, objective screening with standardized tools may be helpful for identifying the underlying conditions and targeting treatment accordingly (
      • Hughes A.J.
      • Bhattarai J.J.
      • Paul S.
      • Beier M.
      Depressive symptoms and fatigue as predictors of objective-subjective discrepancies in cognitive function in multiple sclerosis.
      ;
      • Kalb R.
      • Beier M.
      • Benedict R.H.
      • Charvet L.
      • Costello K.
      • Feinstein A.
      • Gingold J.
      • Goverover Y.
      • Halper J.
      • Harris C.
      • Kostich L.
      • Krupp L.
      • Lathi E.
      • LaRocca N.
      • Thrower B.
      • DeLuca J.
      Recommendations for cognitive screening and management in multiple sclerosis care.
      ).
      Stepleman et al. remark that when depression is untreated, MS-related health may deteriorate; the direct and indirect effect that CDF symptoms may have, not only on one another but also on MS itself, emphasizes the necessity of early provider recognition of and intervention in CDF signs and symptoms (
      • Stepleman L.M.
      • Penwell-Waines L.M.
      • Rollock M.
      • Casillas R.S.
      • Brands T.
      • Campbell J.
      • Ange B.
      • Waller J.L.
      Routine depression screening in an MS clinic and its association with provider treatment recommendations and related treatment outcome.
      ). Despite the high prevalence, however, CDF is frequently underdiagnosed and undertreated among PwMS, with reported rates of missed diagnosis around 23–30% (
      • Marrie R.A.
      • Horwitz R.
      • Cutter G.
      • Tyry T.
      • Campagnolo D.
      • Vollmer T.
      The burden of mental comorbidity in multiple sclerosis: frequent, underdiagnosed, and undertreated.
      ;
      • Marrie R.A.
      • Patten S.B.
      • Berrigan L.I.
      • Tremlett H.
      • Wolfson C.
      • Warren S.
      • Leung S.
      • Fiest K.M.
      • McKay K.A.
      • Fisk J.D.
      CIHR Team in the Epidemiology and Impact of Comorbidity on Multiple, S.E.
      Diagnoses of depression and anxiety versus current symptoms and quality of life in multiple sclerosis.
      ;
      • McGuigan C.
      • Hutchinson M.
      Unrecognised symptoms of depression in a community-based population with multiple sclerosis.
      ;
      • Skokou M.
      • Soubasi E.
      • Gourzis P.
      Depression in multiple sclerosis: a review of assessment and treatment approaches in adult and pediatric populations.
      ). Under-recognition, especially at early stages, could be due to the overlap in symptoms; patient under-reporting (for cognitive impairment); limited time to administer standardized tools; limited evidence of these overlapping conditions in PwMS; insurance coverage; and provider preference for subjective assessments (
      • Bradshaw J.
      Cognition, depression and fatigue in multiple sclerosis.
      ;

      Buckle, G.J., Robertson, D., Moreo, N., Aungst, A., Carter, J., Spair, T., 2018. Screening for cognitive impairment and depression in MS patients: current practices, perceptions, and barriers in selected United States MS clinics. https://actrims.confex.com/actrims/2018/meetingapp.cgi/Paper/2414, (accessed March 23, 2021).

      ;
      • McNicholas N.
      • O'Connell K.
      • Yap S.M.
      Cognitive dysfunction in early multiple sclerosis: a review.
      ).
      Clinical assessment of CDF in PwMS and provider preference around its identification constitute the points of departure for this research. Given the recommendations for annual screening with validated measures, the importance of monitoring CDF over time, the difficulty of detecting certain of its aspects through observation alone, the complex interaction between symptoms, and the high rate of missed diagnosis, it is crucial to understand the rates of and circumstances under which CDF screening with standardized instruments occurs in clinical practice. The primary objective for this study, therefore, was to describe the frequency and types of standardized tools used to identify CDF. The secondary objective was to understand provider preferences and reported barriers and facilitators to the use of these tools.

      2. Methods

      2.1 Design

      This was a single-center study. A mixed-methods approach was used to retrospectively capture and analyze electronic health record (EHR) data for documentation of standardized CDF screenings and treatment among adults with MS with 2–4 visits during a 12-month period between January 2018 and December 2019, inclusively. We assessed 3 primary indicators using structured EHR data to estimate the proportions of patients and encounters with documentation of (1) at least 1 CDF screening with a standardized tool (eg, the Mini-Cog for cognitive impairment, the Patient Health Questionnaire [PHQ] for depression, or the Fatigue Assessment Scale for fatigue); (2) a positive CDF screening result; and (3) a CDF treatment recommendation (pharmacologic or nonpharmacologic). Provider notes were also analyzed for subjective notations of CDF symptoms, defined prior to abstractor training.
      Qualitative data were collected via semistructured interviews conducted in June 2020 with US-based providers. Interviews were analyzed to answer 3 research questions: (1) what are the rates of recognition and documentation of CDF symptoms; (2) what are barriers to/facilitators for assessing CDF symptoms; and (3) what are the attitudes toward/awareness of validated tools and recommended guidelines? Content analysis of interview transcripts and subjective notations of CDF symptoms was performed using NVivo software (QSR International, Melbourne, Australia).

      2.2 Study sample

      Provider participants were neurologists (MDs; n == 5) with at least 30 unique MS patients and MS nurse practitioners (NPs; n == 3) with at least 30 MS patient encounters between 2018 and 2019. Owing to differences in patient volume, NPs were not required to have at least 30 unique MS patients to participate.
      A convenience sample of anonymized patient records produced an initial study population of 264 unique patients representing 500 encounters (22% of the sampling frame; Table 1). Inclusion criteria for patients were age ≥18 years as of January 2018, a G35 ICD-10 diagnosis code (multiple sclerosis), and 2–4 encounters with a qualifying provider at the center. Patients with at least 5 encounters within a 12-month period were excluded as potential outliers, and those with less than 2 encounters were not considered to be routine patients. Patients who saw >1 MD during the study period were excluded to improve attribution; low patient volume made this exclusion impossible for NPs. The final population included 489 outpatient encounters with 260 unique patients.
      Table 1Study participants and patient sample.
      Attrition CategoryDescriptionOverall
      Provider (structured EHR data)Participants entered, n9 (6 MDs; 3 NPs)
      Exclusion, <30 unique patients eligible for patient study sample, n (%; n/N)1 MD (11; 1/9)
      Final participant total, n (%; n/N)8 (89; 8/9)
      Provider (interviews)Participants entered, n8 (5 MDs; 3 NPs)
      Exclusion, provider departed center, n1 (13; 1/8)
      Exclusion, potential bias introduced by PI as study participant, n1 (13%; 1/8)
      Final participant total, n6 (75; 6/8)
      Abbreviations: EHR, electronic health record; MD, physician; NP, nurse practitioner; PI, principal investigator.

      2.3 Data collection

      2.3.1 Chart-abstracted EHRs

      Two medical abstractors were trained to use a data dictionary and data abstraction template, which captured binary and categorical forced-choice responses and free-text fields for subjective notations of CDF. Abstractors entered patient demographics, encounter information, provider information, payer type, documented use of standardized CDF screening tools, screening results, and treatment type. Abstractors reviewed patient charts, copying verbatim text that met inclusion criteria based on the data dictionary definitions for domains, signs, and symptoms of CDF. No personal health information was collected or transmitted for analysis.
      Before analytic sample collection, abstractors participated in 2 rounds of inter-rater reliability testing (n == 25 patients independent of analytic sample) to achieve 90% agreement on critical data elements. Data were assessed at the midpoint and end of the analytic sample data collection to ensure the completeness of the data elements (n == 260 patients).

      2.3.2 Provider interviews

      Data for content analyses were collected via web-based, semistructured interviews after obtaining informed consent from the 7 providers (excluding the principal investigator to reduce bias) who practiced at the center as of June 2020. An experienced qualitative researcher and scribe participated in the discussions. Interview transcripts were coded independently by 2 analysts using an a priori codebook. Using both inductive and deductive methods, an adjudicator reviewed the coded transcripts to ensure that the data were appropriately categorized in NVivo. The protocol was approved by the Georgetown University Institutional Review Board (Washington, DC).

      2.4 Statistical analysis

      Structured EHR data were analyzed to determine the proportions of encounters and patients with a standardized CDF screening, positive CDF screening results, and documentation of a treatment recommendation. The statistical significance of the associations between covariates and study outcomes was tested using a 2-tailed z test for groups with sufficient sample size (n ≥ 30). Data were stratified by provider type and individual provider to assess the heterogeneity of results; z scores and P values were calculated to examine the statistical significance of differences by provider types. For all analyses, P < .05 indicates statistical significance.
      Free-text data fields were imported from the data collection template into NVivo for coding. Content analysis was conducted to identify common themes and key findings. Pearson correlation coefficients (r) tested the linear relationship between interview-reported and chart-abstracted symptoms related to CDF.

      3. Results

      A comparison of the distribution of demographic characteristics in the structured EHR data sample with the center's overall population of patients with MS indicated that the analytic sample was similar to the overall population, with moderate variation in payer type and patient race (Supplementary Material Table 1).

      3.1 CDF screening completion

      Table 2 presents the screening rates for standardized tools. Only 24% of patients (17% of encounters) underwent a standardized depression screening during the study period, with minimal variation by provider (23% [MDs] vs. 26% [NPs]; P == .54). However, only 2% of patients (1% of encounters) underwent a standardized cognitive impairment screening, and none underwent standardized fatigue screening.
      Table 2Standardized screening rates, documented results, and treatment rates.
      DescriptionAll patientsMD PatientsNP PatientsAll EncountersMD EncountersNP Encounters
      Screening rates by conditionCompleted cognitive impairment screenings, % (n/N)2 (4/260)2 (4/169)N/A1 (4/489)1 (4/386)N/A
      Positive00N/A00N/A
      Negative00N/A00N/A
      Unknown100 (4/4)100 (4/4)N/A100 (4/4)100 (4/4)N/A
      Completed depression screenings, % (n/N)24 (63/260)23 (39/169)26 (24/91)17 (84/489)16 (60/386)23 (24/103)
      Positive2 (1/63)3 (1/39)01 (1/84)2 (1/60)0
      Negative92 (58/63)90 (35/39)92 (22/24)93 (78/84)93 (56/60)92 (22/24)
      Unknown6 (4/63)8 (3/39)8 (2/24)6 (5/84)5 (3/60)8 (2/24)
      Completed fatigue screenings, %000000
      PositiveN/AN/AN/AN/AN/AN/A
      NegativeN/AN/AN/AN/AN/AN/A
      UnknownN/AN/AN/AN/AN/AN/A
      Treatment rates by conditionPharmacologic, % (n/N)7 (18/260)9 (16/169)2 (2/91)4 (19/489)4 (17/386)2 (2/103)
      AdamantanesN/AN/AN/A0 (1/489)0 (1/386)0
      AntidepressantsN/AN/AN/A2 (12/489)3 (10/386)1 (1/103)
      Cholinesterase inhibitorsN/AN/AN/A000
      Central nervous system stimulantN/AN/AN/A1 (7/489)1 (5/386)1 (1/103)
      NMDA receptor antagonistsN/AN/AN/A000
      Potassium channel blockersN/AN/AN/A000
      Nonpharmacologic, % (n/N)45 (117/260)43 (73/169)48 (44/91)30 (147/489)26 (102/386)44 (45/103)
      Physical activity recommendationsN/AN/AN/A21 (103/489)19 (72/386)30 (31/103)
      Physical therapyN/AN/AN/A5 (24/489)4 (16/386)8 (8/103)
      Nutrition counselingN/AN/AN/A0 (2/489)N/A0 (1/386)
      Lifestyle recommendationsN/AN/AN/A2 (8/489)2 (7/386)0 (1/386)
      Occupational therapy orderN/AN/AN/A1 (4/489)1 (3/386)1 (2/386)
      Speech/language therapy orderN/AN/AN/A0 (1/489)0 (1/386)N/A
      Behavioral therapy orderN/AN/AN/A1 (7/489)1 (5/386)1 (2/386)
      OtherN/AN/AN/A2 (12/489)2 (7/386)6 (6/103)
      Abbreviations: EHR, electronic health record; MD, physician; N/A, not applicable; NMDA, N-methyl-d-aspartic acid; NP, nurse practitioner.
      Standardized depression screening was significantly associated with white (vs. nonwhite) patient race (P == .01), fewer visits (P == .03), and ≥10 years of provider experience (P == .01) (Table 3). There was no significant relationship between patient age, sex, visit frequency, or provider type and standardized depression screening. Interview responses were compared with screening rates to reveal a significant relationship between provider references to the practice-level PHQ measure and the probability of completion of a depression screening tool.
      Table 3Statistical analysis for depression screening rates based on structured EHR data.
      Category and Difference Between CovariatesCovariateDepression Screening RateConfidence Interval;

      P Value
      Patient race, % (n/N)White31.0 (40/129)2.8 to 23.5; P == .01
       Difference = 13.2%Nonwhite17.8 (23/129)
      No. of patient visits, % (n/N)1 or 2 visits19.3 (69/358)0.6 to 14.5; P == .03
       Difference = 7.6%3 or 4 visits11.7 (15/128)
      Provider experience, % (n/N)<10 years12.5 (6/48)3.5 to 25.8; P == .01
       Difference = 14.6%≥10 years27.1 (57/210)
      Interview reference to practice-level PHQ measureYes36.5 (27/74)21.9 to 44.9; P < .001
       Difference = 33.0%No3.1 (3/97)
      Patient age, % (n/N)<65 years25.2 (54/214)−8.5 to 18.0; P == .48
       Difference = 4.7%≥65 years20.5 (9/44)
      Patient sex, % (n/N)Female24.9 (48/193)−10.1 to 13.7; P == .77
       Difference = 1.8%Male23.1 (15/65)
      Patient encounter type, % (n/N)Urgent7.4 (2/27)0 to 20.9; P == .051
       Difference = 10.5%Nonurgent17.9 (82/459)
      Provider type, % (n/N)MD23.2 (39/168)−14.6 to 7.7; P == .54
       Difference = 3.5%NP26.7 (24/90)
      Abbreviations: EHR, electronic health record; MD, physician; NP, nurse practitioner; PHQ, Patient Health Questionnaire.

      3.2 CDF screening results

      Few positive screens for CDF were documented with a standardized instrument (Table 2). Of the 63 patients screened for depression, only 1 (2%) was documented as having a positive result. None of the charts reviewed contained documentation of a positive result for cognitive impairment.

      3.3 Treatment recommendations

      Table 2 displays the proportion of the MD and NP patient encounters that had documentation of at least 1 pharmacologic or nonpharmacologic treatment in the assessment/plan section of the provider note (note that pharmacologic treatment recommendations were abstracted only for new orders or prescriptions and excluded titration or current medications). Overall, 4% (n == 19) of patient encounters included documentation of at least 1 pharmacologic treatment recommendation, and 30% (n == 147) included documentation of at least 1 nonpharmacologic treatment recommendation. The most common treatment recommendation was physical activity, documented in 21% (n == 103) of encounters.

      3.4 Content analysis

      Despite low recorded use, provider interviews suggested that providers are generally aware of the importance of CDF screenings with standardized tools (Table 4). Initial skepticism about the value of standardized measures was reported to have dissipated in light of the instruments’ robust validation, with one provider noting, “I eventually came around to SDMT [Symbol Digital Modalities Test], which has become much more widely accepted. There's literature to give thresholds for meaningful change. I like that. It's time to incorporate more formal types” (Table 4). Three of the 6 providers referenced the practice's attempts in 2020 to expand the use of the SDMT, while 4 expressed an interest in being able to monitor a score over time using a standardized tool. Providers were less familiar with AAN guidelines recommending annual CDF screenings than they were with specific CDF standardized tools; the SDMT and PHQ-2/PHQ-9 were repeatedly mentioned, while one provider referenced using elements of the Mini-Mental State Examination (MMSE), Montreal Cognitive Assessment, and Mini-Cog instruments to assess cognitive impairment (Table 4). The MMSE has been found to be less sensitive in PwMS than other cognitive screening tools (
      • Macías Islas M.
      • Ciampi E.
      Assessment and impact of cognitive impairment in multiple sclerosis: an overview.
      ).
      Table 4Provider attitudes and awareness toward MS guideline recommendations and standardized tools.
      CategoryAwareNot AwareNo ResponseAttitudes/Comments
      Guidelines: general330• “One problem with guidelines: we are all so egotistical. I have never seen a guideline I agreed with… The concept of doing regular cognitive screening only just begun within 2020. I feel it's important. We did not have good tools before. The other recommendations. I was a little resistant. I would say I am more on board with the principle of it. Regular and consistent using tools. I am on board now and may not have been in past.”
      Guidelines: AAN annual screening recommendations231• “I have seen those guidelines. We are wanting to do the SDMT, which the AAN recommends. That's very difficult now with the pandemic so we have to figure out a way to do it over the e-visit. We are moving more towards doing more. We have a good depression screen, we have a good cog screen, but we have not figure out how to do it virtually.”

      • “We do the PHQ-2/PHQ-9 at least yearly which is not often enough, but I always ask about mood.”
      Standardized tools: general600• “If they are validated tools, I think it would be helpful. I think it would be meaningful for patients to be able to track these values over time and share it with them to give them some insight.”

      • “I think cognition, cognitive deficit and even depression can be subtle, so I think this is where questionnaires might be helpful. I think it's stuff that we may not be able to pick up on.”
      Standardized tools: cognitive impairment501• “The Mini-Mental and Montreal Cognitive Assessment and the Mini-Cog are also ones that are [performed]. We incorporate aspects of that.”

      • “We'd like to do the SDMT. I was not using it on a regular basis, but we were doing it during a clinical trial. It's something we should be doing. I think we will all be transitioning to doing it.”

      • “Along lines of cognition, literally right before COVID, maybe for 1 month. Maybe from Jan 2020 was the first time that I started to incorporate SDMT.”

      • “When somebody is feeling they are declining cognitively, [rather than say], ‘Oh, you are fine. You are not having any flares.’ [Ihad like] to have some tool to say, ‘This is your score last year. There are certainly changes going on’ and acting accordingly.”

      • “I eventually came around to SDMT, which has become much more widely accepted. There's literature to give thresholds for meaningful change. I like that. It's time to incorporate more formal types.”
      Standardized tools: depression600• “For depression, at least I have the PHQ-2, PHQ-9 that are supposed to be standard for each visit.”

      • “We are supposed to. For my MS patients, every visit I aspire to do PHQ-2.”

      • “What I observed with the PHQ-9 … I will say, we would sporadically acquire that info, but nobody noticed. No one was addressing. I do not want a scale and it does not do anything.”

      • “Something new is to have the MA ask screening questions, PHQ-9.”
      Standardized tools: fatigue420• “I do not use it. Is not that terrible? I usually will just write a paragraph as to what's going on rather than. But that's really hard to track over time. It's not as good as a score on something.”

      • “I did not look at anxiety, depression or fatigue scale, mainly because I thought I was not as good at responding to them if they score high. I felt for myself, I might not notice that.”
      Abbreviations: AAN, American Academy of Neurology; PHQ-2/PHQ-9, Patient Health Questionnaire, 2 and 9 items; SDMT, Symbol Digital Modalities Test.
      Providers expressed that standardized tools are most useful if the tools enable early detection and prompt provider action. A prominent theme was the center's attempt to routinely administer the PHQ at the practice level as part of its transformation into a National Committee for Quality Assurance patient-centered specialty practice in early 2018. Depression screening rates were significantly higher among providers who referenced practice-level PHQ measurement standards during interviews than in those who did not (difference: 33%; 95% CI: 21.9–44.9%; P < .001) (Table 3).
      Despite low rates of screening, the content analysis revealed that providers frequently assess and document suspected CDF using subjective evaluations (Table 5). Five providers referenced performing a review of systems for concerns related to cognition, mood, and fatigue (note that a review of systems is an inventory of body systems obtained by verbal history, with the signs and/or symptoms which the patient is experiencing or previously experienced). The spectrum of subjective assessments ranged from proactive, semistructured evaluations to more passive assessments (eg, patient-/caregiver-initiated discussions of symptoms). “Just in talking to people you can pick up if they have cognitive impairment,” said one provider. On the other end of the spectrum, there were 9 examples of tests that providers regularly use in conducting semistructured subjective assessments for cognitive impairment, from naming the current president to spelling “world” and “money” forwards and backwards. Each provider had documentation of use of at least 4 different tests used at least once as part of their semistructured evaluations. As noted, one provider reported incorporating “aspects” of tests, including the MMSE, Montreal Cognitive Assessment, and Mini-Cog; this was, however, the single mention of the Mini-Cog (Table 5).
      Table 5Subjective assessment categories.
      Subjective AssessmentProvider Interview QuotesChart-Abstracted Provider Notes

      (note: not matched by provider interview quote)
      Comprehensive review of systems• “I do a review of neurological systems then for every visit I will tackle mood, fatigue, cognition, and sleep.”

      • “We routinely do a review of systems. We ask questions about cognitive impairment or memory.”

      • “In addition to asking about their physical symptoms, I ask about sleep, fatigue, cognition, and mood.”
      • “ASSESSMENT OF NEUROLOGICAL FUNCTIONS:

      COGNITION: (+) fatigue without cognitive impairment.

      SLEEP: no daytime sleepiness, sleep apnea, or major insomnia.

      MOOD: no history of major depression.”
      Semistructured subjective assessments• “The Mini-Mental and Montreal Cognitive Assessment and the Mini-Cog are also ones that are [performed]. We incorporate aspects of that.”• “Mental Status: A&O to time, place, person, and president of the United States. Registers 3/3 words with 3/3 recall at 5 min. Spells WORLD forward & backward. Calculation intact $2.50=10 quarters.”
      Provider interviews or questioning of patient• “Most of the information is in the question asking in the visit with the patients.”

      • “I just simply ask them, ‘How's your cognition? How's your memory been?’ – things like that. Fatigue comes up. I just simply ask.”

      • “I interview the patient and use my best judgment.”
      • “No new symptoms since the last visit. However, continues to C/O intermittent fatigue”

      • “COGNITION: endorses fatigue, no cognitive impairment. She takes MODAFINIL”

      • “Severe fatigue for the last week. ‘All I do is sleep’. Mental Status: A&O x 3. Headache R51 Cymbalta 30 mg bid.”
      Provider observations• “Just in talking with people, you can pick up if they have cognitive impairment.”

      • “It's not uncommon for a patient to say that they have cognitive issues, but very basic recall issues I do not consider to be pathologic… I usually get worried when they are not so worried.”

      • “You can look at them and tell they are sad.”
      • “Pt was unsure of the month.”

      • “Mental Status: disorganized thought process, poor ST and LT memory, not oriented.”

      • “PHQ-2/9 (Patient Health Questionnaire): Deferred. MOOD: + appears in pain/upset spontaneously at times.”
      Patient concerns• “I do not want to say 50-50, so I will say 60-40 that patients [initiate]… they are very good at initiating if something is going on.”

      • “That a patient brings it up? Very common, but who brings it up first is hard to distinguish.”

      • “A lot of times when we talk about employment, an issue will come up.”

      • “This is a very large population of people who are still working, and their livelihood is a big part of their identity. And if there are changes, either physical or cognitive, fatigue or mood issues that are affecting their ability to function, I would say 60% of my patients will bring that to the table.”
      • “She notes worsening depression recently… She notes brief thoughts of taking a lot of pills several days ago that have now resolved. Depression F32.9.”

      • “Of all her symptoms, most bothered by her mild cognitive difficulties”

      • “Pt reports continued cognitive dysfunction, chronic balance impairment, and sensory dysfunction. Pt does not feel capable to return to work and would like to renew disability.”

      • “Patient states that over the past week, he has noticed increased mental fatigue and fogginess. He has had trouble concentrating and completing his necessary activities at work.”
      Family/caregiver concerns• “I think they are critical. I am aware of the literature that suggest that having somebody accompanying you to a doctor's visit allows you to debrief afterward, but I have found that family members can provide insight that the patients will not provide. Especially for a mood disorder, and that irritability if I ask them, a patient may shrug, ‘I do not know’… But absolutely that outside perspective can be critical, less for fatigue but maybe more for mood disorder and cognition is huge.”

      • “Not so much about depression or mood, that's less so. It's not often that the caregiver will bring [depression or mood] up, but the patient will not, but that's because the patient has already brought it up and the caregiver just agrees.”

      • “I think that when it becomes apparent outwardly to other people around them, they will chime up, ‘Hey, you have an issue.’”

      • “But what we are seeing with the males is if they have a female family member, either a mother or sister or spouse or girlfriend, they seem to be the ones that will bring that information forward.”
      • “He started having some word finding difficulty, which lasts 10-15 seconds and has been ongoing since at least [2018] as per the daughter.”

      • “Husband reports that mood has been much worse lately. Pt endorses depressed mood, loss of interest in activities, fatigue, poor sleep. She continues to take duloxetine 60mg daily (since 2017).”

      • “COGNITION: occasional problem with mental processing speed. Her husband states that she is forgetful.”

      • “Cognitive: his wife believes that his memory is worse. He has not followed up for neurocognitive testing as recommended 1 year ago. He firmly does not believe that he has a problem.”
      Abbreviations: A&O, alert and oriented; C/O, complain of; LT, long-term; PHQ-2/-9, Patient Health Questionnaire, 2 and 9 item; ST, short-term.
      The chart-abstracted provider notes substantiate the range of subjective assessments mentioned in provider interviews (Supplementary Material Figs. 1–3). Evidence-based domains, signs, and symptoms for CDF were identified and analyzed to determine whether they were noted in patient charts. There was moderate agreement between symptoms of cognitive impairment mentioned in provider interviews and abstracted from the EHRs (r == 0.31) and strong agreement for depression (r == 0.72) and fatigue (r == 0.76). The distribution of domains suggests that providers may more readily identify prominent symptoms of CDF, potentially overlooking social cognition, altered thought process, and other aspects that are more challenging to observe.
      Providers noted that the patient relationship and history-taking provided subjective data for assessing CDF and could be more sensitive to changes than standardized tools. One provider remarked that the long-term relationship with patients makes it possible to detect when something is “amiss.” Moreover, when physical symptoms of MS are well controlled, patients may raise neuropsychologic concerns. Provider observations and caregiver concerns are useful for identifying cognitive impairment or depression symptoms unreported or even repudiated by the patient. Although patients may not consistently report CDF symptoms, 2 providers noted that issues that affect employment are likely to prompt a patient to raise CDF concerns (Table 5).

      4. Discussion

      This study found that documented use of standardized screening tools for CDF was generally low, but depression screenings occurred at a relatively higher rate than those for cognitive impairment and fatigue (24% compared with 0% and 2%, respectively). The prevalence of depression based on results of screenings performed using standardized tools was 2%, with no positive screening results for cognitive impairment or fatigue. These findings yield an incomplete picture, because they exclude subjective data from clinical assessments of CDF performed by providers during clinical encounters; in interviews, providers estimated that at least half of their MS patients have some combination of CDF symptoms, most commonly depression and fatigue, and “almost all” have at least 1 symptom.
      These findings align with a related study of EHR-documented use of standardized cognitive impairment and depression screening tools in MS patients, which showed low rates of documented use of screening tools (2% and 3%, respectively) despite providers’ higher estimates of these conditions based on informal clinical assessments (36% and 44%, respectively) (

      Buckle, G.J., Robertson, D., Moreo, N., Aungst, A., Carter, J., Spair, T., 2018. Screening for cognitive impairment and depression in MS patients: current practices, perceptions, and barriers in selected United States MS clinics. https://actrims.confex.com/actrims/2018/meetingapp.cgi/Paper/2414, (accessed March 23, 2021).

      ). These findings indicate that although standardized CDF screening has not been integrated into routine care, the prevalence of CDF in these populations based on informal clinical assessments is in line with that seen in populations studied in epidemiologic studies using standardized tools and self-reported history of diagnosis. In contrast, a study conducted in a Canadian center that reviewed medical records to determine the proportion of patients presenting with cognitive, mental health, and psychosocial concerns revealed low documentation of such signs and symptoms (
      • Walker L.A.S.
      • Gardner C.
      • Freedman M.S.
      • MacLean H.
      • Rush C.
      • Bowman M.
      Research-to-practice gaps in multiple sclerosis care for patients with subjective cognitive, mental health, and psychosocial concerns in a Canadian center.
      ). Having found, like Buckle et al. and this present research, little evidence of the use of standardized assessments for CDF, the authors state that patients at the center were likely to be experiencing symptoms of CDF at expected epidemiological rates, but that their CDF was not identified (
      • Walker L.A.S.
      • Gardner C.
      • Freedman M.S.
      • MacLean H.
      • Rush C.
      • Bowman M.
      Research-to-practice gaps in multiple sclerosis care for patients with subjective cognitive, mental health, and psychosocial concerns in a Canadian center.
      ). A study conducted in the UK with MS “stakeholders,” including patients and clinicians, also found that routine assessment of cognitive impairment was low, despite stakeholder consensus that the prevalence of cognitive impairment was high and that the impact of cognitive impairment on patient quality of life was significant (
      • Elwick H.
      • Smith L.
      • Mhizha-Murira J.R.
      • Topcu G.
      • Leighton P.
      • Drummond A.
      • Evangelou N.
      • das Nair R.
      Cognitive assessment in multiple sclerosis clinical care: a qualitative evaluation of stakeholder perceptions and preferences.
      ).
      All 6 providers interviewed agreed that the overlapping signs and symptoms of CDF are not only challenging to distinguish from one another, but also difficult to identify as primary or secondary symptoms of MS. This study identified several barriers that nevertheless explain the low rates of standardized CDF screenings in the sample. Providers cited lack of support staff to assist with screenings, workflow challenges, inadequate tool sensitivity (particularly for cognitive impairment and fatigue), and inability to address and follow-up with effective treatment on identified issues as key barriers to routine screening. Several of these barriers were also identified by
      • Elwick H.
      • Smith L.
      • Mhizha-Murira J.R.
      • Topcu G.
      • Leighton P.
      • Drummond A.
      • Evangelou N.
      • das Nair R.
      Cognitive assessment in multiple sclerosis clinical care: a qualitative evaluation of stakeholder perceptions and preferences.
      ).
      Of note, the center began expanding use of the SDMT in 2020; however, these efforts to increase employment of the paper-based measure were impeded by COVID-19 and the subsequent move to telehealth. Additional studies could examine the effects of expanding use of SDMT on rates of standardized screening for cognitive impairment (once fully implemented) to determine if they reflect trends similar to those in the use of standardized screening for depression, which likely increased following adoption of practice-wide standards.
      Provider reports suggest a systematic approach to treating CDF symptoms, including identifying lifestyle interventions, reviewing existing medications, and prescribing specific first- and second-line medications. Three providers cited a lower rate of treatment for cognitive impairment, based on the perception that useful treatments for isolated cognitive issues are limited, and noted a desire to manage concurrent depression and fatigue first to determine if cognitive concerns improve as a result. One NP expressed a low level of comfort with prescribing new medications without a psychiatric evaluation. These reported treatment approaches were reflected in the data, which showed a much higher rate of newly recommended nonpharmacologic than pharmacologic treatments and a significantly higher rate of pharmacologic treatments prescribed by MDs than by NPs.
      Chart-abstracted treatment rates show that providers document evidence-based treatment recommendations associated with CDF in their patient charts more frequently than they perform standardized CDF screenings. All providers acknowledged use of subjective assessments to evaluate CDF; it is likely that subjective assessments are informing treatment decisions. The types of subjective assessments vary, and providers inconsistently document the source and level of information guiding the documentation of CDF symptoms (e.g., whether patient-volunteered or provider-observed symptoms, the extent to which a symptom was indicative of CDF), which limits deeper conclusions based on the subjective evaluations. Future research may facilitate standardized recording of provider-recognized CDF symptoms based on subjective assessments and enable analysis of subsequent results and treatment patterns.
      Notably, provider decisions to perform routine screening with a standardized tool do not seem to be influenced by MS guidelines. Instead, providers reported that their care patterns are based on their judgment and fellowship training, citing complex needs of MS patients that are not easily standardized (i.e., providers manage a constellation of symptoms, medications, and adverse effects). Providers noted that longstanding relationships enable them to discern changes in a patient's neurocognitive functioning before they are detectable with standardized scales (e.g., trouble performing work functions, appearing disheveled, missing appointments), which highlights a gap between screening guidelines and clinical practice.
      Provider behavior appears to be influenced by institutional culture and priorities, such as introducing a practice-level initiative prioritizing annual PHQ screenings. Practice initiatives to support routine screening for cognitive impairment and fatigue may further advance the identification and management of these symptoms. Additionally, providers could benefit from guidance on appropriate treatments based on screening results. Providers noted that routine use of the paper-based SDMT was hindered by the move to telehealth during the COVID-19 pandemic; identification of validated screening tools appropriate for remote use could facilitate greater overall adoption if virtual care continues in the future.

      4.1 Strengths and limitations

      Strengths of this study include the mixed-methods approach, which enabled comparison of provider-reported practices and barriers to data reflected in the EHR and strict methods for training and reassessment to support inter-rater reliability in the capture of chart-abstracted data.
      While this study provides rich qualitative and quantitative data on the patterns of screening for and treating CDF in PwMS, there were some limitations. Although the chart abstractors demonstrated a high degree of agreement during reliability testing, human error and variation in provider documentation practices could have resulted in unintentional missing, incomplete, or invalid data. Electronic charts were reviewed for coded and free-text responses, but the CDF recognition rate was based on only standardized screenings to mitigate inconsistencies in how and to what extent providers acknowledged CDF in the EHRs.
      Finally, the study was conducted within 1 medical system in the United States. More research is needed to determine whether findings are generalizable and whether the suggested facilitators could overcome the common barriers to screening. Future studies could explore whether provision of CDF-focused screening tools that include clinical decision support, treatment selection pathways, and/or goal-setting, as well as data that are integrated into the EHR to allow tracking over time, would bolster standardized screening and recognition of CDF among MS patients.

      5. Conclusion

      The prevalence of CDF may be underrepresented in this study owing to the complex and overlapping set of symptoms that are difficult to diagnose and differentiate. Providers recognize the importance of CDF to patients—especially as physical symptoms are better managed—and frequently document subjective notations of symptoms in patient charts. However, standardized screening tools are used infrequently because of the lack of support staff and perceptions of limited treatment options. Efforts to build standardized screening into institutional cultural practices, as observed with routine use of the PHQ, may offer a solution to increase provider adoption of these tools.
      Providers point to clear facilitators for increasing routine screening practices, including expanded access to support staff to conduct screenings, availability of electronic versions of paper-based assessments, and ability to track and easily access previous assessment scores within the EHR. Routine symptom tracking could also offer providers a more comprehensive and longitudinal picture of patients’ symptom progression and trigger providers to use a standardized screening tool. This could include expanded use of previsit or postvisit collection of patient-reported outcomes and in-office use of tablets or similar tools to provide self-report of CDF symptoms. The use of standardized tools provides objective and measurable input on important changes in MS patients’ quality of life and objective functioning.

      Author Contributions

      Joseph Lynch had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
      Carlo Tornatore, Alexis Ahmad, Timothy Pham, Komal Gupte-Singh, Naila Wahid, Joseph Lynch, Kelsey Jones Pratt: Study conceptualization and design
      Carlo Tornatore, Alexis Ahmad, Timothy Pham, Komal-Gupte Singh, Naila Wahid, Joseph Lynch, Kelsey Jones Pratt: Data acquisition, analysis, or interpretation
      Naila Wahid, Kelsey Jones Pratt: Roles/Writing – original draft
      Carlo Tornatore, Alexis Ahmad, Komal-Gupte Singh, Timothy Pham: Writing – review & editing
      Joseph Lynch, Naila Wahid: Statistical analysis
      Carlo Tornatore, Komal Gupte-Singh, Kelsey Jones Pratt: Supervision

      Declaration of Competing Interest

      Carlo Tornatore provided research, consultation, and speaker services to Biogen, Serono, Mapi, Celgene, and Genentech.
      Alexis Ahmad was an employee of Georgetown University at the time of study completion and is currently an employee of Biogen Inc.
      Naila Wahid and Joseph Lynch are currently employees of Avalere Health; Avalere received support from Bristol Myers Squibb in a collaborative partnership in the conduct of the study and development of the manuscript.
      Kelsey Jones Pratt was an employee of Avalere Health at the time of the study.
      Timothy Pham was an employee of Bristol Myers Squibb at the time of the study and may be a shareholder in the company.
      Komal Gupte-Singh is currently an employee of Bristol Myers Squibb and may be shareholders in the company.

      Funding/Support

      This study was supported by Bristol Myers Squibb (Princeton, NJ). Project management and analytic support for this manuscript provided by Avalere Health, an Inovalon company (Washington, DC), were funded by Bristol Myers Squibb. Editorial support for this manuscript provided by Samantha Rivera, MS, and Eleanor Bush, MA, of Peloton Advantage, LLC (Parsippany, NJ), an OPEN Health company, was funded by Bristol Myers Squibb.

      Role of the funding source/sponsor

      The study sponsor had a role in the design and conduct of the study; interpretation of the aggregated data; preparation, review, and approval of the manuscript; and decision to submit the manuscript for publication.

      Acknowledgments

      The authors thank the following individuals for their work on the research and development of this article: Kristi Mitchell, Penelope Solis, and Oladeji Faborode from Avalere Health assisted in the research, data analysis, and preparation of the manuscript; Francis Lobo of Bristol Myers Squibb contributed to the concept and design of the study; and Luke Lovelace of the Georgetown University Multiple Sclerosis and Neuroimmunology Center served as a medical chart abstractor.

      Appendix. Supplementary materials

      References

      1. Bever, C., Rae-Grant, A., for the American Academy of Neurology Multiple Sclerosis Work Group Members, 2014. Multiple sclerosis quality measurement set. https://www.aan.com/siteassets/home-page/policy-and-guidelines/quality/quality-measures/14msmeasureset_pg.pdf, (accessed March 22, 2021).

        • Bradshaw J.
        Cognition, depression and fatigue in multiple sclerosis.
        Adv. Clin. Neurosci. Rehabil. 2008; 8: 15-17
      2. Buckle, G.J., Robertson, D., Moreo, N., Aungst, A., Carter, J., Spair, T., 2018. Screening for cognitive impairment and depression in MS patients: current practices, perceptions, and barriers in selected United States MS clinics. https://actrims.confex.com/actrims/2018/meetingapp.cgi/Paper/2414, (accessed March 23, 2021).

      3. Costello, K., Kalb, R., for the 2019 Revision-Writing Team, 2019. The use of disease-modifying therapies in multiple sclerosis. Principles and current evidence [a consensus paper]. https://www.nationalmssociety.org/NationalMSSociety/media/MSNationalFiles/Brochures/DMT_Consensus_MS_Coalition.pdf, (accessed March 23, 2021).

        • Elwick H.
        • Smith L.
        • Mhizha-Murira J.R.
        • Topcu G.
        • Leighton P.
        • Drummond A.
        • Evangelou N.
        • das Nair R.
        Cognitive assessment in multiple sclerosis clinical care: a qualitative evaluation of stakeholder perceptions and preferences.
        Neuropsychol. Rehabil. 2021; 2021: 1-19https://doi.org/10.1080/09602011.2021.1899942
        • Geerlings M.I.
        • Jonker C.
        • Bouter L.M.
        • Adèr H.J.
        • Schmand B.
        Association between memory complaints and incident Alzheimer's disease in elderly people with normal baseline cognition.
        Am. J. Psychiatry. 1999; 156: 531-537https://doi.org/10.1176/ajp.156.4.531
        • Hughes A.J.
        • Bhattarai J.J.
        • Paul S.
        • Beier M.
        Depressive symptoms and fatigue as predictors of objective-subjective discrepancies in cognitive function in multiple sclerosis.
        Mult. Scler. Relat. Disord. 2019; 30: 192-197https://doi.org/10.1016/j.msard.2019.01.055
        • Kalb R.
        • Beier M.
        • Benedict R.H.
        • Charvet L.
        • Costello K.
        • Feinstein A.
        • Gingold J.
        • Goverover Y.
        • Halper J.
        • Harris C.
        • Kostich L.
        • Krupp L.
        • Lathi E.
        • LaRocca N.
        • Thrower B.
        • DeLuca J.
        Recommendations for cognitive screening and management in multiple sclerosis care.
        Mult. Scler. 2018; 24: 1665-1680https://doi.org/10.1177/1352458518803785
        • Kletenik I.
        • Alvarez E.
        • Honce J.M.
        • Valdez B.
        • Vollmer T.L.
        • Medina L.D.
        Subjective cognitive concern in multiple sclerosis is associated with reduced thalamic and cortical gray matter volumes.
        Mult. Scler. J. 2019; 52055217319827618https://doi.org/10.1177/2055217319827618
        • Macías Islas M.
        • Ciampi E.
        Assessment and impact of cognitive impairment in multiple sclerosis: an overview.
        Biomedicines. 2019; 7: 22https://doi.org/10.3390/biomedicines7010022
        • Marrie R.A.
        • Horwitz R.
        • Cutter G.
        • Tyry T.
        • Campagnolo D.
        • Vollmer T.
        The burden of mental comorbidity in multiple sclerosis: frequent, underdiagnosed, and undertreated.
        Mult. Scler. 2009; 15: 385-392https://doi.org/10.1177/1352458508099477
        • Marrie R.A.
        • Patten S.B.
        • Berrigan L.I.
        • Tremlett H.
        • Wolfson C.
        • Warren S.
        • Leung S.
        • Fiest K.M.
        • McKay K.A.
        • Fisk J.D.
        • CIHR Team in the Epidemiology and Impact of Comorbidity on Multiple, S.E.
        Diagnoses of depression and anxiety versus current symptoms and quality of life in multiple sclerosis.
        Int. J. MS Care. 2018; 20: 76-84https://doi.org/10.7224/1537-2073.2016-110
        • McGuigan C.
        • Hutchinson M.
        Unrecognised symptoms of depression in a community-based population with multiple sclerosis.
        J. Neurol. 2006; 253: 219-223https://doi.org/10.1007/s00415-005-0963-0
        • McNicholas N.
        • O'Connell K.
        • Yap S.M.
        Cognitive dysfunction in early multiple sclerosis: a review.
        QJM. 2018; 111: 359-364https://doi.org/10.1093/qjmed/hcx070
      4. Minden, S., Turner, A., Kalb, R., Burke, D., 2014. Emotional disorders in multiple sclerosis clinical bulletin; information for health professionals. http://www.nationalmssociety.org/NationalMSSociety/media/MSNationalFiles/Brochures/Clinical-Bulletin-Emotional-Disorders-5-5-14.pdf, (accessed May 3, 2021).

        • Rae-Grant A.
        • Bennett A.
        • Sanders A.E.
        • Phipps M.
        • Cheng E.
        • Bever C.
        Quality improvement in neurology: multiple sclerosis quality measures: executive summary.
        Neurology. 2015; 85: 1904-1908https://doi.org/10.1212/WNL.0000000000001965
        • Romero K.
        • Shammi P.
        • Feinstein A.
        Neurologists׳ accuracy in predicting cognitive impairment in multiple sclerosis.
        Mult. Scler. Relat. Disord. 2015; 4: 291-295https://doi.org/10.1016/j.msard.2015.05.009
        • Skokou M.
        • Soubasi E.
        • Gourzis P.
        Depression in multiple sclerosis: a review of assessment and treatment approaches in adult and pediatric populations.
        ISRN Neurol. 2012; 2012427102https://doi.org/10.5402/2012/427102
        • Stepleman L.M.
        • Penwell-Waines L.M.
        • Rollock M.
        • Casillas R.S.
        • Brands T.
        • Campbell J.
        • Ange B.
        • Waller J.L.
        Routine depression screening in an MS clinic and its association with provider treatment recommendations and related treatment outcome.
        J. Clin. Psychol. Med. Settings. 2014; 21: 347-355https://doi.org/10.1007/s10880-014-9409-0
        • Walker L.A.S.
        • Gardner C.
        • Freedman M.S.
        • MacLean H.
        • Rush C.
        • Bowman M.
        Research-to-practice gaps in multiple sclerosis care for patients with subjective cognitive, mental health, and psychosocial concerns in a Canadian center.
        Int. J. MS Care. 2019; 21: 243-248https://doi.org/10.7224/1537-2073.2017-090