Highlights
- •The value of the relative risk reduction depends on the event rate of placebo.
- •A high relative risk reduction in a low placebo event rate means little clinically.
- •A moderate relative risk reduction in a high placebo event rate is more significant.
- •Numbers needed to treat and absolute risk reductions clarify the above.
- •Cross-comparison between trials with different background risks is misleading.
Abstract
A reduction in relapse rate is the main primary outcome in most clinical trials in patients with multiple sclerosis (MS), with the effect of a treatment commonly expressed as relative risk reduction for this outcome. Physicians often assume that a drug with a higher relative risk reduction demonstrated in one trial is more effective than a drug with a lower relative risk reduction in another, and may pass this idea on to younger physicians and to patients. The use of the relative risk reduction as a measure of drug efficacy can be misleading, as it depends on the nature of the population studied: a treatment effect characterized by a lower relative risk reduction may be more clinically meaningful than one with a higher relative risk reduction. This concept is especially important with regard to clinical trials in patients with MS, where relapse rates in placebo groups have been declining in recent decades. Direct, head-to-head comparisons are the only way to compare the efficacy of the different treatments for MS.
Abbreviations:
BID (twice-daily administration), BEYOND (trial) (BEYOND: Betaferon/Betaseron Efficacy Yielding Outcomes of a New Dose in Multiple Sclerosis (MS) Patients), CARE MS I (trial) (Comparison of Alemtuzumab and Rebif® Efficacy in Multiple Sclerosis), DMT (disease-modifying therapy), EVIDENCE (trial) (EVidence of Interferon Dose‐response: European North American Comparative Efficacy), INCOMIN (trial) (Independent Comparison of Interferon), RCT (randomised, controlled trial), REGARD (trial) (REbif vs Glatiramer Acetate in Relapsing MS Disease), TENERE (trial) (A Study Comparing the Effectiveness and Safety of Teriflunomide and Interferon Beta‐1a in Patients With Relapsing Multiple Sclerosis), TRANSFORMS (trial) (Efficacy and Safety of Fingolimod in Patients With Relapsing‐remitting Multiple Sclerosis With Optional Extension Phase)Keywords
1. Introduction
Randomized, controlled trials (RCTs) represent the highest level of evidence-based medicine. Appropriate interpretation of the results of these trials is essential to guide the correct application of therapies in everyday clinical practice. Recent years have seen the introduction of novel, oral disease-modifying therapies (DMT) into the management of multiple sclerosis, based on the results of RCTs comparing them directly with interferon-β, the current standard of care for this condition.
Comparisons of annual relapse rates are key outcomes of most RCTs in this area. Relative risk reductions are often used to express differences between treatments in this and other efficacy parameters and, importantly, relative risk reductions are widely used in marketing activities by their pharmaceutical sponsors. It is assumed widely that a higher relative risk reduction means greater clinical efficacy: a dangerous assumption in any field of medicine and especially so in multiple sclerosis, where the nature of the patient population has changed over time. This misconception remains prevalent among healthcare professionals caring for patients with MS and in this article I will discuss the dangers inherent in over reliance on the relative risk reduction as a measure of clinical efficacy, and how we should compare the effectiveness of different DMTs.
2. Absolute risk, relative risk and number needed to treat
Table 1 provides mathematical definitions of absolute risk reduction, relative risk reduction and number needed to treat, while Table 2 shows how these parameters relate to each other, based on a hypothetical evaluation of a new DMT (
Spitalnic, 2005
, Cutforth,
). Intervention in patients at low risk of relapse (Trial 1 in Table 2) provides a much lower absolute risk reduction compared with patients at intermediate (Trial 2) or high risk (Trial 3). However, the relative risk reduction is the same for the intermediate- and low-risk populations, and higher than that for the high-risk population. Numbers needed to treat are consistent with these observations: we would need to treat 50 low-risk patients to prevent a single relapse, but only 3–5 of the higher-risk patient groups.Cutforth R RLO: relative risk reduction (RRR) and absolute risk reduction (ARR). Available at 〈http://www.nottingham.ac.uk/nmp/sonet/rlos/ebp/risk_reduction/index.html〉, 2015 [accessed January 2015].
Table 1Mathematical definitions of absolute risk reduction, relative risk reduction and number needed to treat to prevent one event.
| Parameter | Definition |
|---|---|
| Event rate | Number of patients with the event/total number of patients in the group |
| Absolute risk reduction | Control event rate–intervention event rate |
| Relative risk reduction (%) | 100×(absolute risk reduction [%])/(control event rate [%]) |
| Number needed to treat | 1/absolute risk reduction (if the event rate is expressed as a ratio) |
| 100/absolute risk reduction (event rate expressed as a percentage) |
a Expressed as a ratio or percentage value.
b The absolute risk reduction will be expressed in the same units as the event rates on which it was based.
c The relative risk reduction is often shown shown as a percentage (as shown) but can also be shown as a ratio (omit the “100×”).
Table 2Absolute risk reductions versus relative risk reductions for comparing changes in annual relapse rate in different hypothetical clinical trial settings.
| Annual relapse rate (%) | Absolute risk reduction (%) | Relative risk reduction (%) | Number needed to treat | |
|---|---|---|---|---|
| Control | New DMT | |||
| Trial 1: Low background event rate: | ||||
| 4 | 2 | 2 | 50 | 50 |
| Trial 2: Intermediate background event rate: | ||||
| 40 | 20 | 20 | 50 | 5 |
| Trial 3: High background event rate: | ||||
| 90 | 60 | 30 | 33 | 3 |
See Table 1 for definitions and calculation of terms shown. The absolute risk reduction is expressed as a per cent value in this example, but will generally be expressed in the units of the event rates used to calculate it.
Thus, the relative risk reduction tells us little about the absolute clinical benefit delivered by the new treatment, as a high value may result from a clinically insignificant change in the event rate in a population at low background risk. It is meaningless to compare relative risk reductions between trials where populations may have a different background risk of relapse. The absolute risk reduction and number needed to treat, by contrast, provide important additional information on the actual magnitude of clinical benefit that a treatment provides. Unfortunately, these parameters are not reported routinely in reports of clinical trials. Incorporating the absolute risk reduction (with the number needed to treat) would lead to more appropriate treatment decisions than those based on consideration of the relative risk reduction alone.
3. Special relevance to clinical trials in patients with MS
The apparent background severity of relapsing-remitting MS, as indicated by the relapse rate, has been declining in clinical trial populations since the pivotal trials that established interferon preparations as the standard of care for pharmacologic intervention in this disease (
Klawiter et al., 2009
); this has resulted in a clear decline in the relapse rates observed in the placebo groups of clinical trial populations with MS (Table 3). Given the underlying decline in relapse rates over time, it is unsurprising that absolute risk reductions achieved with active treatments relative to placebo have also tended to decline over this period (Table 3). Relative risk reductions vs. placebo for active treatments, by contrast, have increased in later trials (Table 3). The number needed to treat has also increased, consistent with the tendency to a reduction in absolute risk (Table 3).Table 3Temporal trends in placebo-controlled trials in MS from 1993 to 2012, demonstrating decreasing absolute risk reductions and increasing numbers-needed-to-treat despite increasing relative risk reductions.
| Trial with year of publication | Annualized relapse rate for placebo | Relative risk reduction (%) | Absolute risk reduction | Number needed to treat | |
| IFN-β1b Sc (Pivotal Trial, 1993) | 1.27 | 34 | 0.43 | 2.3 | |
| Glatiramer acetate (Pivotal Trial, 1995) | 0.84 | 29 | 0.24 | 4.2 | |
| IFN-β1a IM (MSCRG trial, 1996) | 0.90 | 32 | 0.29 | 3.4 | |
| IFN-β1a Sc (PRISMS Trial, 1998) | 1.28 | 33 | 0.42 | 2.4 | |
| Natalizumab (AFFIRM Trial, 2006) | 0.78 | 68 | 0.50 | 2.0 | |
| Fingolimod (FREEDOMS trial, 2010) | 0.40 | 54 | 0.22 | 4.5 | |
| Cladribine (CLARITY trial, 2010) | 0.33 | 58 | 0.19 | 5.3 | |
| Teriflunomide (TEMSO trial, 2011) | 0.54 | 31.5 | 0.17 | 5.9 | |
| CONFIRM trial, 2012 | 0.4 | BG-12 bid | 44 | 0.18 | 5.6 |
| BG12 tid | 51 | 0.20 | 5.0 | ||
| Glatiramer acetate | 29 | 0.11 | 9.1 | ||
| BG–12 (DEFINE trial, 2012) | 0.36 | 53 | 0.19 | 5.3 | |
These parameters are shown as in original reports where quoted; otherwise, they have been estimated from data presented. References for individual trials:
a
IFNB Multiple Sclerosis Study Group, 1993
.b
Johnson et al., 1995
.- Johnson K.P.
- Brooks B.R.
- Cohen J.A.
- Ford C.C.
- Goldstein J.
- Lisak R.P.
- et al.
Copolymer 1 reduces relapse rate and improves disability in relapsing-remitting multiple sclerosis: results of a phase III multicenter, double-blind placebo-controlled trial. The copolymer 1 multiple sclerosis study group.
Neurology. 1995; 45: 1268-1276
c
Jacobs et al., 1996
.d
PRISMS (Prevention of Relapses and Disability by Interferon beta-1a Subcutaneously in Multiple Sclerosis) Study Group, 1998
.e
Polman et al., 2006
.f
Kappos et al., 2010
.g
Giovannoni et al., 2010
.h
O׳Connor et al., 2011
.i
Fox et al., 2012
.j
Gold et al., 2012
; see also Klawiter et al., 2009
.It is important to note that these observations do not necessarily indicate differences in therapeutic efficacy between agents studied in older trials (interferons and glatiramer acetate) and those studied in more recent trials (BG-12, natalizumab, cladribine, teriflunomide and fingolimod), as these numbers may differ for the same DMT in different trial settings. The CONFIRM and DEFINE studies illustrate the potential pitfalls in using relative risk reductions to compare treatments where event rates are low (
Fox et al., 2012
, Gold et al., 2012
). The relative risk reduction for BG-12 in DEFINE was an impressive 53% vs. placebo (Gold et al., 2012
). In CONFIRM, however, although no prespecified statistical comparison was performed between the two BG-12 arms and the glatiramer acetate arms of this trial, a post-hoc comparison demonstrated no significant difference between the approved dose of BG-12 (240 mg BID) and glatiramer acetate, despite an apparently far superior relative risk reduction of 44% vs. 29%, respectively (Fox et al., 2012
).More randomized, head-to-head comparisons of DMTs in patients with MS are required, as these are the only reliable way to evaluate the comparative efficacy of treatments. Such trials have already demonstrated statistically and clinically significant benefits between different interferon regimens (EVIDENCE (
Panitch et al., 2002
) and INCOMIN (Durelli et al., 2002
) trials); for s.c interferonβ1a vs. teriflunomide 7 mg (but with no difference vs. the 14 mg dose; the TENERE trial (Vermersch et al., 2014
)); for alemtuzumab vs. s.c interferonβ1a (the CARE MS I trial (Cohen et al., 2012
)); and for fingolimod vs. i.m. interferon-β1a (TRANSFORMS trial (Cohen et al., 2010
; Khatri et al., 2011
)). In addition, the BEYOND trial demonstrated similar efficacy between interferon-β1b (s.c. doses of 250 μg or 500 μg) and glatiramer acetate (O׳Connor et al., 2009
) and the REGARD trial demonstrated similar efficacy between s.c. interferonβ1a and glatiramer acetate (Mikol et al., 2008
).- Mikol D.D.
- Barkhof F.
- Chang P.
- Coyle P.K.
- Jeffery D.R.
- Schwid S.R.
- et al.
Comparison of subcutaneous interferon beta-1a with glatiramer acetate in patients with relapsing multiple sclerosis (the REbif vs Glatiramer Acetate in Relapsing MS Disease [REGARD] study): a multicentre, randomised, parallel, open-label trial.
Lancet Neurol. 2008; 7: 903-914
These studies provide only part of the puzzle, although such studies are expensive to conduct and may be limited by regulatory issues for some drug comparisons. In particular, pharmaceutical sponsors of a new drug may be reluctant to undertake an expensive trial comparing a candidate agent with a potentially equivalent comparator, since regulatory bodies require demonstration of superior efficacy over a comparator in a pivotal study. Companies may be reluctant to increase the risk of such a study failing to meet its primary outcome. Once the new drug is on the market, there is risk – but little incentive – for a pharmaceutical sponsor to incur the expense of a randomized controlled trial comparing their drug to another which might not result in a benefit for their product (see, e.g. the REGARD and BEYOND studies, above).
More such trials are needed for us to understand the relative benefits and harms of different treatments for MS.
4. Conclusions
The magnitude of the relative risk reduction for the relapse rate with a DMT in a patient with MS depends on the background risk of relapse. Thus, low relapse rates in the control group may yield a high relative risk reduction arising from a low and clinically insignificant reduction in absolute risk, as reflected by a high number needed to treat. Conversely, a modest relative risk reduction may be clinically meaningful where the background relapse rate of the control group is high: this situation provides scope for a more substantial reduction in the absolute risk reduction, with a consequently lower number needed to treat.
Each trial design and patient population has its own specific characteristics and cross comparison between different trials for relative risk reductions (especially), or even for absolute risk reductions or numbers needed to treat is misleading, as any measure of the risk of any given outcome will vary with the characteristics of the trial. Importantly, these indices may vary for the same DMT in different trial settings. Generalizing results between trials is not an appropriate manner in which to apply evidence-based medicine.
Conflict of interest statement
The author declares no conflict of interest with regard to the preparation of this article.
Acknowledgements
A medical writer (Dr Mike Gwilt, GT Communications) edited material provided by the author for English and style (no payment was made for this editorial assistance).
References
- Oral fingolimod or intramuscular interferon for relapsing multiple sclerosis.N Engl J Med. 2010; 362: 402-415
- Alemtuzumab versus interferon beta 1a as first-line treatment for patients with relapsing-remitting multiple sclerosis: a randomised controlled phase 3 trial.Lancet. 2012; 380: 1819-1828
Cutforth R RLO: relative risk reduction (RRR) and absolute risk reduction (ARR). Available at 〈http://www.nottingham.ac.uk/nmp/sonet/rlos/ebp/risk_reduction/index.html〉, 2015 [accessed January 2015].
- Every-other-day interferon beta-1b versus once-weekly interferon beta-1a for multiple sclerosis: results of a 2-year prospective randomised multicentre study (INCOMIN).Lancet. 2002; 359: 1453-1460
- Placebo-controlled phase 3 study of oral BG-12 or glatiramer in multiple sclerosis.N Engl J Med. 2012; 367: 1087-1097
- A placebo-controlled trial of oral cladribine for relapsing multiple sclerosis.N Engl J Med. 2010; 362: 416-426
- Placebo-controlled phase 3 study of oral BG-12 for relapsing multiple sclerosis.N Engl J Med. 2012; 367: 1098-1107
- Interferon beta-1b is effective in relapsing-remitting multiple sclerosis. I. Clinical results of a multicenter, randomized, double-blind, placebo-controlled trial.Neurology. 1993; 43: 655-661
- Intramuscular interferon beta-1a for disease progression in relapsing multiple sclerosis. The Multiple Sclerosis Collaborative Research Group (MSCRG).Ann Neurol. 1996; 39: 285-294
- Copolymer 1 reduces relapse rate and improves disability in relapsing-remitting multiple sclerosis: results of a phase III multicenter, double-blind placebo-controlled trial. The copolymer 1 multiple sclerosis study group.Neurology. 1995; 45: 1268-1276
- A placebo-controlled trial of oral fingolimod in relapsing multiple sclerosis.N Engl J Med. 2010; 362: 387-401
- Comparison of fingolimod with interferon beta-1a in relapsing-remitting multiple sclerosis: a randomised extension of the TRANSFORMS study.Lancet Neurol. 2011; 10: 520-529
- The present efficacy of multiple sclerosis therapeutics: is the new 66% just the old 33%?.Neurology. 2009; 73: 984-990
- Comparison of subcutaneous interferon beta-1a with glatiramer acetate in patients with relapsing multiple sclerosis (the REbif vs Glatiramer Acetate in Relapsing MS Disease [REGARD] study): a multicentre, randomised, parallel, open-label trial.Lancet Neurol. 2008; 7: 903-914
- 250 microg or 500 microg interferon beta-1b versus 20 mg glatiramer acetate in relapsing–remitting multiple sclerosis: a prospective, randomised, multicentre study.Lancet Neurol. 2009; 8: 889-897
- Randomized trial of oral teriflunomide for relapsing multiple sclerosis.N Engl J Med. 2011; 365: 1293-1303
- Randomized, comparative study of interferon beta-1a treatment regimens in MS: the EVIDENCE trial.Neurology. 2002; 59: 1496-1506
- A randomized, placebo-controlled trial of natalizumab for relapsing multiple sclerosis.N Engl J Med. 2006; 354: 899-910
- Randomised double-blind placebo-controlled study of interferon beta-1a in relapsing/remitting multiple sclerosis.Lancet. 1998; 352: 1498-1504
- Risk assessment I: relative risk and absolute risk reduction.Hosp Physician. 2005; 41: 43-46
- Teriflunomide versus subcutaneous interferon beta-1a in patients with relapsing multiple sclerosis: a randomised, controlled phase 3 trial.Mult Scler. 2014; 20: 705-716
Article info
Publication history
Published online: April 10, 2015
Accepted:
March 31,
2015
Received in revised form:
March 29,
2015
Received:
January 29,
2015
Identification
Copyright
© 2015 Elsevier B.V. Published by Elsevier Inc. All rights reserved.
