Methods of pharmaceconomic
evaluation: Cost of Illness, Cost-Minimization Analysis, Cost-Effectiveness Analysis
1 Methods
of pharmaceconomic evaluation
3.1
Cost of Illness
3.1.1
Approaches
3.1.2
Methods
3.2
Atopic Dermatitis
3.2.1
Therapy-Specific Cost
3.2.1.1
Topical Corticosteroids
3.3.2.2
Topical Immunomodulators
3.4
HPV
6.1
Cost-Minimization Analysis
6.1.1
What is Meant by Therapeutic Equivalence?
6.1.2
Optimizing Evidence from Clinical Trials
6.2
Sources of Clinical Trial Evidence
6.2.1
Superiority Trials
6.2.2
Equivalence Trials
6.2.2.1
Characteristics of Equivalence Trials
6.2.2.2
Equivalence Range or Margin
6.2.3
Non-Inferiority Trials
6.2.3.1
Characteristics of a Non-Inferiority Trial
6.2.3.2
Non-Inferiority Range or Margin
6.3
Other Issues To Be Addressed In Evaluating Equivalence
6.3.1
Statistical versus Clinical Significance
6.3.2
Equivalence in Single or Multiple Outcomes?
6.3.3
Whose Views of Clinical Equivalence Should be
Preeminent?
6.3.4
Over What Period Should We Evaluate Clinical Equivalence?
6.4
Effectively Targeting the Use of CMA
7.1
The Rationale for Cost-Effectiveness Analysis
7.2
The Cost-Effectiveness Plane
7.3
Basic Components of a Cost-Effectiveness Analysis
7.3.1
Enumeration of the Options
7.3.2
Perspective of the Analysis
7.3.3
Time Horizon
7.3.4
Scope of the Analysis
7.3.5
Measuring and Valuing Costs
7.3.6
Measuring and Valuing Outcomes
7.3.7
Time Preference
7.3.8
Choice of Analytic Modeling Method
7.3.9
Accounting for Uncertainty
7.4
Calculation of Incremental Cost-Effectiveness Ratios
7.4.1
Dominance and Extended Dominance
7.4.2
Sensitivity Analysis
7.4.3
Interpretation of CEA Results
1 Methods of pharmaceconomic
evaluation
Pharmacoeconomic methods: Economic (Cost consequence, Cost benefit, Cost effectiveness,
Cost minimization, Cost utility), Humanistic (Quality of life, Patient
preferences, Patient satisfaction).
There
are several types of pharmaceconomic evaluation, each
of which is useful in different circumstances:
• Cost-of-illness analyses consider the
costs of a given disease without considering the outcome.
• Cost-minimization analyses compare the
costs of interventions that provide the same outcome, with the ultimate aim of identifying
the cheapest option.
• Cost-effectiveness analyses involve the
comparison of cost per standardized unit of effectiveness for two or more
interventions that provide varying outcomes.
• Cost-utility analyses aim to compare
the cost per quality adjusted life-year for two or more interventions that
provide varying outcomes.
•
Finally, cost-benefit analyses
compare the costs and benefits of two or more interventions that provide
varying outcomes, where outcome is measured in monetary terms.
Cost
minimisation analysis (CMA)
This
involves measuring only costs, usually only to the health service, and is
applicable only where the outcomes are identical and need not be considered
separately. An example would be prescribing a generic preparation instead of
the brand leader (lower cost but same health outcomes).
Cost
effectiveness analysis (CEA)
The
term cost effectiveness is often used loosely to refer to the whole of economic
evaluation, but should properly refer to a particular type of evaluation, in
which the health benefit can be defined and measured in natural units (eg years of life saved, ulcers healed) and the costs are
measured in money. It therefore compares therapies with qualitatively similar
outcomes in a particular therapeutic area. For instance, in severe reflux oesophagitis, we could consider the costs per patient
relieved of symptoms using a proton pump inhibitor compared to those using H2
blockers. CEA is the most commonly applied form of economic analysis in the
literature, and especially in drug therapy. It does not allow comparisons to be
made between two totally different areas of medicine with different outcomes.
The broad form of these evaluations are shown in box
1, and the key measure is the incremental cost effectiveness ratio (ICER).
Cost
utility analysis (CUA):
This
is similar to cost effectiveness in that the costs are measured in money and
there is a defined outcome (box 2). But here the outcome is a unit of utility
(e.g. a QALY). Since this endpoint is not directly dependent on the disease
state, CUA can in theory look at more than one area of medicine, e.g. cost per QALY
of coronary artery bypass grafting versus cost per QALY for erythropoietin in
renal disease. In practice this is not so easy since the QALY is not a well defined fixed unit transferable from study to study. We
should be particularly wary of attempts to draw up league tables of QALYs to
allow comparisons between a range of therapies. The
values in such tables have usually been derived at different times and in different
ways and are not comparable.
Cost
benefit analysis (CBA)
Here,
the benefit is measured as the associated economic benefit of an intervention (eg monetary value of returning a worker to employment
earlier), and hence both costs and benefits are expressed in money. CBA may
ignore many intangible but very important benefits not measurable in money
terms, e.g. relief of anxiety. CBA may also seem to discriminate against those
in whom a return to productive employment is unlikely, eg
the elderly, or the unemployed.
However
the virtue of this analysis is that it may allow comparisons to be made between
very different areas, and not just medical, e.g. cost benefits of expanding
university education (benefits of improved education and hence productivity)
compared to establishing a back pain service (enhancing productivity by
returning patients to work). This approach is not widely used in health
economics, although many economists like it on theoretical grounds and because
it removes some of the “sacred cow” protection which surrounds health care.
They argue that health should be another commodity, and not necessarily valued
more than other possible uses of the resources.
Cost
consequences and other types of evaluation
Other
forms of quasi-health economic evaluation may be seen in the literature but are
not true economic evaluations because they do not weigh costs and benefits in
an incremental manner. In some cases, often where studies consider multiple
outcomes, costs and benefits are presented in a disaggregated form (e.g. health
profiles). These evaluations are frequently referred to as cost consequences
analyses. Burden of disease (also known as cost of illness) studies attempt to
measure the health and resource implications arising to society from a
particular disease.
3.1 Cost of Illness
Cost-of-illness
(COI) analysis measures the economic burden of disease and illness on society.
It is often called burden-of-illness (BOI). The components of a pharmacoeconomic or cost-effectiveness analysis include
costs and consequences. Costs can be divided into direct and indirect costs.
Direct medical costs are those related to providing medical services, such as a
hospital stay, physician fees for outpatient visits, and drug costs (including
the cost of the medication itself and any downstream adverse events that may
arise as a result of drug administration). Direct nonmedical costs are those
related to expenses, such as transportation costs, that are a direct result of
the illness. Direct costs are most frequently included in a COI study, whereas indirect
costs, those associated with changes of individual productivity, are often not included
in a COI study, because they are difficult to obtain. Examples of indirect costs
are lost time from work (absenteeism) and unpaid assistance from a family member.
In addition, intangible costs, such as pain and suffering, may be included in
the analysis. Analyses can be done from one or several perspectives, which will
help in determining the distribution of disease costs across multiple
stakeholders. The societal perspective typically includes indirect, as well as
direct, medical costs because these are costs to society, that is, as
previously mentioned, lost time from work. The payer perspective typically
includes only direct costs.
COI
analyses are used to aid in policy making; resource allocation—that is,
prioritizing resource use for disease treatment and prevention—and as baseline
research from which to determine the potential benefit of new therapies.
3.1.1 Approaches
There
are two approaches to conducting COI analyses, the prevalence-based approach and
the incidence-based approach. The prevalence-based approach considers the cost
of disease within a specified time period. The prevalence-based approach is most
appropriate for diseases or illnesses that are measured within the time period of
analysis and that do not change much over time (e.g., migraine) or acute
diseases (e.g., asthma, eczema).
This
is in contrast to the incidence-based approach, which calculates the lifetime costs
of disease. This approach is most appropriate for chronic diseases, such as
hypertension, or diseases that take a long time to progress, such as diabetes. This
approach considers disease progression and survival probability. The disease is
first defined using existing disease definitions or classification systems,
such as International Classification of Diseases--Ninth Revision (ICD-9-CM)
codes. To accurately capture the disease COI over the appropriate timeframe,
depending on the aforementioned approaches, one must take into consideration
the epidemiology of the disease under study and the demographic profiles of the
typical patient population.
3.1.2 Methods
A
micro-costing method has been used in many studies to examine COI. The direct costs
included in this method typically comprise out-of-pocket expenses for
noninsured items (over-the-counter medications, visits to out-of-plan health
practitioners, laundry/clothing, and specialty items) and co-payments for
prescription medications and clinic visits determined from insurance claims
databases as well as the usual direct cost items previously outlined.
Two
examples of COI studies, atopic dermatitis (AD) and human papillomavirus (HPV), will now be examined.
3.2 Atopic Dermatitis
AD
is a chronic disease that affects the skin of children and adults. It results
in itchy, flaky skin and demonstrates a considerable impact on patient QoL, as well as a substantial monetary burden. Direct and
indirect costs for AD have been measured in various countries and are
substantial from both a patient and a societal perspective. The direct costs
have been reported to range from $71 to $2,559 per patient per year. This
variation in cost is due to differences in study methodology as well as differences
in health care systems of the various countries. Most of the costs of AD consist
of indirect costs associated with time lost from work, lifestyle changes, and non-traditional
or over-the-counter treatments for AD. The financial burden on the health care
system and on society is expected to grow because the prevalence of the disease
is increasing.
Indeed,
studies in the past 7 years, using a prevalence-based approach to calculate COI,
have demonstrated direct costs ranging from US$150 (using the approximate US$
equivalent in 2005) to US$5806 per patient per year, with differences varying
due to different cost-accounting methods. Table 3.1 lists numerous references
in which US$ (or equivalent) per patient COI were calculated.
Typically,
outpatient visits and medications composed the majority of direct costs,
ranging from approximately 62% to >90%.8 The distribution of AD-associated
direct costs from Fivenson and colleagues is shown in
Figure 3.1.8 In those studies that examined indirect costs (e.g., the patient
out-of-pocket costs for co-pays, medications, household items, loss of
productivity) they made up substantial percentages of the total, e.g., 36% or
73%. Several studies showed increasing costs with worsening disease severity in
adults. Using a micro cost-accounting approach, whereby costs of
hospitalizations, consults, drug therapy, treatment procedures, diagnostic
tests, laboratory tests, clinic visits, and urgent care visits were summed, Fivenson, Arnold, and colleagues (Table 3.2) reported an
average annual per patient direct cost ranging from $435 in mild patients to
$3229 in severe patients.
Figure
3.1 Distribution of atopic dermatitis-associated direct costs in a U.S. health
plan.
Indirect
costs also increased by worsening disease severity—by more than twofold to
threefold11 to as much as almost tenfold.8 Similarly, Ehlken
and co-authors showed a greater than twofold increase in total (both direct and
indirect) costs for patients with mild vs. severe disease.
3.2.1 Therapy-Specific Cost
Several
studies have compared the cost of different uses of topical corticosteroids (TCS)
vs. topical immunomodulators (i.e., pimecrolimus and tacrolimus) and
of the topical immunomodulators against each other.
Some of these are detailed below.
3.2.1.1 Topical Corticosteroids
Green
and colleagues undertook a systematic review of 10 randomized controlled trials
(RCTs) in patients with AD.9,10 Their literature
search at the time revealed no published studies of this nature. The authors
noted a wide variation in price and product availability, with the lowest price
being generic hydrocortisone (£0.60 [approximately US$1.09]) to the
highest at that time being mometasone furoate (Elocon) of £4.88
(approximate US$8.80 equivalent).
Six
of the RTC studies favored the once-daily option as the lowest-cost treatment and
four favored a twice-daily option, with successful outcome being defined by overall
response to treatment, relapse or flareup rate,
adverse effects, compliance, tolerability, patient preference measures, and
impact on quality of life. One of the twice-daily-favored studies achieved a
greater benefit (number of successful treatment responders) at a greater cost.
However, it was felt that this greater cost would still likely be very
cost-effective, given the relatively low prices of TCS. The limitations noted
in the review were that of potentially low generalizability due to 80% of the
RCTs’ referring to potent TCS in patients with moderate-to-severe disease, whereas
the majority of patients with AD have mild disease and lack of information on
quantity of product usage.
3.3.2.2 Topical Immunomodulators
Clinical
data show that topical immunomodulators are effective
in AD, yet do not cause the significant adverse effects associated with TCS. Delea and colleagues retrospectively compared 157 pimecrolimus patients with 157 tacrolimus
patients previously receiving TCS in a large claims database of managed care
patients in terms of resource utilization (concomitant medications) and
AD-related follow-up costs. They used propensity matching to control for
differences between the groups in baseline demographic and clinical characteristics
and utilization of AD-related services prior to assessment of disease severity.
Patients in the pimecrolimus group had fewer pharmacy
claims for TCS (mean 1.37 vs. 2.04, P = 0.021); this occurred primarily in the
high-potency topical corticosteroid category. Fewer patients in the pimecrolimus group also received antistaphylococcal
antibiotics during the followup period (16% vs. 27%,
P = 0.014) and total AD-related costs during this time were lower in this group
than in the tacrolimus group (mean $263 vs. $361, P =
0.012).
3.4 HPV
Persistent
infection with cancer-associated HPV (termed oncogenic or high-risk HPV) causes
the majority of squamous cell cervical cancer, the most common type of cervical
cancer, and its histologic precursor lesions, the low-grade cervical dysplasia Cervical
Intraepithelial Neoplasia-1 (CIN1) and the moderate-to-high-grade dysplasia CIN
2/3. Multiple HPV strains cause varying degrees of invasive cervical cancer
(ICC) and its CIN precursors. HPV strains 16 and 18 cause approximately 70% of
all cervical cancers15,16 and CIN3, specifically, and
50% of CIN2 cases. In addition, HPV 16 and 18 cause approximately 35 to 50% of
all CIN1. Low-oncogenic HPV risk types 6 and 11 account for 90% of genital wart
cases. Unfortunately, cytological and histological examinations cannot reliably
distinguish between those patients who will progress from cervical dysplasia to
ICC from those whose dysplasias will regress
spontaneously, the latter being the vast majority of cases. This inability to
definitely ascertain the natural history of HPV infection is one of the primary
reasons for the dilemma with HPV vaccination.
Although
cervical cancer screening programs, such as the use of routine screening via
the Papanicolaou (Pap) cervical smear, have substantially
reduced the incidence and mortality of ICC in developed countries over the past
50 years, there has been a slowing of these declines in recent years due to
poor sensitivity of cervical cytology, anxiety and morbidity of screening
investigations, poor access to and attendance of screening programs, falling
screening coverage, and poor predictive value for adenocarcinoma, an
increasingly common cause of ICC. HPV is the most common sexually transmitted
disease in the United States and virtually 100% of cervical cancer is due to
HPV. HPV is also linked to head and neck cancer in men. There are more than 100
HPV strains (thereby potentially reducing vaccine efficacy for oncogenic
strains not covered by the vaccine); HPV infection is often self-limited. A
mitigating factor for the argument against using the vaccine is the fact that
the cost-effectiveness of screening with Pap smears is reduced (improves) from
USD1 million/QALY if patients continue to be screened annually, as is the common
current recommendation, to USD150,000/QALY if patients
are screened every 3 years, the latter a likely scenario if the vaccine is
used.
Worldwide,
the incidence of cervical cancer is 470,000 new cases and 233,000 deaths per
year; it is the second-leading cause of cancer deaths, with 80% of these cases
observed in developing countries. Women in developing countries are especially vulnerable
as they lack access to both cervical cancer screening and treatment. The
demographics of cervical cancer in the United States show that 9710 new cases of
ICC were expected to be diagnosed in 2006 and about 3700 deaths in women were
expected from ICC. The National Cancer Institute estimates an annual incidence of
new genital HPV infections of 6 million. Quadrivalent
Human Papillomavirus (HPV) Vaccine recombinant (Gardasil®), the vaccine
recently approved for use in the United States and Europe, covers the two major
oncogenic HPV strains (16 and 18) for cervical cancer. In addition, it covers
HPV strains 6 and 11, the primary causes of genital warts. Therefore, the
vaccine does not offer full protection against cervical cancer, because it does
not protect against HPV strains 31 and 45, which are also implicated in ICC and
cervical dysplasia. To significantly reduce the rate of cervical cancer in the
population as a whole, 70% of girls need to be vaccinated to achieve what is
called “herd immunity”—when the vaccine’s impact goes beyond just people who
are inoculated. So far, it is unknown if HPV strains will mutate as the vaccine
is introduced, although this is not very likely, seeing that HPV is a DNAbased virus.
Insinga and
colleagues used administrative and laboratory data from a large U.S. health
plan to examine costs, resource utilization, and annual health plan
expenditures for cervical HPV-related disease. An episode of care was defined
as beginning with a routine cervical smear, that is, one that required no
evidence of follow-up for a previous Pap smear abnormality or ICD-9 diagnosis
of a cervical abnormality during the previous 9 months. If CIN or cancer was
not detected during an episode of care, biopsy results were termed
false-positive. Because the data source was a prepaid health plan without
direct billing for procedures or services, service-specific costs were assigned
from the Medstat Marketscan
database as a proxy for the health plan costs. Because of the small number of
cervical cancer cases in the data set, costs were assigned on an age- and
stage-specific basis using the Surveillance Epidemiology and End Results
Program (SEER; National Cancer Institute; U.S. Department of Health and Human
Services, Bethesda, MD) and an Agency for Healthcare Researchand
Quality evidence report. All cost estimates were converted to 2002 dollars
using the Medical Care component of the Consumer Price Index.
The
authors found that episodes of care after an abnormal routine cervical smear were
$732 on average, compared with $57 for visits with negative results, with a statistically
significant trend toward higher costs with increasing grade of initial cytologic abnormality. False-positive cervical smears cost
$376 annually, while incomplete follow-up was $79. Regardless of age group,
cervical HPV-related disease annual health care costs were $26,415 per 1000
enrollees, with the greatest costs of $51,863 being observed in the 20- to
29-year-old age group. The largest cost contribution was that of routine
screening at 63.4% of total costs (range by age group of 54.1% to 70.8%),
followed by cost of CIN 2/3, then cancer, false-positive smear,CIN
1 and incomplete follow-up (see Figure 3.2).
Figure
3.2 Distribution of cervical HPV-related disease direct costs in a commercial
U.S. health plan.
Insinga and
co-authors extrapolated their results to the general U.S. population to derive
a total health care cost for HPV-related disease in 1998 of $3.4 billion, with expenditures
for routine screening accounting for $2.1 billion, false-positive Pap test $300
million, CIN 1 $150 million, CIN 2/3 $450 million, and IC $350 million in 2002
dollars. A follow-up study by the same authors estimated the annual direct costs
of abnormal cervical findings and treating cancer at $3.5 billion in 2005 US$. Annual
direct cost estimates in 2005 US dollars have been as high as $4.6 billion and
adding in costs of anogenital warts and other cancers
associated with oncogenic HPV strains raises the total estimated economic
burden to as high as US$5 billion in 2006 US$.
Insinga and
colleagues also estimated indirect costs, assuming that there were 130,377
women who would have been alive during 2000 had they not died from cervical cancer
during that or a previous year, >75% of these women died before age 60, with
>25% dying prior to age 40, and that 37,594 (29%) of these women would have had
labor force earnings during 2000. Using these data, the total productivity loss
in 2000 owing to cervical cancer mortality was estimated at $1.3 billion,
several times higher than recent estimates of the annual U.S. direct medical
costs of US$300 to $400 million associated with cervical cancer. As in the AD
studies, therefore, indirect costs are thought to account for a much greater
burden than direct costs of HPV.
In
summary, COI or BOI lays the foundation on which to frame the different types of
analyses (see Chapters 4 through 9) that are used to make decisions in
allocation of healthcare resources. As indirect costs, that is, productivity,
often account for a substantial portion of the burden, these should be assessed
as part of the COI computation whenever possible.
6.1 Cost-Minimization
Analysis
The
principal issues that are addressed in this chapter are:
1.
The circumstances in which cost-minimization analysis (CMA) is an appropriate methodology
to undertake health economic evaluations.
2.
Steps that can be taken to improve the quality of CMAs and, hence, their reliability
as a basis for healthcare decision making.
The
appropriateness of any economic methodology depends on the nature and quality
of the underlying clinical evidence, with evaluations based on inappropriate or
poor quality clinical data’s failing to provide a reliable basis for health
care decision-making. The primacy of clinical data is particularly evident in
the case of CMA in which, conditional on health benefits between two equivalent
competing options, the least expensive option is preferred. Perhaps as a
consequence of this apparent simplicity, scant attention has been previously
paid to the theoretical and practical methods used to inform the analysis or to
establish the appropriateness of this choice of methodology.
Many
sources of clinical evidence can be used to support economic analyses; however,
the “gold standard” is normally considered to be the randomized controlled trial
(RCT), which holds everything constant with the exception of the drug being evaluated.
Given that, by definition, the results of clinical trials cannot be known in advance,
it is impossible to plan to undertake a CMA alongside an RCT because it is not
certain that the health outcomes being compared will be equivalent (Donaldson, Hundley
et al. 1996). Therefore, no prospective economic evaluation starts out as a CMA;
only when the health outcomes generated are empirically demonstrated to be “identical
or similar” will the CMA be adopted as an appropriate methodology by the health
economist.
CMA
is frequently portrayed as being the “poor relation” among health economic methodologies,
with its apparent simplicity making it unworthy of being considered alongside
more theoretically rigorous health economic methodologies. However, it is
important that health economists recognize and acknowledge that the theoretical
underpinnings of CMA are just as rigorous as those underpinning other methods of
economic evaluation. Perhaps as a consequence of the comparative disdain in which
CMA has been held, its use to date appears to have been poorly conceived and frequently
inappropriate. In this regard, CMA has been frequently employed as an evaluative
tool to support and justify the introduction of cheaper, but potentially less effective,
treatments. The usual procedure is for the analyst to simply assume that the
benefits of a new health technology are equivalent to the existing “gold
standard” therapy without having sufficient evidence to justify such a claim.
For example, by assuming a class effect for similar types of drugs (each drug
in a class having equivalent outcomes) it then becomes possible to base
subsequent analysis solely on a comparison of costs—an attractive strategy if
you are introducing a cheaper but less effective drug.
The
methods currently used to justify equivalence in outcomes in a CMA therefore appear
to be inherently flawed and indicate an urgent need to improve the theoretical rigor
underlying this aspect of CMAs if they are to be taken seriously as a method of
economic evaluation. The current haphazard approach leads to a situation in
which CMA is typically described in health economics textbooks as a form of economic
evaluation where “… the decision simply revolves around the costs” (Gold, Siegel
et al. 1996 p. 165).
This
interpretation ignores the extreme rigor that should be required to ensure equivalence
in health benefits prior to deciding on the appropriateness of employing CMA as
an economic methodology. The crucial decision relates to the fact that CMA has
been defined as being an appropriate methodology. Underpinning this decision is
a detailed analysis of clinical data that convinces the analyst that the interventions
being compared lead to equivalent health outcomes. Only in these strictly
controlled circumstances is it legitimate for CMA to concentrate on costs alone.
As such, a crucial and indispensable element underpinning the decision to use
CMA as an economic methodology is the need to unambiguously determine the
therapeutic equivalence of competing interventions (Newby and Hill 2003). In practice,
therefore, the extent to which CMA represents an appropriate methodological structure
is entirely determined by the interpretation that can be placed on the available
clinical evidence.
6.1.1 What is Meant
by Therapeutic Equivalence?
The
extent to which alternative health care technologies are sufficiently similar
to justify the use of CMA is an area of theoretical uncertainty and, thus,
still open to subjective interpretation, with the majority of published CMAs
appearing to be based on assumptions rather than evidence of clinical
equivalence. This primacy of hope over experience may cause misleading
recommendations to be made for health care resource allocation.
Given
this fact, it is perhaps surprising that the exact nature of the evidence base required
to support therapeutic equivalence and, hence, the
appropriateness of CMA as an economic methodology has not been subject to more
intense scrutiny. CMAs are frequently based on the results of clinical trials
that have attempted but failed to identify the superiority of a new drug over
the existing “gold standard” therapy. This occurs despite the obvious fact that
the inability of a health intervention to prove superiority in a superiority
trial (ST) in no way indicates that this necessarily implies clinical
equivalence. Recent advances in clinical trial design have made it easier to directly
compare clinical equivalence in a more meaningful manner with the development of
non-inferiority trials (NIs) allowing this issue to be directly addressed. Alternatively,
where a trial is initially designed as a ST but such superiority remains unproven,
the analysis can be switched from superiority to non-inferiority in appropriate
cases. The use of such improvements in trial design should enable CMAs to be more
effectively targeted in a manner that ensures that they are undertaken only in appropriate
circumstances using rigorous sources of evidence. In this manner, only CMAs
that meet minimum standards with regard to clinical equivalence will be accepted;
CMAs that fail to meet such criteria will be dismissed. Such an approach would
enable health economics to gain enhanced credibility from the use of this potentially
valuable economic methodology.
6.1.2 Optimizing Evidence from
Clinical Trials
If
CMAs are to form a reliable basis for health care decision-making, due
consideration must be given to the claims of clinical equivalence that are
crucial to the adoption of the CMA methodology. The implications of adopting an
inappropriate clinical trial design or misinterpreting the results of a
clinical trial are often considerable:
“…
wrongly discounting treatments as ineffective will deprive patients of better
care.
Wrongly accepting treatments as effective exposes
patients to needless risks and wastes.” (Tarnow-Mordi and Healy 1999 p. 210).
RCTs
typically compare the gold standard existing treatment with a new intervention (Tramer, Reynolds et al. 1998). RCTs can be structured to
evaluate ST, therapeutic equivalence (ET) or therapeutic NI. The trial designs
differ in terms of their objectives and these differences have significant
implications for the use of the CMA methodology. The greatest support for the
use of CMA occurs when an ET proves that two health care technologies are
clinically equivalent; however, there exists a myriad of “gray” areas that may
be indicative of therapeutic equivalence and, hence, require more careful
analytical consideration and judgement. Such gray
areas are analyzed in detail in the remainder of this chapter.
6.2 Sources of Clinical Tria l Evidence
6.2.1 Superiority Trials
The
extent to which clinical evidence can be used to inform CMAs is dependent on the
design of the RCT. STs are specifically designed to show a difference in health
benefits between two health care technologies. Typically, the primary objective
of the research is to determine whether an experimental intervention is more
efficacious than the established gold standard treatment. To identify whether
there is a difference in health benefits between two health care technologies
it is necessary to begin with a null hypothesis that treatment X yields the
same health benefits as treatment Y.
The
ST estimates the probability that the effect exists when the null hypothesis is
true using the test statistic (p-value). The smaller the size of the p-value
the more likely it is that the null hypothesis is false and that a difference
does exist between the health benefits generated by
the treatments. P-values, therefore, can identify statistically whether an
effect is likely by conveying information about the probability of an incorrect
inference given the observed effect but can say nothing about the size of the
effect or its clinical relevance.
Newby
and Hill (2003) emphasize the inadequacy of using p-values obtained in STs to
interpret the results of clinical trials and recommend the use of confidence intervals
and personal judgment when determining clinical equivalence before accepting or
rejecting an equivalence claim:
“…
leaving it up to the reader to decide whether the
confidence interval includes or excludes potentially clinically important
differences between two treatments. If it does not exclude
differences … assume that the two drugs are not the same” (Newby and Hill
2003).
When
the original objective of an ST is not achieved, there is an obvious incentive to
refocus the analysis to support more restricted claims of clinical equivalence.
However, STs are specifically designed to demonstrate that there is, indeed, a
difference and, thus, to reject the null hypothesis in favor of the alternative
hypothesis (i.e., that there is a difference). In STs, it is impossible to
prove that the null hypothesis is true, as the aim is to reject it by proving
that the observed difference is unlikely to be commensurate with equivalent
health outcomes of the competing health care interventions.
In
CMAs the clinical evidence from failed STs is often misinterpreted as proving that
the health care interventions being compared are clinically equivalent. Such
methodological flaws resulting from the misinterpretation of clinical trial
results can also “…lead to false claims, inconsistencies and harm to patients”
(Greene 2000 p. 715).
However,
if appropriately planned for, it is possible to switch the focus of the analysis
from superiority to non-inferiority in a single trial. Thus, failed STs that
are well designed and have adequate sample size could therefore potentially be
used to provide evidence of health equivalence for use in CMAs.
6.2.2 Equivalence Trials
6.2.2.1 Characteristics of
Equivalence Trials
ETs
are intended to demonstrate that the effect of a new treatment is not worse
than the effect of the current treatment by more than a specified equivalence
margin. The aim of an ET is, therefore, to specifically rule out significant
clinical differences between the treatments by directly evaluating the extent
to which two health care interventions have equivalent therapeutic effects.
Briggs and O’Brien (2001) argue that CMA should be used only when clinical
evidence has been obtained from an ET. They argue that it is inappropriate to
use the results of a failed ST to demonstrate clinical equivalence “… unless a
study has been specifically designed to show the equivalence of treatments it
would be inappropriate to conduct cost-minimization analysis” (Briggs and
O’Brien 2001).
However,
even where an equivalence trial indicates clinical equivalence in primary outcomes,
scrutiny of secondary outcomes may reveal significant differences in safety,
cost, or convenience. “… one therapy may offer clinical benefits such as a more
convenient administration schedule, less potential for drug interaction or
lower cost” (Hatala et al. 1999 p.9).
Reliance
on a single clinical measure of effectiveness may potentially be misleading as
it may fail to capture an important difference in health outcome between two alternatives.
Thus, ideally, clinical equivalence should be established for a range of health
outcomes before the use of CMA can be supported. In addition, in evaluating claims
of clinical equivalence it is important to acknowledge that:
“It
is never correct to claim that … there is no difference in effects of
treatments.…
There
will always be some uncertainty surrounding estimates of treatment effects, and
a small difference can never be excluded” (Alderson and Chalmers 2003 p.476).
Even
if one compared a drug with itself, there would be a difference; therefore, it
cannot be unequivocally claimed that two health care technologies are
clinically equivalent. Thus, even where the results of ETs indicate no
difference, this may simply indicate that the true difference exists outside of
the specified probabilities of error.
If
clinical equivalence is demonstrated in a good quality ET, there remain two other
issues that must be addressed prior to unambiguously supporting the use of the cost-minimization
approach. First, the primary health outcome must encompass the main benefit(s)
of the treatments being compared. Second, any differences in other health
outcomes, e.g., secondary health outcomes, must be sufficiently small so as not
to attain clinical significance. If these assumptions cannot be substantiated,
then it would not be appropriate to adopt the CMA approach despite the
availability of equivalence obtained in an ET.
6.2.2.2 Equivalence Range or Margin
A
crucial step in the design of an ET is the definition of clinical equivalence.
The equivalence margin attempts to incorporate all values that represent
unimportant clinical differences in treatment and must be stipulated in advance
of the clinical trial. The equivalence range, therefore, includes the largest
difference between treatments that is clinically acceptable before treatments
become defined as providing significantly different benefits. The first step in
any ET is, therefore, to define the smallest unacceptable degree of inferiority
or superiority to ensure that the ET can be appropriately powered. For example:
“…
if the difference between the two groups in respect of change in pulmonary
function was within +/– 1.5 units, then the treatments would be considered clinically
equivalent” (Huson 2004 p.2).
This
means that if treatment A is better or worse than treatment B by more than a
1.5 unit change in pulmonary function, the two treatments cannot be considered
to be clinically equivalent. Clinical equivalence can be claimed if the 95% CI
around the difference in treatments is found to lie entirely within the
predetermined clinical equivalence margin. The setting of the equivalence
margin communicates a judgment about what is and what is not clinically and
statistically acceptable (Pater 2004).
Clearly,
different clinical situations require different equivalence margins and
analysts must justify their chosen range with regard to clinician opinion and
previous trials comparing active controls with placebo. An equivalence margin
that is too wide could mean that significantly different treatments are
considered to be clinically equivalent; conversely an equivalence margin that
is too narrow could mean that clinically equivalent treatments are mislabeled
as being significantly different. It is important that good clinical judgment
be combined with sound clinical and statistical reasoning to ensure that the
chosen margin is clinically relevant and statistically feasible.
A
negative study result from an ET can take two forms. The CI around the
treatment difference may lie partially within the equivalence margin or it can
lie entirely outside, leading to the conclusion that the probability of a
difference between the two treatments has not been rejected (see Figure 6.1).
Figure
6.1 Interpretation of equivalence trials.
6.2.3 Non-Inferiority Trials
6.2.3.1 Characteristics of a
Non-Inferiority Trial
The
rationale behind a NI trial is to demonstrate that the new health care
technology is not worse than the current health care technology by a pre-stated
clinical margin. This type of trial is useful when the clinical issue relates
to the extent to which the new health care technology is as good as current
therapy. In NIs, analysis is focused entirely in one direction—typically the
new treatment is not worse than the established therapy by more than the
non-inferiority margin specified. An improvement of any size fits within the
definition of non-inferiority. Span and colleagues published the first paper
that acknowledged the link between CMA and NIs:
“…
the most efficient analysis of the clinical effect in
a cost minimization study is the non-inferiority analysis” (p.262). They
conclude that: “… to obtain valid results from a cost-minimization study, care
has to be taken to adapt the correct methodology for non-inferiority testing in
clinical outcomes” (Span, TenVergert et al. 2006
p.261).
To
ensure a robust interpretation of trial results, some analysts call for both
per protocol (PA) and intention to treat (ITT) analyses to be conducted and
only if both types of analysis support the hypothesis should non-inferiority be
claimed (Snappinn 2000). Therefore, the extent and
nature of the evidence of non-inferiority that is required to provide an
acceptable platform on which to base a CMA is still open to debate.
6.2.3.2 Non-Inferiority Range or
Margin
The
non-inferiority range should be set in relation to the clinical notion of a
minimally important effect. An acceptable non-inferiority margin depends on
defining a difference that has previously been identified as not being
clinically significant. To do this, two additional conditions must be met.
First, the smallest expected effect of the active control over placebo must
exceed this margin to ensure that no positively harmful treatments can be
introduced and, second, the margin must be no greater than the difference
between active treatments judged clinically important.
In
a NI, non-inferiority is demonstrated when the CI around the treatment
difference lies entirely to the
right of the lower bound of the non-inferiority margin. Non-inferiority is not
demonstrated if the lower bound of the CI lies to the left of the non-inferiority
margin (see Figure 6.2).
Figure
6.2 Interpreting NIs using CIs.
6.3 Other Issues To
Be Addressed In Evaluating Equivalence
6.3.1 Statistical versus Clinical
Significance
One
of the failings of statistical analyses undertaken in the context of an ST is
that statistical significance may differ from clinical significance. Variables
that are identified as exhibiting statistically significant differences may be
entirely unimportant from a clinical perspective, whereas clinically crucial
differences remain crucial even if they fail to achieve statistical
significance. In contrast, in ETs and NIs, statistical and clinical
significance are inextricably linked via the setting of equivalence and
non-inferiority margins.
6.3.2 Equivalence in Single or
Multiple Outcomes?
In
any clinical trial it is necessary to identify a primary health outcome that is
common to the competing alternative interventions. Choice and measurement of
such an outcome measure is a crucial step in determining the appropriateness of
the trial as an evidence source on which to undertake CMAs. To be of value, the
primary health outcomes must be the dominant outcome from the perspective of
both patients and clinicians and capture the most clinically relevant benefits
of the competing treatments. If not, claims of clinical equivalence, even when
based on ETs, are not sufficient to support the use of CMA.
In
clinical practice it is highly unlikely that two health care interventions will
yield exactly the same health benefits in all dimensions of clinical and
patient outcomes. Typically, the design of ETs and NIs identifies a single
endpoint for comparison despite the perception that one of the treatments is
likely to offer significant advantages in another area. For example, where two
treatments have equal efficacy, yet one is more convenient to patients, the
extent to which CMA can be appropriately utilized depends largely on the
perspective adopted by the analysis. Where equivalence is not demonstrated for
all important outcomes, the analyst must provide explicit justification for
using the cost-minimization approach in light of the study question and
perspective. In large part, the interpretation of clinical equivalence will
depend on the specific circumstances of the clinical trial, the range of
outcomes being measured and the judgement of the
analyst. In such cases it is difficult to provide specific guidance that would
be appropriate in all cases.
6.3.3 Whose Views of Clinical
Equivalence Should be Preeminent?
Definitions
of clinical equivalence will depend on whose views we consider to be the most
important (patients, clinicians, or society). Generally, lead investigators in clinical
trials specify the primary and secondary health outcomes to be measured, with the identification of the primary outcome
measure’s being based on relevant clinical experience, published clinical
evidence, and knowledge of patient needs. The crucial factor is to ensure that
the choice of health outcome measures used to determine clinical equivalence is
clinically meaningful to the patient.
6.3.4 Over What Period Should We
Evaluate Clinical Equivalence?
The
benefits of health care technologies will vary in relation to the time point at
which they are measured. In a clinical trial the primary health outcome measure
might exhibit statistically significant differences at 3 months but not at 6 or
12 months. In such circumstances, do we interpret the therapeutic interventions
as being equivalent and, hence, appropriate for analysis in the context of a
CMA? It is important to acknowledge that subsequent reanalysis is required to
demonstrate continued clinical equivalence.
6.4 Effectively Targeting the Use of
CMA
The
current use of RCT evidence to support statements of clinical equivalence is
inadequate, and clear and appropriate decision rules are required in the future
to ensure that unambiguous evidence of clinical equivalence is a feature of
future CMAs. In the absence of such evidence it would be potentially misleading
to use flawed analyses as the basis for health care decision-making. While it
is comparatively simple to identify circumstances in which the use of CMA as an
economic methodology is clearly inappropriate, it is more difficult to specify
unambiguous decision rules that identify circumstances in which clinical
evidence clearly supports the use of CMA. The appropriateness of using CMA must
be judged in the light of the totality of the clinical evidence supporting or
refuting the hypothesis of therapeutic equivalence between two competing
interventions, combined with the specialist knowledge and expertise required to
place such evidence in context. However, certain limited guidance can be
provided with regard to effectively targeting CMAs.
First,
clinical evidence from a well-designed ET represents the gold standard in supporting
claims of clinical equivalence in support of the use of CMA. However, even
where data is available from an ET it still remains important to consider the extent
to which the primary health outcome fully captures the benefits being derived from
the health care treatments being compared. If other benefits are clinically meaningful
to patients and clinicians, additional comparisons of clinical equivalence may
be required.
Second,
failure to prove clinical superiority should not be interpreted as providing evidence
of clinical equivalence. In certain circumstances, and if planned into trial
design, trial data may be re-analyzed to assess clinical equivalence, but such reinterpretation
of the dataset requires further analysis if the use of CMA is to be justified.
In particular, a non-inferiority statement should be stipulated in the clinical
trial protocol to ensure that valuable information can still be derived even if
superiority is not proven.
Third,
the extent to which data from NIs can be used to justify CMAs is currently subject
to a great amount of uncertainty. In particular, to what extent proof of noninferiority represents an acceptable approximation of
“therapeutic equivalence” and, hence, justifies the use of CMAs, is still open
to debate.
Finally,
where CMAs are based on valid claims of clinical equivalence derived from
appropriate sources of RCT evidence, it represents an appropriate and powerful method
of economic evaluation. However, it is crucial that in interpreting the results
of CMAs, the informed decision-maker uses his or her clinical judgment to assess
the quality and quantity of the evidence in support of therapeutic equivalence and,
hence, identifies the theoretical justification for the use of CMA. In cases
where the decision-maker does not accept claims of clinical equivalence, the
results of the CMA should clearly not be used as the basis for decision-making.
The
cost-minimization method of economic evaluation has always been employed in a
more haphazard manner than other methods of economic evaluation. It is crucial to
rectify this situation to ensure that only techniques that prove to be robust
and reliable in improving health care decision-making are incorporated into the
toolkit employed by the health economist. However, exactly how similar do
outcomes have to be to support the application of this powerful economic
methodology? The most appropriate design for a clinical trial to generate
evidence that two health care technologies are identical or similar is the ET.
Such trials are specifically designed for this purpose and, therefore, any
differences that are identified between the health interventions being compared
are neither clinically nor statistically significant.
It
is essential that health economists and decision-makers are clear on what is meant
by the concept of clinical equivalence and to acknowledge that, given the heterogeneous
nature of patient populations and treatment outcomes, it is likely to prove
impossible to achieve exact equivalence between competing health care
interventions. Ultimately, it is up to the health economist to justify the use
of CMA just as it is up to the decision-maker to judge the extent to which the
results obtained should be influential in determining decision making.
7.1 The Rationale for Cost-Effectiveness
Analysis
As
noted in prior chapters, the economic evaluation of pharmacotherapies and other
health care interventions is growing in importance as the resources directed
toward health care account for progressively larger portions of the budgets of
governments, employers, and individuals. Making rational decisions under
conditions of resource constraints requires a method for comparing alternatives
across a range of outcomes, allowing a direct ranking of the costs and benefits
of specific strategies for preventing or treating a particular illness.
Cost-effectiveness
analysis (CEA) provides a framework to compare two or more decision options by
examining the ratio of the differences in costs and the differences in health
effectiveness between options. The overall goal of CEA is to provide a single
measure, the incremental cost-effectiveness ratio (ICER), which relates the amount
of benefit derived by making an alternative treatment choice to the
differential cost of that option. When two options are being compared, the ICER
is calculated by the formula:
In
medical or pharmacoeconomic cost-effectiveness
analysis, health resource costs (the numerator) are in monetary terms,
representing the difference in costs between choosing option 1 or option 2. In
cost-effectiveness analysis, the differential benefits of the various options
(the denominator) are non-monetary and represent the change in health
effectiveness values implied by choosing option 1 over option 2. Typically,
these health outcomes are measured as lives saved,
life years gained, illness events avoided, or a variety of other clinical or
health outcomes. Unlike CEA, cost-benefit analysis values both the costs and
benefits of interventions in monetary terms. Cost-utility analysis, a subset of
CEA where intervention effectiveness is adjusted based on the desirability (or
utility) of the resulting health states, is discussed in Chapter 9 as it
relates to the cost-effectiveness of human papillomavirus (HPV) vaccine.
7.2 The Cost-Effectiveness Plane
A pharmacoeconomic analysis is often interested in how much
more of a health outcome can be obtained for a given financial expenditure.
Limited resources may, many times, constrain choices between medical options.
The cost-effectiveness plane serves to clarify when these choices may be easy
or difficult.1 The cost-effectiveness plane is typically drawn with the
differences in cost (or the incremental cost) on the y-axis and the differences
in effectiveness (or incremental effectiveness) between the two options on the
x-axis (Figure 7.1). In this example we will compare an existing program with a
new program. The existing program, acting as the comparator, will be at the
origin of both the cost and effectiveness axes, depicting the current level of
expenditure and benefit with which a new therapy is compared. The new therapy
can be more expensive, less expensive, or equivalent in costs to the current
option. Similarly, the new option can be more effective, less effective, or
equivalent in clinical effectiveness as compared with the existing strategy or
therapy.
Figure
7.1 The cost-effectiveness plane.
This
produces four possible options for the results of the analysis of a new
strategy compared with an existing one. If the new program is less expensive
and more effective than the existing program, then the point representing the
new program falls into the southeast (SE) quadrant of the cost-effectiveness
plane. Points in this quadrant are called dominant, and strategies that have
such a characteristic should be chosen over the existing strategy due to their
superior outcome at diminished costs. These strategies are “cheaper and better”
than current therapy and should be adopted. Examples of strategies in this
quadrant are laparoscopic cholecystectomy compared with other therapies for
symptomatic gallstones or interventions to decrease cigarette smoking.
If,
on the other hand, the new program is more expensive and less effective than the
existing one, then this program falls into the northwest (NW) quadrant of the plane.
Strategies in this quadrant are considered to be dominated by the current strategy
and should not be chosen due to poorer outcomes at greater cost. Although existing
strategies in this quadrant are perhaps relatively rare, there are examples of strategies
that do not appear to derive a benefit, yet incur substantially more health care
costs than other options. Examples include amoxicillin prophylaxis compared with
no antibiotic for dental procedures in patients at moderate risk for infective
endocarditis and magnetic resonance imaging vs. endocrinologic
follow-up of patients with asymptomatic pituitary microadenomas.
If
the new program is either dominant or dominated (i.e., in the SE or NW
quadrants), a formal CEA is not needed to assist the decision—the decision is
(or should be) obvious. However, if the new program is both more effective and
more costly, falling in the northeast (NE) quadrant, then a CEA would be useful
to define the tradeoff between increases in costs and effectiveness and to
calculate the cost per unit of effectiveness gained. Similarly, a CEA would
also be useful if the new strategy fell into the SW quadrant as being both less
costly and less effective than the existing program, once again to define the
tradeoffs between programs and to ascertain the cost-effectiveness ratio. This
graphical display emphasizes one of the most fundamental and important concepts
of cost-effectiveness analysis; it is useful only when there is a tradeoff
between the cost of a strategy and the benefit derived from that strategy.
7.3 Basic Components of a
Cost-Effectiveness Analysis
Several
factors should be considered in the construction of a CEA (Table 7.1). A high-quality
analysis will include and describe the relevant options, clearly state the perspective
of the analysis, choose a relevant time horizon over which to track costs and
effects, consider the appropriate population, accurately measure the costs and effectiveness
of the competing options, account for the differential value of costs and outcomes
that occur at different times in the future, and account for uncertainties of
assumptions and values in the context of an appropriately constructed analytic model.
Following is a description of these concepts in more detail.
7.3.1 Enumeration of the Options
A
CEA requires a comparison between two or more options. A single option cannot be
cost-effective in isolation—an option can be considered cost-effective or not
cost-effective only in comparison with other options. Additionally, the cost-effectiveness
of a strategy is highly dependent on the specific choice of comparators included
in the analysis, and care must be taken to include all of the clinically reasonable
options. At a minimum, the comparators include the current standard of care and
a range of typically utilized options. A cost-effectiveness analysis of a new
therapy compared with a strategy that is not typically used, or is used only in
atypical circumstances, is not useful for clinicians or policy makers. It is
often reasonable to include a “do nothing” option,
especially if doing nothing is a legitimate clinical strategy, but also as a
baseline comparator to assess the clinical realism of the model and analysis.
In all cases, the strategies should be described in sufficient detail such that
readers could replicate or implement the strategy in their own settings.
7.3.2 Perspective of the Analysis
Choosing
the perspective or set of perspectives to be considered in a CEA is essential, as
this choice determines the cost values to be contained in the analysis. For
example, an analysis from the societal perspective considers all costs, while an
analysis from the patient perspective would consider only costs borne by the
patient. Other possible perspectives include the third-party payer (insurance)
or health system perspective where costs for which these entities are
responsible are considered in the analysis; the hospital or health agency
perspective includes the costs of providing various health services. Whenever
possible, the societal perspective should be included in the set of
perspectives to be considered in analysis, because it is the broadest and is recommended
for the reference case analysis by the Panel on Cost-Effectiveness in Health
and Medicine.
7.3.3 Time Horizon
The
analyst must decide a priori how long the costs and effects of the various
interventions in the analysis will be tracked. This is usually determined by
the clinical features of the illness or its treatment. For example, a CEA of a
new antibiotic for acute dysuria treatment in otherwise healthy women might
appropriately have a very short time horizon of only a month, as there are
virtually no long-term effects of either the disease or its treatments. On the
other hand, cost-effectiveness analyses designed to value the effects of
cardiovascular risk reduction need to assess the outcomes for much longer time
periods; typically such an analysis would follow treatments and effects until
death. In any case, all of the strategies must be followed or modeled for the
same time horizon. Methods for modeling costs and effects,
even in situations where this modeling extends beyond the existence of specific
data.
7.3.4 Scope of the Analysis
An
analysis might be relevant for an entire population or for only a relatively
small population subgroup; the analyst will need to appropriately choose the
cohort to be considered in the analysis. For example, if an intervention is to
be directed toward elderly patients with diabetes in order to prevent diabetes
complications, limiting the scope of the analysis to an elderly, diabetic
population is a logical choice, while if the question is regarding diabetes
prevention in adults, a broader population scope is required. The scope of
outcomes to be considered is another important consideration. In the example
above, a broad or narrow range of diabetes outcomes could be considered in an
analysis of elderly diabetics. If a small number of complications are modeled,
the data requirements of the model would be less but the conclusions might be
limited compared with a model with a broader range of complications considered.
However, a more comprehensive model would have greater data needs and require more
complex model construction. Choosing the scope of an analysis often means finding
a balance between simplicity and complexity, frequently determined by the clinical
situation modeled and the question to be examined.
7.3.5 Measuring and Valuing Costs
Data
sources for costs must be found and incorporated into the analysis. Cost data can
be obtained from clinical trials, but more often other sources will need to be
utilized. In addition, the analyst will need to choose between micro-costing or
macrocosting methodologies or some mix of the two,
often based on the perspective taken in the analysis. Micro-costing enumerates
and identifies each item that is incorporated into a particular service,
requiring detailed data on supplies used, personnel, room, and instrument
costs, and often needing time-and-motion studies to accurately capture medical
service costs. Macro-costing (or gross costing) uses data, often from large
government databases, to estimate average costs for a care episode, for example
the average cost of coronary artery bypass grafting or of a hospital stay for
pneumonia. In the US, Medicare reimbursement data or the Healthcare Cost and
Utilization Project (HCUP) database are often used for this purpose.
7.3.6 Measuring and Valuing Outcomes
The
effectiveness outcome for the analysis must be chosen and outcomes data found, often
based on data availability. Randomized trials are excellent data sources on the
effects of therapies, but study entrance criteria frequently limit
applicability to a more general patient population. Cohort studies are useful
for risk factor determination and for determining the natural history of an illness.
Administrative databases are excellent sources for broad population-based estimates
of disease and for the effectiveness of therapies, unlike randomized trials which,
in general, estimate efficacy. However, administrative databases often pose
difficulties in accounting for possible confounding variables in the data set.
Meta-analyses provide summary measures for parameters, but studies considered
are generally limited to randomized trials, thus limiting generalizability. The
perspective of the analysis may also influence the effectiveness outcome
chosen. Life years or quality-adjusted life years (QALYs) gained are certainly
relevant for analyses using the relatively broad-based societal or health
system perspectives, but may not be as important when a narrower perspective is
chosen, such as that of an individual hospital, when effectiveness measures
such as bed day saved or drug administration error avoided might be more
relevant.
7.3.7 Time Preference
The
differential timing of costs and outcomes should be considered in the analysis.
This is typically accomplished through the use of discount rates, where costs
and outcomes that occur in the present have higher values than those in the
future.
7.3.8 Choice of Analytic Modeling
Method
The
analytic model must also be selected. Cost data from clinical trials can allow
relatively straightforward calculation of incremental cost-effectiveness ratios
between management options, often the intervention arms of the clinical trial.
More often, data for the analysis must come from a variety of sources and may require
a decision analysis model as a framework for data synthesis.
7.3.9 Accounting for Uncertainty
Finally,
a sensitivity analysis to elucidate the effects of uncertainty on model results
should be performed. There are many goals of sensitivity analysis, and methods
for conducting such analyses. During model construction and validation,
sensitivity analysis is useful as a “debugging tool” to assure that the model
behaves as it was designed to behave. After the model is finished, sensitivity analysis
is useful to determine which variables have a large impact on the outcomes. Sensitivity
analyses can be used to determine the cost-effectiveness ratio in specified subgroups
of an analysis, as well as to determine how much a change in one variable will
alter the cost-effectiveness ratio. Finally, probabilistic sensitivity analyses
can be used to produce a version of a confidence limit or probability range
around the cost-effectiveness ratio.
7.4 Calculation of Incremental Cost-Effectiveness
Ratios
The
ICER requires a detailed enumeration of the costs and benefits of the
strategies being compared. Methods for measuring and estimating the costs and
benefits of strategies and interventions are often quite complicated. In this
section, we use the results of two existing pharmacoeconomic
studies to illustrate the calculation and use of the ICER. Details of the
enumeration of costs and outcomes in these studies are detailed in the studies
themselves.
The
following example considers low molecular weight heparin (LMWH) compared with
warfarin for the secondary prevention of venous thromboembolism in patients
with cancer. Aujesky used a decision analysis model
and data from a variety of sources to estimate the incremental
cost-effectiveness of two anticoagulant regimens. Analysis results, with
effectiveness in life years, are outlined in Table 7.2.
Typically,
the first step in calculation of ICERs among mutually exclusive options is to
order the options by cost. LMWH is both more costly and more effective than warfarin,
thus, neither strategy is dominant or dominated and a
CEA would be useful. Subtracting the cost of the warfarin strategy from that of
the LMWH strategy produces the incremental cost; the difference in life
expectancy between strategies is the incremental effectiveness. Dividing the
incremental cost by the incremental effectiveness produces the ICERs, $115,847
per life year gained, the unit cost of an additional life year occurring as a
result of LMWH use rather than warfarin.
7.4.1 Dominance and Extended
Dominance
Calculation
of the ICER can be more complicated when more than two strategies are being
considered. One of the complicating characteristics of the analysis of many options
is that some strategies may be dominated by others and should be removed from
further analysis. As noted in the description of the cost-effectiveness plane,
any strategy that is more expensive and less effective than an existing option
for the same illness (e.g., is in the left upper quadrant compared with the
existing strategy) is said to be strictly dominated; one would never choose
such a strategy when an alternative would produce a better outcome at a cheaper
price. Strict dominance is also termed strong dominance by some authors. A
second type of dominance occurs when a particular strategy is more expensive
and less effective than a linear combination
of two other strategies. This is called extended
dominance, and represents a situation where one could achieve a better
outcome at less cost by treating a proportion of the population with a
combination of two alternative strategies. Extended dominance can also be
referred to as weak dominance. We illustrate both types of dominance in the
following example.
Using
a decision analysis model, we performed a CEA of testing and antiviral treatment
strategies for adult influenza, using days of influenza illness avoided as an effectiveness
term in the analysis. Cost and effectiveness values estimated by this analysis
are shown in Table 7.3. (Please note that in a separate analysis the other neuraminidase
inhibitor, oseltamivir, was substituted for zanamivir, with similar cost-effectiveness results.)
Once
again, the first step in calculation of incremental cost-effectiveness ratios
among mutually exclusive options is to order the options by cost. Doing so with these data results in Table 7.4.
Next,
options of lesser effectiveness and of equal or greater cost than another
option are removed due to strict, or strong, dominance. These strictly
dominated options, which are inferior both in terms of cost and effectiveness,
do not need to be considered further in the analysis. In this example,
“Testing, then amantadine” costs more and is less effective than “Amantadine
(without testing).” Thus, “Testing, then amantadine” is strictly dominated and
can be removed from consideration. Similarly, “Testing, then rimantadine” also costs more and is less effective than the
“Amantadine” strategy and the “Rimantadine (without
testing)” strategy and, thus, can be eliminated due to strict dominance.
Removal of these two strategies results in Table 7.5.
Then,
starting with the second row, the differences in cost and effectiveness between
that row and the preceding row are calculated. These results are the
incremental cost and incremental effectiveness between the two adjacent
strategies. The incremental cost divided by the incremental effectiveness
produces the ICER, the cost per illness day prevented. This same procedure is
then followed for the remaining rows in Table 7.6.
Next,
the calculated ICERs are examined for extended, or weak, dominance of
strategies. This occurs when the ICER of a strategy is greater than the
strategy below it, signifying that the subsequent strategy would be preferred.
In this case both “Rimantadine” and “Test/Zanamivir” have higher ICERs than Zanamivir;
thus, these strategies would not be preferred over Zanamivir
due to extended dominance and can be removed from consideration. Removing these
strategies from the table and recalculating the ICER of Zanamivir
compared with Amantadine results in Table 7.7.
This
same procedure can be performed graphically using the cost-effectiveness plane.
Figure 7.2 depicts all the testing and treatment strategies on the
cost-effectiveness plane. Starting with “No testing or treatment,” the least
costly option, a line is drawn to the strategy that produces the shallowest
slope (i.e., the smallest ICER), which is “Amantadine.” From Amantadine, the
shallowest positive slope is to Zanamivir. The
resulting line is the cost-effectiveness efficient frontier; any point not on
this frontier is dominated, either by strict dominance or extended dominance, as
illustrated by the “Testing” strategies and by the “Rimantadine”
strategy.
Figure
7.2 Cost and effectiveness values for influenza management strategies plotted
on the cost-effectiveness plane. The line represents the cost-effectiveness
efficient frontier, gray points denote strategies that are strictly dominated, and open points show strategies that are eliminated
from consideration by extended dominance.
All
reasonable strategies should be included in cost-effectiveness analyses so that
true ICERs can be calculated. For example, if the Amantadine strategy were omitted
from the analysis above, the ICER of Zanamivir would
be $60 per illness day avoided when compared with “No testing or treatment”
rather than $198 when compared with Amantadine. Omitting Amantadine would not
give a true picture of the incremental value of Zanamivir,
i.e., it would not tell us how much more would be paid for the gains in
effectiveness seen with Zanamivir compared with all
other reasonable strategies.
Similar
considerations apply to the average cost-effectiveness ratio, here the cost divided
by the illness days avoided; for example, the average cost-effectiveness ratio for
Zanamivir is $137.1/0.74 or $185.27 per illness day
avoided. When comparing mutually exclusive strategies, as we are in this
example, the absence of incremental comparisons between strategies in the
average cost-effectiveness calculation does not allow for elimination of
dominated strategies or for calculation of incremental gains and costs between
strategies. The average cost-effectiveness ratio is useful in the evaluation of
mutually compatible programs that are subject to a budget constraint, where
programs are ranked, lowest to highest, by average cost-effectiveness ratio,
then funded in that order until the budget is exhausted. Use of the average
cost-effectiveness ratio in this fashion would maximize the health benefit for
a given monetary expenditure; however, its use for this purpose has been largely
theoretical to this point.
7.4.2 Sensitivity Analysis
The
next step in a CEA is the performance of sensitivity analyses. Typically, univariate, or one-way, sensitivity analyses are performed
on parameter values, and further multiple parameter sensitivity analyses may
also be performed.
7.4.3 Interpretation of CEA Results
To
reiterate a prior point, CEA hinges on comparisons between strategies. A single
option alone cannot be cost-effective; options can only be cost-effective
compared with other options. The relative cost-effectiveness of one option
compared with another is subject to interpretation and, perhaps as a result,
the term “cost-effective” has been misused (although perhaps less so now than
in the past, due to increasing familiarity with the true meaning of the term).
Cost-effective does not necessarily mean cost-saving. New health programs that
are less costly and more effective than existing programs are clearly good
buys, but a new program that costs more and is more effective than the existing
program can be cost-effective without costs being saved, depending on how much
is willing to be paid for a given health benefit. Cost-effective has also been
incorrectly used to mean cost-saving when no determination of effectiveness differences
between options has been performed; buying health insurance from one carrier
that costs less than insurance from another carrier is not making a
cost-effective decision when there is no comparison of health benefits between
insurance plans; this would be a cost-minimization evaluation. Similarly,
“cost-effective” has been misused to mean “effective” when there is no cost
comparison. The correct meaning of “cost-effective” is that a program or strategy
is worth the added cost because of the benefit it adds compared with other interventions.
The application of the method requires a determination of the value of health
care benefits as well as costs.
Returning
to our influenza example, how can one interpret the incremental cost-effectiveness
ratios of the amantadine and zanamivir strategies?
One of the first steps in interpreting cost-effectiveness analyses is to
understand what cost-effectiveness cannot do. It cannot make the “correct”
choice; instead, it provides an analysis of the consequences of each choice.
Cost-effectiveness analysis is not designed to address the social, political,
or legal issues that might arise from a medical decision. Thus, if differing
strategies involve questions of equity, social justice, legal responsibilities,
or public opinion that need to be weighed in making a medical decision,
consideration of more than strategy cost-effectiveness is necessary.
Cost-effectiveness is one of many aspects of a decision to be considered and
interpreted by decision-makers, be they physicians in the care of an individual
patient or health policy makers in a broader population-based medical care
context.
Let
us assume for now that sociopolitical issues are similar between our example strategies,
allowing us to concentrate on the cost-effectiveness results as a major basis for
the decision. In this case the question is: which strategy should we choose
based on the ICERs calculated for each strategy? Or more bluntly, which
strategy is the most “cost-effective”? The answer depends on the willingness-to-pay
per unit health outcome (here, per illness day avoided). If the
willingness-to-pay is less than $9 per illness day avoided, then “No testing or
treatment” would be chosen, since the ICERs of the other strategies are
≥$9 per illness day. If willingness-to-pay thresholds are higher, other
strategies would be chosen: Amantadine is chosen if the willingness-to-pay is $9
– $197, and Zanamivir is chosen if the
willingness-to-pay is ≥$198 per illness day avoided.
How,
then, is a reasonable cost-effectiveness willingness-to-pay threshold
determined? This is a difficult question with no clear answer at this point,
complicated by the many possible effectiveness values (life years gained, lives
saved, illness days avoided, etc.) that could be considered. Cost-effectiveness
comparisons between interventions using a common effectiveness measure can be
useful in gaining a sense of an intervention’s relative value. For example, if
Treatment x for Disease X costs $100 per illness day prevented and is
considered economically reasonable while Treatment y for Disease Y costs $500
per illness day avoided and is considered too expensive, then Treatment z for
Disease Z costing $550 per illness day prevented might also be considered too
expensive. However, the usefulness of this comparison depends on the similarity
of illness days between Diseases X, Y, and Z. If Disease Z is worse than X or
Y, then there might be a higher willingness-to-pay to avoid a more severe
illness day from Disease Z than to avoid a more moderate illness day due to X
or Y.
Sensitivity
analysis may also be useful in the interpretation of results. If variation of
analysis parameter values does not change the conclusion drawn from the base case
analysis results, the analysis is said to be “robust,” and increases the
confidence in analysis results. Analyses that are not robust, where conclusions
may change with variation of one or more parameter values,
are termed “sensitive to variation,” and their results are viewed with less
confidence. Depending on the data used in the analysis, this confidence or
uncertainty can be quantified through development of confidence intervals for
cost-effectiveness ratios in empiric data sets or the use of probabilistic
sensitivity analysis and acceptability curves when empiric data sets are not
available.
A
number of other factors can make interpretation of CEAs challenging. Differences
in analysis results can be due to methodologic
differences between analyses. Cost-effectiveness analysis results are often
dependent on the perspective, time horizon, and assumptions used in the
analysis and, unless these factors are well-aligned between analyses,
discordant results can arise based solely on these technical differences.
Analyses using effectiveness values that are very specific to the medical
scenario being examined, such as deep venous thrombosis prevented or lumbar
discectomies avoided, may have few similar analyses available for comparison,
making interpretation of their results challenging. Even if analyses with
similar effectiveness values are available, their results could be difficult to
compare with those of interventions for other disease processes using other effectiveness
measures, thus limiting their comparability and interpretability. In these
cases, a common effectiveness measure would facilitate cost-effectiveness comparisons
over a broad spectrum of medical interventions. The use of quality-of-life
utilities and QALYs in cost-utility analysis, along with methodologic
recommendations to standardize analysis practices, such as those of the U.S.
Panel on Cost-Effectiveness in Health and Medicine, is largely motivated by the
need to facilitate such comparisons, and has resulted in resources such as the
online CEA Registry from Tufts University to make direct comparisons possible.
Cost-effectiveness
analyses compare medical intervention strategies through the calculation of the
incremental cost-effectiveness ratio, a measure of the cost of changes in
health outcomes. These analyses can be performed on clinical trial data when
information on both costs and effectiveness is available or, more commonly, through
the use of decision analysis models to synthesize data from many sources. Interpretation
of CEA results can be challenging due to the variety of health outcomes that can
be used as the effectiveness term in these analyses and to the absence of a
definitive criterion for “cost-effective.” A subset of CEA, cost-utility
analysis, attempts to make interpretation of results less difficult through the
use of a common effectiveness term, the QALY.
Using Cost-Effectiveness Analysis for
Setting Health Priorities
Governments
around the world face budget constraints that compel them to make tough
decisions about how best to invest funds for public health. They need a way to
evaluate which investments will address the most pressing health problems and bring
the greatest health gains. Cost-effectiveness analysis is an essential
evaluation tool that allows policymakers and health planners to compare the
health gains that various interventions can achieve with a given level of
inputs. Getting the most value for money has been a central thrust in the
analysis presented in Disease Control Priorities in Developing Countries, 2nd
edition (DCP2). The basic concepts underlying the analysis, as well as needed
improvements, are described here.
What
Is Cost-Effectiveness Analysis?
Cost-effectiveness
analysis is the primary tool for comparing the cost of a health intervention
with the expected health gains. An intervention can be understood to be any
activity, using human, financial, and other inputs, that aims to improve
health. The health gain might be reducing the risk of a health problem, reducing
the severity or duration of an illness or disability, or preventing death. If
the health outcome is the same, say preventing death from measles either by
immunizing a child or by treating the disease, then analysts need only compare
the costs of different interventions that can achieve that outcome. The result
is a costeffectiveness ratio, expressed as cost per
outcome, which can be compared across various types of services or various
service locations that perform the same function. The ratio is always discussed
in relative terms, as there is no “best” or absolute level of
cost-effectiveness.
The
cost-effectiveness of an intervention can vary greatly depending on a program’s
size and scope. Typically, as program coverage expands and more people are
served, the cost per outcome drops. For example, if more children can be
immunized with the same fixed costs like nurses and clinics, then each additional
immunization will be cheaper until the service approaches full capacity.
On
the other hand, costs can rise as coverage expands if it becomes harder to
reach additional patients. Therefore, depending on the comparison undertaken,
an analyst might look at the average cost-effectiveness ratio or the
incremental cost-effectiveness ratio. The average cost-effectiveness ratio looks
at total costs and total results, starting from zero, while the incremental
ratio compares additional costs and additional results, starting from the
current level of coverage or services.
Using
child immunizations as an example, the incremental cost of adding mobile
vaccination teams might be lower than expanding fixed clinic services,
particularly if the unvaccinated children are dispersed and hard to reach.
How
Do Analysts Measure and Compare Different Health Outcomes?
Cost-effectiveness
analysis requires health outcomes to be expressed in common units so that
comparisons among interventions can be made. All analyses start with some unit,
such as cases of a disease or injury, deaths, or numbers of people who quit
smoking or adopt some other behavior. All interventions that prevent death are
alike in terms of the common outcome. However, when lives are saved at different
ages—averting a death from malaria at age 2 versus a heart attack at age 50—the
outcome is no longer identical, and some adjustment must be made for the
difference in years of life saved.
For
interventions that prevent death, the analysis starts by estimating the deaths
prevented and the age at death to yield the number of life-years saved. The
number of life-years saved is the difference between age at death and life
expectancy remaining at that age. Standard economic analysis then discounts the
future years to take account of uncertainty and the advantage gained by
investing early.
Discounting means reducing the value of the outcome in
each future year by an amount that increases over time.
Thus,
if values are discounted at 3 percent annually, that means dividing the values
for year 1 by 1.03, those for year 2 by 1.03 squared, and so on. At that rate,
preventing an infant death saves not all of the 60 to 80 years of life expectancy
at birth (depending on the country), but at most 30 discounted years. Even when
discounted, however, saving infants’ lives yields substantial health gains.
The
Disease Control Priorities Project discounts years of life saved at a constant
3 percent per year. The same logic applies to interventions that avert a
chronic condition or disability, except that different disabilities must be
compared in severity. For short (acute) episodes of illness, age is not relevant—all
life years are regarded as equally valuable—and therefore discounting has no
effect.
New
Metric Used in Cost-Effectiveness Analysis: The DALY
The
disability-adjusted life year (DALY) was introduced by the World Health
Organization and the World Bank in 1993 and has been used since, with some
variations, for two related purposes. One is to measure the “burden of
disease,” the extent to which premature deaths and disabilities cause a loss of
health status compared to everyone’s living to old age in good health. The
other purpose is to compare the value of health interventions that have
multiple or different health outcomes occurring at different ages. In
particular, it allows for measuring and comparing health outcomes other than saving
lives. Used in DCP2, the DALY—or common unit of health loss or gain—takes into
account the duration and severity of a health problem and discounts future
years.
In
cost-effectiveness analysis, the DALY represents the number of years of
disability-free life that would be gained from a particular health
intervention—yielding a cost per DALY where cost data are available or can be
inferred.
Gaining
a DALY through a health intervention reduces the burden of disease; it is the
same as averting the loss of a DALY. The calculation includes assumptions about
severity (if the health condition is not fatal), the age at which an illness or
intervention occurs, the duration of ill health with and without the intervention,
and the remaining life expectancy at the age when the gain occurs.
For
interventions that aim to reduce risk factors for health (such as stopping
smoking) rather than directly affect illnesses or injuries, the analysis
includes estimates of reductions in ill health that result from changes in the
level of risk. Stopping smoking, for example, reduces deaths from both
cardiovascular disease and cancer.
DALYs
allow analysts to compare the cost-effectiveness of different interventions and
different health outcomes, by expressing diverse health outcomes in a common
unit. As a result, it can help guide where best to invest scarce health resources.
For example, a coronary artery bypass costs, on average across world regions,
US$37,000 per DALY gained—far beyond the per capita income of most countries—compared
with an average of only US$409 for the polypill (several
medications for preventing heart disease in a single pill). The latter is a
“best buy” for developing countries.
However,
both interventions are much less cost-effective than saving life years of a
middle-aged person by treating active tuberculosis (and thereby preventing
transmission) at a cost as low as US$15 per DALY.
In
high-income countries, some analyses include quality-adjusted life years
(QALYs), an alternative measure of how much a year of life is diminished if a
person suffers health limitations. This measure can account for people
suffering from more than one illness or disability and in varying degrees. But
it is used little in the Disease Control Priorities Project that focuses on
burden of disease in low- and middle-income countries.
Some
Shortcomings of Cost-Effectiveness Analysis
Cost
data can be extremely hard to find in developing countries. Ideally,
cost-effectiveness analysis should include direct costs (such as doctors’ or
nurses’ time and supplies used) as well as indirect costs (such as a portion of
administrative costs). The cost of equipment also needs to be spread across its
many uses. These costs are usually not readily available, however, and thus the
costs of interventions reported in developed countries are often used and
adjusted for developing-country settings. Alternatively, a study conducted in
one low-income setting is sometimes used to estimate costs in all or several
low-income countries.
Cost-effectiveness
is only one criterion for judging whether an intervention has merit.
Policymakers also must take into account total cost and whether an intervention
is affordable at all, the capacity of the system to deliver it, and whether people
will demand and use the service provided. Also, costeffectiveness
analysis might show that an intervention is worth doing, but it doesn’t
necessarily mean that the public sector should undertake all of it.
Private-sector services might be available and affordable for some portion of the
population.
Equity
is also a concern, because it can be more cost-effective to serve many people
in large urban centers, where the cost per outcome is relatively low. Providing
the same service in a poor, rural area—where fewer patients are seen or where
staff and other inputs are harder to make available—might be less cost-effective
but more worthy of public investment because it is more equitable.
What
About Benefits Other Than Health?
Cost-effectiveness,
whether expressed as a single health outcome or a DALY, only takes account of
health benefits, which means it may underestimate the total benefit of some
health interventions that also improve people’s productivity and other aspects
of quality of life. Piped water and sanitation, for example, bring environmental
as well as health benefits to communities, and save people’s time. Some
interventions may not appear costeffective on health
grounds alone but may be justified by large non-health gains.
Estimating
the value of all benefits of an intervention, including both health and
non-health outcomes, requires expressing gains in monetary terms because
“apples and oranges” such as lifeyears, income, and
better school performance can’t be added together. Such comparisons are the
domain of cost-benefit analysis, where the aim is to compare the total gains
from different investments.
The
preference for cost-effectiveness analysis in health stems in part from concern
about the ethical implications of placing a monetary value on people’s lives.
In cost-benefit analysis, the value of life years is most often expressed in
terms of income lost or gained, which can be impractical to compute and also problematic
to justify when addressing the needs of vulnerable (old, young, or
disadvantaged) populations.
Improvements
Needed
More
and better data are needed in low- and middle-income countries so that analysts
do not need to use cost data and assumptions from high-income countries or rely
on expert judgments. The need for information starts, in some cases, with better
estimates of the incidence and prevalence of particular diseases, and with data
on the coverage and outcomes of health interventions. In most countries,
estimating what it would cost to expand the coverage of existing interventions
or to add new interventions relies heavily on assumptions.
Whenever
possible, cost-effectiveness analyses should be conducted at the national or
subnational level. This would allow planners to take full account of all the
reasons cost-effectiveness varies from one place to another, and to develop
priorities on the basis of analysis appropriate to local circumstances.
Conclusion
While
many considerations, such as affordability, equity, and non-health benefits,
may factor into decisions about health spending, cost-effectiveness analysis is
an essential tool for decision makers. It can guide decisions about where best
to spend limited resources and what to include in a package of health services
that responds to a population’s greatest health needs.