Technology Assessment, Outcome Analysis Research, Comparative Effectiveness, and Evidence-Based Radiation Oncology

Carlos A. Perez, Edward C. Halperin, and Yolande Lievens

There is worldwide concern about rising medical costs. The problem is particularly acute in the United States, which has led to multiple efforts to institute health care reforms. Health care expenditures constitute about 17% of the U.S. gross domestic product, compared to 6% to 11% for Canada, Germany, Japan, and the United Kingdom.¹ The cost of administration of U.S. health care is approximately one-fourth of the total national expenditure on health care. Health care administrative costs in the United States exceed those in many other developed countries. In 2002 the administrative cost of the federal Medicare program was 3% of the program’s total budget, whereas the administrative costs of the federal and state Medicaid programs was 6.7% compared to 12.8% of total revenues for private health insurance plans. One reason for the low administrative cost of Medicare is its lack of advertising and marketing expenses. Medicare pays for roughly 1 employee for 10,000 beneficiaries, whereas most large private insurers hire 15 or more employees per 10,000 enrollees. The Kaiser Foundation Health Plan, one of the largest and best- known group-health maintenance organizations (HMOs), has self-reported administrative costs of 4%.²

According to the 2011 report of the Organisation for Economic Co-operation and Development, the average annual growth rate in health care spending between 2000 and 2006 ranged from 1.4% in Germany to 10.7% in Korea. With health expenditure of $6,714 per person per year, the United States largely outspent the other countries surveyed.³

Borger et al.⁴ projected that in 2015 the United States will spend >$4 trillion, about 20% of the gross domestic product, on health care. Over the last 30 years overall inflation in the United States was 106%, whereas health care costs grew by 251%.⁵ It is estimated that >$100 billion are devoted yearly to research, development, and regulatory approval of new technologies, but <$1 billion is spent on assessment of these new technologies.⁶ Concern over the magnitude and continued growth of expenditures for health care in the United States has led to economic pressure on health service providers to contain costs.

Powerful forces affect the assessment of new technologies in radiation therapy. Inventors and developers, medical equipment manufacturers, corporate stock holders, and equipment salespeople seek rapid entry of new technologies into the marketplace at a minimum cost for as rapid a financial return as possible. Radiation oncologists desire new technology either for a real or perceived improvement in health care, career development, competitiveness with other practitioners, and/or personal profit. Patients and family members hope to see an improvement in clinical outcomes. Third-party payers seek cost reductions or stabilization, stable budgetary expenditures, and a plausible explanation of the “medical necessity” of the new technology. Finally, policy makers seek to balance the competing pressures that they feel from medical equipment manufacturers, physicians, and patients and the overall need for cost containment.⁷

Historically, regulatory approval of new drugs and devices was reliant on well-funded developers with a significant revenue stream conducting and funding randomized, prospective clinical trials. Many new technologies in radiation therapy, however, are introduced by small startup companies that are smaller, less well capitalized, and often single-product vendors. They are not capable of funding randomized, prospective trials and they have a powerful financial impetus to bring their product to market as quickly and as profitably as possible.

In 1985 the Blue Cross Blue Shield Association adopted criteria that required “adequate scientific evidence” to determine that technology improved health outcomes. This approach quickly became the norm among public and private payers. In an interesting development, in 2005, the Centers for Medicare and Medicaid Services (CMS) posted on its website draft guidelines describing a new approach to the determination of coverage policies called “Coverage with Evidence Development.”⁷ This proposal argued that Medicare coverage of promising technologies or services could be linked directly to a requirement that the beneficiary be a participant in a trial or registry and that the CMS had statutory or regulatory authority for that requirement. In the United Kingdom there was a similar proposal for funding “only-in-research” programs. Such programs were used for the evaluation of laparoscopic surgery for colorectal cancer, photodynamic therapy for age-related macular degeneration, the National Emphysema Treatment Trial to determine whether lung reduction surgery was superior to pulmonary rehabilitation without surgery, a comparison of balloon angioplasty plus carotid artery stenting versus carotid endarterectomy in patients at high risk for stroke, and the role of positron emission tomography scanning in suspected dementia.⁸

ECONOMIC EVALUATION IN HEALTH CARE

Economic evaluation is routinely becoming an important criterion for evaluation of therapeutic strategies. Unfortunately, consistent criteria are not followed for evaluation of the economics of treatment, which reduces the usefulness and credibility of comparative cost and other financial data/statistics across studies or over time.⁹

Health technology assessment (HTA) plays an essential role in modern health care by supporting evidence-based decision making in policy and practice. HTA involves addressing five questions¹⁰:

1. Can a health care intervention achieve its expected goal when used in optimal circumstances? (Efficacy)

2. Does the intervention do more good than harm when used in routine practice? (Effectiveness)

3. What is the balance between the health outcome obtained and the resources required to deliver the intervention? (Efficiency or Cost-Effectiveness)

4. Is the supply of services matched to locations where they are accessible to persons that need them? (Availability)

5. Who gains and who loses by choosing to allocate resources to one health care program instead of another? (Distribution).

A full economic evaluation typically involves quantitative examination of both costs and outcomes, or consequences, of competing interventions possible, including no intervention. An appropriately performed economic evaluation is incremental, that is, it measures the extra cost incurred to obtain the incremental improvement in outcome. Drummond et al.¹¹ identified different types of full economic evaluation based on the denominator chosen (Table 98.1). The most widely used approach, cost-effectiveness analysis (CEA), compares the incremental costs of a new technology or intervention to the incremental gain or loss of clinical outcome of the new intervention, ideally measured in life-years gained (LYG). Other potential denominators in a CEA are disease-free survival or number of cancers prevented or detected, but the use of such intermediary outcome measures hampers comparison among different CEAs and should therefore be discouraged. The evaluation will result in a ratio of the cost/outcome, for example, cost/life-year gained ($/LYG, €/LYG), called the incremental cost-effectiveness ratio.

TABLE 98.1 LEVELS OF EVIDENCE AND GRADING OF EVIDENCE FOR RECOMMENDATIONS

When the outcome of the evaluated interventions is expected to be the same, the economic evaluation will be limited to a cost comparison called cost-minimization analysis (CMA). Assuming a well-defined comparison of two radiation therapy (RT) approaches and a sufficiently long follow-up, prolongation of life, life-years gained, can be measured, but it can be questioned whether this is the metric that has most relevance to patients. The third type of full economic evaluation, cost–utility analysis (CUA), acknowledges quality of life after anintervention by weighting the LYG with a “utility factor” typically ranging from 0 (death) to 1 (perfect health). For example, if a patient had a utility or preference for a health state of 0.5 and he survived in that state for 1 year, then he would have had a quality-adjusted survival of 6 months. A CUA employs quality-adjusted life years (QALYs) in the denominator, with the result expressed as cost/QALY. Utilities or patient preferences for a certain health outcome can be measured with instruments such as the Health Utilities Index III¹² and EuroQol¹³ or tests such as Time-Trade Off or Standard Gamble.¹⁴ CUAs are helpful in trying to compare nonsimilar health interventions, such as a prostate cancer–screening program with a child immunization program, for example.

The last type of full economic evaluation is cost–benefit analysis, in which both cost and outcome are valued in terms of currency. Where the benefit in monetary units is greater than the cost, the medical intervention is considered worthwhile. Putting a dollar value on the quantity and quality of a life is, however, a highly specialized and controversial area. Full economic evaluation in RT, requiring detailed input on short- and long-term costs and outcomes, can be difficult to perform. One such approach addresses costs and outcomes separately and leads to cost and outcome descriptions, respectively.¹¹ However, there are dangers with separating costs and outcomes, and one has to be cautious that the information generated will not be misused.

The most useful descriptions of outcome of a new treatment strategy, particularly from the patient’s perspective, result from the conduct of clinical trials. Randomized and controlled phase III trials have been categorized as efficacy or effectiveness evaluations.¹¹ It can be argued, however, that such comparisons may indirectly lead to the increased cost of health care as funders come under pressure to broadly adopt new interventions with only marginal clinical benefit and no consideration of the opportunity cost. A pure cost comparison, in contrast, entails the opposite risk. If the tendency to reduce health care spending is strong, this might result in the simple choice of the cheapest approach, which may, in turn, lead to reduced quality, hence inferior outcome. Provided that there is sufficient evidence that the outcome of interest for the experimental treatment or intervention does not differ from the standard treatment, a CMA is undertaken (discussed earlier), that is, a mere cost comparison, with the intervention resulting in the lower cost being the favored one from a health economics’ perspective.

In evaluating medical care, the term efficacy is not interchangeable with effectiveness. The maximum possible reduction in a disease due to the use of medical intervention is properly termed efficacy, which is generally measured with randomized, controlled trials, which will achieve the maximum possible benefit when patients are carefully selected and stratified, randomization arms are properly designed, compliance is high, and trial participants are often free of other diseases or conditions that might interfere with the intervention being evaluated. The benefits established in a randomized, controlled trial are only partially applicable to day-to-day practice in a general population. The value of a medical intervention in the general population is termed effectiveness. The difference between efficacy and effectiveness can be very large, and obtaining a realistic measurement of effectiveness is often difficult.¹⁵

There are three approaches to health economic evaluation for comparing two therapies: (a) cost minimization, in which one assumes or observes no difference in effectiveness, (b) incremental cost-effectiveness, and (c) incremental net benefit. The last can be expressed in units of either effectiveness or costs. When analyzing data from a clinical trial, expressing incremental net benefit in units of cost allows the investigator to examine all the approaches in a single graph, complete with the corresponding statistical inferences.^15a

A computerized literature search of >14,000 articles published since 1994 having cost as a keyword was carried out. Criticisms of these costing studies focus on three areas:

1. Costs are not measured correctly^16–18 because most studies use charge or reimbursement dollars as a proxy for cost. This is an incorrect measure because neither value is representative of the resources consumed or foregone in the process of providing a good service; hence, it is not a cost; the former figure is a construct, and the latter is the result of a negotiation between a payer and a provider.

2. Costs are not compiled using a proper perspective and scope.^17,19 Most studies only tabulate the cost of the health care provider and not to the patient, the family, or society.

3. There is a lack of consistency in reporting results,^16,20,21 which prevents interstudy and interinstitutional comparison.²²

As a result of these problems, the data and conclusions from many cost-based studies are suspect.¹⁹

Clearly, a better costing method is required. Russell²³ emphasized that to serve its purpose, a model must produce accurate predictions and low probability for substantial variation in the factors that influence costs and effects. He identified three aspects of modeling: validating effectiveness estimates, modeling costs, and the implications of common statistical forms. Validation procedures similar to those for effectiveness estimates are proposed for costs. Modelers need to pay more attention to ensuring that the events described by a model represent costs and effects. Modelers can also help improve the epidemiologic and clinical research on which cost-effectiveness analyses depend by showing the implications for resource allocation of the statistical forms conventionally used in these fields.

Comparative effectiveness research (CER) aims to improve the quality, effectiveness, and efficiency of health care and to help patients, health care professionals, and purchasers make informed decisions regarding various treatment options, particularly for chronic illnesses or cancer. CER is moving forward, with recently defined priorities and a newly funded Patient-Centered Outcomes Research Institute, which, it is hoped, will survive cost cutting in the U.S. Congress. Economic analyses are most valuable to health policy analysts and health care managers who must allocate resources and establish benefit packages. An economic health care analysis tries to directly relate the incremental cost of an intervention to its potential benefit. The intervention is always evaluated relative to an alternative form of treatment.⁹

Technology assessment, cost–benefit outcome analysis, clinical trials supported by evidence-based medicine, and comparative effectiveness together can enhance the rationale and quality of medical care provided to patients at a competitive cost.

TECHNOLOGY ASSESSMENT

Technological change is the key driver of health care expenditure growth because it is generally associated with increased rather than reduced cost. As the economist Henry Aaron is quoted as saying, “Rapid scientific advance always raises expenditures even as it lowers prices. Those who think otherwise need only to turn their historical eyes to automobiles, airplanes, television, and computers. In each case, massive technological advance drove down the price of services, the total outlay soared.”² A good example in medicine of this phenomenon is laparoscopic cholecystectomy. Although the price of a laparoscopic procedure is less than the price of an open cholecystectomy, after the introduction of the new technology the rates of both types of surgery increased, and the growth in the quantity of services overwhelmed the impact of per-unit reduction in price. Similar examples can be found in the widespread use of magnetic resonance imaging, computer tomography, coronary artery bypass grafting, angioplasty, cardiac intensive care units, neonatal intensive care units, and positron emission tomography and the growth of radiation oncology facilities. Technology may also increase costs by either premature adoption of the technology without adequate evidence that it improves outcome or overuse of technology, regardless of value, to provide a competitive market advantage or to generate revenues.²⁴

Technology assessment is undertaken to determine (a) the appropriateness of adopting a new procedure consistent with the health care organization’s mission and strategic plan; (b) the safety, efficacy, and cost-effectiveness of the new technology; (c) distinctions between appropriate and inappropriate use for the new technology; and (d) quality improvement methods to optimize the use of the new technology.²⁵ Technology assessment is a complex process that involves financial considerations, therapeutic outcome (locoregional tumor control, amelioration of symptoms and survival), treatment-induced morbidity, quality of life, impact on caregivers, family, and coworkers, and lost-opportunity costs.

When new technology emerges and priorities for allocation of increasingly scarce resources are considered, it is imperative to carry out economic analyses and also to assess the positive effects to the population who will benefit from the new technology compared with standard treatment techniques. The U.S. Center for Medicare Management concluded that cost-effectiveness should be considered in approving reimbursement by Medicare for new technology procedures.

Halperin²⁶ pointed out the importance of technology assessment in determining the merits of new devices or modalities and the impact they may have on the cost of health care. In a provocative editorial, Lee²⁷ pointed out that although technology is ubiquitous in radiation oncology, it is our professional responsibility to critically evaluate the merits of new technological developments. Yet, technology assessment can be a thorny and complex subject. Many of the technological advances in radiation oncology have not undergone a strict test of worthiness in formal clinical trials. Simulation, positioning techniques, computed tomography scanning for treatment planning, online imaging, and even intensity-modulated radiation therapy have had multiple dosimetric validations, but no randomized trial has documented their impact on outcome of large numbers of patients. However, lack of this evidence would not lead us to discontinue any of these procedures, which have been shown to improve in some way (i.e., less toxicity) the quality of care provided to patients.

The profusion of new technologies in medicine is related to the number of specialists in an area. Specialists receive income from new technologies and prompt hospitals to invest in them. A hospital that is constructing innovative facilities may attract specialists to the community and improve its revenue. It is also likely that public attitudes are influenced by medical providers and suppliers that advertise new technologies through the mass media.

Many nations have agencies to conduct health technology assessment, and there is an international network of agencies for health technology assessment that can share such data. The United States, however, has no national coordinating body on health technology assessment. Assessments are conducted through organizations such as the Veterans Administration, the Blue Cross Blue Shield Association, and professional organizations like the American College of Radiology, the Medicare Coverage Advisory Committee, and HMOs.

Health care organizations use a variety of models to assess and acquire technology, some of which include outcome. Reliable data for such benchmarking is frequently scarce. Some compare new and emerging technology against existing technology, although reliable outcome data are rare. Most institutions or individuals buy what physicians, advocacy groups, philanthropists, or governing boards want and use manufacturers and physicians as the primary sources of technology information. Cowan and Berkowitz²⁴ pointed out several key mistakes that occur during the diffusion of technology:

1. Technologies are adopted simply because they are new, without assessing their impact on outcomes.

2. Equipment is replaced solely because of its age. To avoid overuse of technology, health care organizations must carefully match the capacity of existing technology with projected utilization in a world of limited resources.

3. Applications of technology expand without adequate justification based on outcome or cost considerations.

4. Numerous vendors are used, and numerous models of products with duplicative specifications are purchased. Reducing the number of brands and models of devices used may provide an opportunity to reduce purchasing and inventory costs, as well as maintenance and repair costs. On the other hand, competitive pressures in the marketplace may provide downward pressure on equipment prices and argue in favor of the existence of multiple vendors.

5. Economies of scale in regionalizing technology decisions are missed because of lack of consolidation of services. This is sometimes related to inadequate systems for patient transport to treatment centers.

6. Duplication of services sometimes adds cost without significant increase in work volume.

A number of studies on the costs of treating cancer patients with radiation therapy^28–31 used either relevant costs defined in a number of ways or different proxies for actual costs (billed charges, revenue received, Medicare allowable). Recent reviews illustrate the wide variety of cost information and cost methods used in the literature and call for a consistent method for estimating site- and intent-specific treatment costs.^{9,18,19,22,32} In estimating costs of health care interventions, several decision tools, including cost analysis, cost-effectiveness, and cost–utility and cost–benefit analysis, seek to define a unit of health outcome per unit input of cost. Before employing any of these decision tools, proper definition and calculation of cost is essential. The literature on the cost-effectiveness of particle therapy for cancer is scarce, studies are often not comparable, and many studies are not performed according to standard health assessment criteria. Evidence on the cost-effectiveness of particle therapy is, therefore, limited. One would hope that adequate reimbursement can be found to support innovative yet costly treatment to learn whether it is truly more efficacious.

TYPES OF COST IN HEALTH CARE

Costs During Treatment

Direct medical costs include the value of all the goods and services consumed in the provision of an intervention or dealing with the side effects of treatment. These costs include the consumption of all capital and human resources such as professional time, diagnostics, treatment, and medications.

Direct non–health care costs include all costs as a consequence of undergoing a treatment but not related to the treatment itself. Examples are the cost of transportation while patients are receiving treatment. Other costs sometimes included are changes in use of informal caregiver time (including the cost of family or other providers to provide health care in the home or continuous nursing care for disabled individuals) and patient time costs (referring to the time patients spend receiving care, including the value of the time consumed in receiving the treatment as well as travel and waiting time).

Indirect medical costs include all costs in the time after the treatment has been terminated and include costs of late toxicity or related diseases. In RT, these costs can differ quite substantially between different approaches if the side effects are very different or if one of the interventions has a higher curative potential. Typically, however, these costs occur many years after termination of the treatment.

Indirect nonmedical costs refer to costs that are important for society, in that they measure loss of productivity due to long-term disability or premature death. Costs to the family or society can be substantial. These include the patient’s travel costs, the patient’s or care provider’s time off from work, and the considerable strain on caregivers. There may also be “downstream” effects. If a caregiver is devoting time to an ill spouse, he or she is taking time away from work or child care responsibilities, for example.

Global cost estimates of the cost of RT are the aggregate costs of the human and capital resources consumed in the delivery of a RT service. They do not contain the fine detail required when comparing, say, one RT protocol with another. However, they can have value as a background to allocating national health care budgets between different classes of medical intervention.

The data show that even in countries with optimal infrastructure, the budget assigned to RT is roughly 5% of the total amount spent on cancer care.³³ In the early 1990s, the European Union estimated the average cost per course of RT at about 3,000€, compared to roughly 7,000€ and 17,000€ for surgical and chemotherapy treatments, respectively. However, in spite of the perception of high capital costs, it is the personnel cost that dominates the cost of the provision of the service (about 70%), with capital investment typically representing <30% of the total RT budget.^3,14,34

Reimbursement-Based Costs

Whereas economic evaluation theoretically advocates the use of cost figures that reflect actual resource consumption, this is frequently not achievable, and charges for service are used as a proxy. It should, however, be realized that large discrepancies may exist between the reimbursement of treatments and the costs of resources consumed while delivering a treatment. Due to the fact that reimbursement often lags behind the introduction of novel technologies, in many countries there is a tendency for high-tech treatments to be underreimbursed. The inverse seems to hold for the United States, where the reimbursement is likely to be higher for new technologies, although it decreases with time.

Besides numerical differences between costs and charges, other, more practical and theoretical problems may arise when utilizing reimbursement figures for economic evaluation. This was observed by the Radiation Therapy Oncology Group when performing an economic evaluation based on combining clinical trial data with cost data obtained from Medicare for each patient.³⁵ The first problem was that Medicare-managed care products do not have individual claims because managed care companies receive capitated payments and therefore may not record and/or report individual claims to Medicare. A more accurate estimation of costs would result from using administrative claims data to calculate costs, provided that they contain all claims of care a patient received. Another potential problem is that an underestimation of costs could occur if codes with only a cancer diagnosis were used, thus neglecting costs related to treatment toxicity or indirect medical costs occurring in the further follow-up of the patient. It may also be difficult to generalize the results of an analysis based on Medicare patients older than 65 years to the general population. Finally, it is apparent that obtaining administrative claims data from Medicare can be a costly procedure in itself.³⁶

ACTIVITY-BASED COST ANALYSIS

Activity-based costing (ABC) is a system widely used in manufacturing but rarely in health care. ABC dissects the process of providing patient care into a series of microlevel tasks and assigns costs to these tasks. As a result, ABC directly identifies resources consumed by cost objects (treatments, patients, or health care providers).¹² Tabulating these tasks and resources consumed allows for comparison of cost values across institutions and time in a consistent and credible way. From this, the economic impact of differences in operating policies can be quantified and the comparison of cost to health outcome will be improved.

The traditional cost models that hospitals have used are outdated and ill equipped to provide accurate information on costs of a specific medical procedure or global cost of medical care for a patient from initial diagnosis to death. Cost models break organizations into departments for financial oversight and budgetary control; the focus of these cost models is departmental budgets, and they are not specific for a patient or a procedure, when it is really patients and procedures, not the department, triggering costs. For example, the accounting system of a hospital may be able to provide a fair assessment of the total budget (cost) of its radiation oncology department. However, the same system is of little help in assessing the actual cost of treating a patient with, for instance, prostate cancer using radiation therapy. This lack of applicability of cost information at the patient or the procedure level often leads to faulty decision making because analysis of the whole department does not necessarily translate into data about an individual patient.

FIGURE 98.1. Process flow chart. Activity map and corresponding mean (median) times for definitive irradiation of cancer of prostate. A: Evaluation and management. B: Treatment planning. C: Weekly management

ABC is concerned with measuring resource consumption by an individual patient. There are two general advantages of using ABC: increased accuracy of cost measurements and transparency of cost-incurring sections. The latter advantage—increased accounting transparency—means that there is a more lucid and readily apparent correlation between costs and actions. This allows managers to evaluate each step of the process, perceive inefficiencies in the system, and quantify economic benefits of process improvements.

The cost data of any technology should reflect the facilities and equipment costs (including depreciation and replacement), space (including housekeeping and utilities), and recorded time and effort of all professional staff on hospital payroll treating patients. Specific technical approaches should be identified, for example, in patients undergoing definitive radiation therapy, standard two-dimensional (2D) or three-dimensional (3D) conformal RT (CRT) or intensity-modulated RT (IMRT) or image-guided RT (IGRT). Time and effort observations for multiple tasks must be recorded by the respective staff members as they take place. The numbers of observations provide a representative sample of treatment procedures.

Technical cost is defined as the actual expenditures recorded in hospital accounting records. The total technical cost of a course of therapy includes staff time and effort, compensation including fringe benefits, space, utilities, maintenance, equipment depreciation, supplies, support services, liability cost, and administrative overhead. The ABC method^37–40 was used to derive technical (hospital) resource utilization costs for treatment procedures in treating localized prostate cancer.^9,30 The costing method consists of four major steps:

1. Creating a process flow map that identifies and defines all of the activities and steps associated with the treatment of patients with standard irradiation or with 3D CRT (Fig. 98.1).

2. Collecting and recording the staff time expended on each activity.

3. Identifying and accumulating all hospital costs from the general accounting ledger system related to the patients treated at a radiation oncology center in a given period.

4. Allocating actual cost to each activity to derive total specific costs of technical resource utilization for a course of 2D or 3D CRT, IMRT, or IGRT.

An example of an ABC cost-calculation model is the one developed in by Lievens et al.³⁴ Because it was felt that current RT protocols, with a shift toward more complex treatment planning and delivery techniques and more thorough quality assurance procedures, were no longer accurately captured within the original model, a novel version has recently been developed.⁴¹ Because evolving technology over the last decade was more complex, it was found that the introduction of IMRT compared to 3D CRT translated into an almost doubling of procedure costs. ABC analysis allowed the authors to identify exactly where these extra costs were incurred. It was shown that the cost increase was the consequence not of the IMRT technique as such but of the combined effect of IMRT using more fields and delivering higher fraction numbers and, very important, the introduction of more, often daily, online imaging for position verification. If, however, techniques such as IMRT or gated radiotherapy delivery make it possible to reduce the number of fractions using simultaneous integrated boosts or accelerated hypofractionation, the resulting costs may decrease while yielding potentially improved treatment outcomes.³⁶

Time and Effort Analysis

An integral phase of ABC is identifying process flow, listing each individual involved in the process and recording the actual amount of time spent to perform an activity by the staff using electronic or paper daily activity logs. Determining the participant’s task time should follow the monopolizing rule: timing begins when a specific patient requires the participant’s attention and stops when he or she is no longer doing so. Whether or not the patient is still present in the activity area is immaterial. For example, for an office visit, the nurse’s time to prepare the patient’s paperwork should be attributed to the patient, even before the patient arrives or after he or she has left the office. The actual activity time may be adjusted upward to recognize some normal unproductive time of activity participants, such as time spent in work breaks. Tables are generated to show the time and effort involved in each of the functions necessary to register and evaluate the patient, perform treatment planning, deliver external-beam irradiation, and supervise the management, as well as in preparation of the appropriate records to document all information, including quality assurance requirements.

Times are collected for each activity in the treatment process beginning with the initial consultation through the first follow-up visit after full treatment. These data points are analyzed and used to generate average times for patients treated with a specific technique along with the standard deviation of times.

RESOURCE UTILIZATION COSTS

Four types of resources should be catalogued and measured for each task:

1. Activity participants (e.g., radiation oncologists, physicists, therapists, dosimetrists, nurses, receptionists)

2. Physical resources (e.g., space, equipment, machines)

3. Supplies (e.g., x-ray films, office supplies)

4. Support services (e.g., administration, centralized information system)

Patient-related expenses include salaries and fringe benefits of the staff, Social Security contributions, retirement annuities, health, life disability and long-term care, dental insurance, tuition and childcare benefits, and disability insurance. Average hourly salary and related costs for each staff member are computed.

There are many supplies, such as office paper and latex gloves, for which the cost is trivial, and it is not economically worthwhile to track their specific use for individual patients. An average cost rate can be computed for such supplies by dividing their total cost by the total number of patients, which is applied to each patient. However, supplies used only by patients with specific diseases or procedures (e.g., contrast material, immobilization devices) need to be recorded for individual patients.

For hospital-based departments, indirect operating costs are classified into general hospital operating expenses and administrative overhead, which includes hospital liability insurance and administrative costs such as shared activities (administrative personnel for purchasing and payroll functions, human resources, public relations, and financial operations). The allocation of indirect operating costs among different departments of the hospital is complex. Models include allocating them as a proportion of personnel costs and many others. The allocation of cost among various units in a hospital can also be the source of pitched battles between different units. In many hospitals allocation of costs is a mechanism of cross-subsidization among different departments (e.g., the surgery department subsidizing primary care). Administrative cost can be determined by computing prorated allocation, dividing the total hospital indirect expenses by the proportion of the radiation oncology center technical budget to the total hospital annual budget.

Technical equipment expenses for treatment of patients are included using American Hospital Association standard depreciation schedules. The cost of depreciation of all equipment involved in the management of these patients, including treatment planning and delivery, should be accounted for on a prorated basis.

COST BENEFIT

If all that matters is minimizing expense, then the lowest-cost treatment (or no treatment) would be preferred. However, cost benefit, which incorporates the incremental impact of a new technology on outcome, and cost utility, which also relates additional costs to impact on survival as well as on the patient’s quality of life and productivity, are important parameters in the assessment of new technology.⁹ The projected cost of treating a patient who initially has control of the local tumor and no distant metastases is about one-third of that for a patient who develops a treatment failure.⁴²

As a concomitant to an increasing proportion of the health insurance market being controlled by managed care, with its cost-cutting pressures, there may be a tendency to de-emphasize the importance of quality of care. However, it is critical, for instance, to document the alleged decreased morbidity of treatment with 3D CRT, IMRT, or protons in comparison with treatment of patients with carcinoma of the prostate⁴³ with conventional external-beam radiation therapy. This benefit, if it is real, is of great financial significance because treatment of major therapy complications may be costly.

Owen et al.⁴⁴ conducted economic studies for two phase III Radiation Therapy Oncology Group studies that compared different irradiation fractionation schedules (1-04 and 90-03, brain metastasis and head and neck cancer, respectively). Expected quantities of current procedural terminology codes and relative value units (RVUs) were modeled. Institutions retrospectively provided procedure codes, quantities, and components, which were converted to RVUs used for Medicare payments. The median and mean RVUs were within the range predicted by the model for all arms of one study and above the predicted range for the other study. The model predicted resource use well for patients who completed treatment per protocol. Actual economic data can be collected for critical cost items. Some institutions experienced difficulty collecting retrospective data, and prospective collection of data is necessary to validate these studies.

We must move from a fee-for-service mentality to a more critical and fiscally responsible health care system delivering high-quality care at a competitive cost. Based on current trends, 70% of hospital revenues will be capitated in a not-too-distant future. A challenge for future health care initiatives will be to strive for the lowest (optimally speaking, most competitive) cost with the highest value for the patient, including decreased morbidity of treatment and enhanced quality of life.⁴⁵

The potential contribution of more complex and expensive procedures to the outcome of cancer therapy needs to be carefully evaluated.⁴⁶ Obviously, a significant portion of the increase in health care costs is linked to new technology; a great deal of this technology is welcome, but cost benefit should be demonstrated. Some technologic advances improve health care and can be justified economically if we document a positive impact on tumor control, patient survival, morbidity of treatment, and quality of life. The efficacy of diagnostic and therapeutic procedures can be related to outcome studies, and costs should be carefully evaluated based on both monetary and health-related considerations. Doubilet et al.⁴⁷ cautioned against the indiscriminate use of “cost-effectiveness” as the criterion on which medical decisions should be made, and thus we must carefully define the endpoints on which these studies are based.

Models can be constructed using a Markov model and available data on the natural history of a tumor, prognostic factors, efficacy and morbidity of a given therapeutic modality (based on outcome analysis), and reimbursement data for treating a specific patient population. When patterns-of-failure data are used, the cost differential can be calculated for a patient treated successfully in contrast to one who fails in a specific anatomic site (locoregional failure, distant metastasis, or combination), incorporating expenditures related to management of the treatment failure. The model can be made more complete (and complex) by adding costs of the morbidity of therapy. With the information and epidemiologic data currently available in the United States, annual costs of care can be computed and compared for various therapeutic modalities. Using Monte Carlo simulation, sensitivity analysis of cost-effectiveness may be performed.

Total cost-of-care projections should be carried out using a Markov-type analysis based on clinical experience, which will provide more accurate cost data for different management options. Providing cost–benefit information to patients (consumers) will be necessary when evaluating diagnostic and treatment options, particularly in carcinoma of the breast and prostate, in which great controversy exists regarding the merits of various modalities.

This information also will be very helpful in contract negotiations with health care organizations and third-party payers and in setting policies for establishing therapy guidelines and justification of health care expenditures.

Cost–utility analysis calculates the value of an intervention as the ratio of its incremental cost divided by its incremental survival benefit, with survival weighted by utilities to produce QALYs. Many studies are at variance with current standards; only 20% of studies took a societal perspective, more than one-third failed to discount both the costs and QALYs, and utilities were often simply estimates from the investigators or other physicians. There remains much room for improvement in the methodologic rigor with which utilities are measured. Considering quality-of-life effects by incorporating utilities into economic studies is particularly important in oncology, when many therapies obtain modest improvements in response or survival at the expense of nontrivial toxicity.

Stinnett and Mullahy⁴⁸ introduced the concept of net health benefit as an alternative to cost-effectiveness ratios for the statistical analysis of patient-level data on the costs and health effects of competing interventions. Net health benefit addresses a number of problems associated with cost-effectiveness ratios by assuming a value for the willingness-to-pay for a unit of effectiveness. Willan⁴⁹ extended the concept of net health benefit to demonstrate that standard statistical procedures can be used for the analysis power and sample size determines cost-effectiveness data. He showed that by varying the value of the willingness to pay, the point estimate and confidence interval for the increment cost-effectiveness ratio can be determined.

The benefit of 3D CRT, IMRT, or IGRT is hypothetically linked to improved local tumor control because of better coverage of the target volume with a specific dose of irradiation, less acute and late morbidity, and possibility of carrying out dose-escalation studies if morbidity is held to an acceptable level, resulting in improved survival.⁴² Several institutions, the Radiation Therapy Oncology Group, and 10 institutions under cooperative agreement with the National Cancer Institute conducted phase I/II dose-escalation studies in carcinoma of the prostate and are carrying out a study on carcinoma of the lung. Depending on the results, the cost benefit of 3D CRT, IMRT, or IGRT must be further evaluated in dose-escalation and in larger multi-institutional phase III studies, comparing it with standard techniques to justify its somewhat higher initial cost. Cases that are most likely to benefit from 3D CRT, IMRT, or IGRT include patients with (a) tumors in sites with complex anatomy, (b) irregular-shaped tumors, (c) tumors adjacent to radiation-sensitive normal structures, (d) tumors that undergo significant changes in volume during treatment, and (e) small-volume or high-dose treatment.

The introduction of protons and heavy ions (hadrons) into the radiation therapy armamentarium has brought additional controversy in technology assessment because of the higher cost of facilities, equipment, and operation, with diverse opinions as to the cost-effectiveness of these innovative modalities. A 2008 review of the literature showed that only 17 publications (out of 777 on particle therapy) dealt with economic aspects, and only 5 of them were on cost-effectiveness.⁵⁰

Many articles have been published on the physical advantages of protons or heavy ions in comparison to photons, resulting in better dose distribution in the target volume with less radiation administered to normal tissues, which may decrease morbidity and eventually the risk of second malignancies.^51,52 However there is a lack of randomized trials documenting superior clinical outcomes with these particles.

Opinions on the need for or ethical considerations to subjecting patients to prospective trials have been expressed.^53,54 Perrier et al.⁵⁵ used an ABC approach similar to the examples discussed previously to evaluate the cost of carbon-ion therapy. It allowed an in-depth definition of the cost of treatment preparation and the actual irradiation, depending on various parameters such as actual treatment time and patient setup time. The approach of Peeters et al.,⁵⁶ conversely, was more aggregate, using a spreadsheet model that assumes that treatment costs scale linearly with the number of fractions. This, obviously, is a somewhat simple approximation of reality, which results in less accurate estimates.³⁴

Implementing new technologies in radiation oncology with advanced engineering, such as linear accelerators, cone beams, tomotherapy, treatment planning and delivery techniques, and accessories that enhance the efficiency and accuracy of operation, may decrease overall cost of treatment computed over the lifetime of the patient.

The American Society for Therapeutic Radiology and Oncology maintains the position that new technologies and modifications of existing technologies representing significant paradigmatic shifts in treatment approach should be implemented into clinical practice in such a manner as to ensure safety, efficacy, and, ideally, cost-effectiveness.⁵⁷

EVIDENCE-BASED MEDICINE AND EVIDENCE-BASED ONCOLOGY

There have been significant scientific advances in biologic sciences and health care. At the same time, there has been increased accountability and in some instances attempts to ration services; as a mechanism to accomplish this, arbitrary clinical practice guidelines have been promulgated. A more rational effort to define optimal heath care should be based on identification of innovative approaches, outcome analysis in properly designed clinical trials, and careful assessment of current practice; more important, we have a dire need for solid and credible clinical research and evidence-based decision making in medicine.

The growth in basic and translational research data to guide medical practice has made it critical for clinicians to appraise and use published data for medical decisions. Evidence-based medicine (EBM) is the compelling idea that any recommendation of a specific medical procedure should be supported by empirical evidence for its superior efficacy/toxicity ratio compared with alternative treatment options in a patient with a given presentation.^43,58,59

Evidence-based medicine helps correct an imbalance in contemporary medicine in which clinicians are being trained to maintain high standards of critical consciousness in methodologic domains but not in the broader historic and sociocultural domains that surround them.⁶⁰

Evidence-based health care is the conscious use of current best evidence in making decisions about the care of individuals or the delivery of health services. Current best evidence is up-to-date information from relevant valid research about the effects of different forms of health care, the potential for harm from exposure to particular agents, the accuracy of diagnostic tests, and the predictive power of prognostic factors.

Evidence-based clinical practice is an approach to decision making in which the clinician uses the best evidence available, in the context of consultation with the patient, to decide on which option suits the patient best. Evidence-based medicine is a conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.

Evidence-based medicine or evidence-based practice aims to apply the best available evidence gained from the scientific method to clinical decision making. It seeks to assess the strength of evidence of the risks and benefits of treatment or lack of treatment and diagnostic tests. Evidence quality can range from meta-analysis and systematic reviews of double-blind, placebo-clinical trials at the top end down to conventional wisdom at the bottom.

The systematic review of published research is the major method used for evaluating particular treatments. The Cochrane Collaborations are among the best known and respected examples of systematic reviews. Like other collections of systematic reviews, it requires authors to provide a detailed and repeatable plan of literature search and evaluation of the evidence. Once all the evidence is assessed, treatment is categorized as “likely to be beneficial,” “likely to be harmful,” or “evidence did not support either benefit or harm.”

Evidence-based medicine categorizes different types of clinical evidence and rates or grades them according to the strengths of their freedom from the various biases that have beset medical research. The strongest evidence for therapeutic intervention is provided by systematic review of randomized, triple-blind, placebo-controlled trials with allocation concealment and complete follow-up involving a homogeneous patient population and medical condition. In contrast, patient testimonials, case reports, or even expert opinion have little value as proof because of the placebo effect, the biases inherent in observation in reporting cases, and difficulty in ascertaining who is an expert.

Randomized, controlled trials are highly desirable to answer questions about technology assessment, but there are alternatives. Carefully controlled observational studies are good predictors of the outcome of randomized trials as long as enough essential information is collected to ensure that the patients are standardized. It is equally important in an observational series to understand which patients have been excluded.⁶¹

The American College of Radiology (ACR) created a Task Force on Appropriateness Criteria. This task force incorporates attributes for developing exceptional medical practice guidelines that have been used by the Agency for Health Care Research and Quality as designated by the Institute of Medicine. The ACR adopted the AQA’s⁶² definition of appropriateness: “The concept of appropriateness, as applied to health care, balances risk and benefit of treatment, test, or procedure in the context of available resources for an individual patient with specific characteristics. Appropriateness criteria provides guidance to supplement the clinician’s judgment as to whether a patient is a reasonable candidate for the given treatment, test, or procedure.”

The ACR Task Force on Appropriateness Criteria established 20 expert panels. Ten are concerned with diagnostic radiology and 10 with radiation therapy. In this latter group there are panels for bone metastases, brain metastases, breast, gynecologic, head and neck malignancies, Hodgkin disease, and lung, prostate, and rectal/anal malignancies. Each expert panel is chaired by an individual who is deemed by the College to have leadership capabilities and a national recognition of expertise in the area of focus. Volunteer physicians are involved in the criteria development process. Expert panelists go through a process of literature searches, acquisition of scientific articles, drafting of evidence tables, dissemination of materials for the Delphi process, collation of results, conference calls, and documentation processing. For multiple radiation oncology treatments and procedures, there are published ACR appropriateness guidelines that one can consult to garner an opinion about the “consensus view.”⁶²

The Delphi methodology is a technique used to arrive at appropriateness ratings. A series of surveys is conducted to elicit members of the expert panel’s interpretation of the evidence concerning the appropriateness of a therapeutic procedure for specific clinical scenario. The panelist receives a survey with an evidence table and narrative. Each panelist is asked to interpret the available evidence and rank each procedure. The survey is completed by each panelist without consulting fellow panel members. Ratings are done on a scale of 1 through 9, which is further divided into three categories, 1 to 3 is defined as usually not appropriate; 4 to 6 is defined as maybe appropriate; and 7 to 9 is defined as usually appropriate. Each panel member assigns one rating for each radiation therapy procedure or intervention per survey round. Surveys are collected and the results are tabulated, deidentified, and redistributed after each round. A maximum of three rounds is conducted. This modified Delphi technique enables each panelist to express individual interpretations of the evidence and his or her expert opinion without receiving bias from fellow panelists in a simple, standardized, and economical process. A consensus is defined as 80% agreement within a rating category.

Practicing evidence-based medicine requires recognition that, in most encounters with patients, questions arise that should be answered to provide the patient with the best available medical care. Appropriate and relevant clinical questions contain four elements: (a) a patient or problem, (b) an intervention, (c) a comparison intervention (if necessary), and (d) an outcome. Important practical steps in practicing evidence-based medicine include the following:

1. Formulation of the patient’s clinical problem

2. Search of literature for relevant and reliable data

3. Evaluation of validity and usefulness of data

4. Integration of critical evaluation of evidence with clinical judgment

5. Implementation of useful information

6. Assessment of outcome to improve medical practice

Once the right questions have been formulated, the best source for finding most types of best evidence is by searching Medline (National Library of Medicine), PubMed (National Library of Medicine), OVID, the Cochrane Database, or a similar database by computer.

The quality (strength) of evidence is based on a hierarchy, proceeding from the most reliable results of systematic reviews of well-designed prospective clinical trials to results of one or more well-designed meta-analysis studies, results of large retrospective case series, expert opinion, and personal experience.⁶³ Once the best data have been found, the evidence-based medicine approach involves critically appraising the quality of the evidence, determining its magnitude and precision, and applying it to a specific patient.^64,65

The Cochrane Collaboration was established in 1993 as an international network to help health care providers, policy makers, patients, their advocates, and their caregivers make well-informed decisions about health care by preparing, updating, and promoting the success of Cochrane reviews. More than 4,500 such reviews have been published so far online in the Cochrane Library.⁶⁶

The Cochrane Collaboration traces its origins to a 1972 publication by the British epidemiologist A. Cochrane entitled Effectiveness and Efficiency: Random Reflections on Health Services.⁶⁷ This book called attention to the collective ignorance concerning the effects of health care. In 1979 Cochrane published an essay in which he suggested that “it is surely a great criticism of our profession that we have not organized a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials.”⁶⁸ Responding to Cochrane’s charge, the Cochrane Collaboration undertook such a crucial but enormous undertaking.

There are >28,000 people working within the Cochrane Collaboration in >100 countries. Its central functions are funded by royalties of its publisher, which comes from sales from subscription to the Cochrane Library. The individual entities of the Cochrane Collaboration are funded by a large variety of governmental, institutional, and private funding sources and are bound by organization policy limiting uses of funds from corporate sponsors.

Evidence-based medicine promotes rule-based behavior on the part of physicians in an effort, among other things, to eliminate variations in medical practice. However, professionals do not follow rules per se; they intuit what is right in a situation, including, sometimes, that it is right to defer to a rule.⁶⁹

The practice of evidence-based medicine (careful clinical judgment in evaluating the “best available data”) should be differentiated from the special collection of data regarded as “suitable evidence.” The new collection of best available information has major constraints for the care of individual patients; derived almost exclusively from randomized trials and meta-analyses, the data do not include many types of treatments or patients seen in clinical practice, and the results show comparative efficacy of treatment for an “average” randomized patient, not for pertinent subgroups formed by such cogent clinical features as severity of symptoms, illness, comorbidity, and other clinical nuances. The intention-to-treat analyses in clinical trials reporting do not reflect important postrandomization events leading to altered treatment, and the results seldom provide suitable background data when therapy is given prophylactically rather than remedially. The authoritative aura given to the “collection of data” may lead to abuses that produce inappropriate guidelines or poorly supported doctrinaire dogmas for clinical practice.⁷⁰ There are a variety of mechanisms to apply evidence-based medicine to clinical practice, including formulation of treatment pathways, practice guidelines, appropriateness criteria, consensus conferences, patterns of care, and institutional peer review. A fundamental part of evidence-based medicine is the filtering of evidence through the clinical skill of the clinician. It is not easy to determine which level of evidence is required, and it is estimated that only 20% of medical procedures currently in practice can be justified by evidence-based medical standards. In addition, EBM has a dark companion: what Bentzen and Wasserman⁵⁸ called lack-of-evidence-based medicine (LEBM), a methodology used in the United Kingdom by the National Institute for Clinical Excellence to restrict access to a number of therapies on the National Health Service. Whereas evidence-based medicine is concerned with weighing level I evidence for or against a give intervention, LEBM is concerned with the systematic review of the lack-of-level I evidence on a given topic.

When does a new cancer therapy cease being experimental and become a component of the standard of care? This distinction is subject to powerful influences for for-profit pharmaceutical companies and medical equipment manufacturers wanting to increasing their sales and financial return to shareholders, advocacy groups, politicians, and a public anxiously awaiting the next “cancer breakthrough.” A dispassionate set of systems for technology assessment of cancer therapies is essential to evidence-based oncology practice.

In 1993 in the United States the American Society of Clinical Oncology (ASCO) formed a Health Services Research Committee (HSRC) to perform technology assessments and to develop guidelines for cancer treatment. The Committee is composed of clinical researchers and clinicians from academic centers and private practice and performs technology assessment after hearing from expert advisors. The committee does not develop guidelines; it assembles expert panels to develop guidelines.⁷¹

Among the most important initial tasks of the HSRC was to create an outcome working group to define the outcomes of cancer treatment that should be considered for technology assessment in forming cancer treatment guidelines. Not surprisingly, patients, physicians, researchers, payers, and policy makers all have different ideas about which outcomes of cancer treatment are important. The main purpose of ASCO’s technology assessments and guidelines is to define what constitutes the best cancer treatment, not to inform policy development and payment decisions. The working group on outcomes, therefore, considered outcomes primarily from the perspective of the clinical investigators who produced evidence for the guidelines, individual doctors and patients who have to make decisions about treatment, and health services researchers who evaluate the quality and cost of care produced by the guidelines. Because it is clear that there are health policy implications of ASCO’s technology assessments and guidelines, the working group also considered outcomes that would convince health policy makers that ASCO’s technology assessment was sound and that its guidelines represented worthwhile treatment.^25,71 For technology assessment the ASCO working group is composed of members of the Health Service Research Committee and selected members. They conduct a literature review and analyze the outcomes-based evidence. Testimony is collected from invited experts and interested parties. After evaluating the information, the working group defines specific questions and provides a report outlining conditions under which the proposed procedures are warranted to assist both physicians and patients in making informed decisions regarding the technologist or procedures.

The working group identifies specific questions to be addressed by the technology assessment, develops a strategy for completing the assessment, and reviews the available literature and evidence. The process may include face-to-face meetings of available working groups and members, circulation of draft forms of the technology assessment, and opportunities for comment. Working groups do not attempt to codify established practice. They review the available evidence and add their best clinical judgment to make final recommendations.

FIGURE 98.2. Conceptual framework of outcome research. Interaction is shown between research topics, endpoints, analytic techniques, and applications in defining outcomes research. Depicted in the upper left corner are the classic clinical trials and analytic techniques that are not outcomes research. In the upper right corner are shown the study topics, endpoints, and analytic techniques that are considered to be outcomes research. Outcomes depicted in the center box may or may not constitute outcomes research, depending on the context. For example, overall survival as measured in a phase III trial is not an outcomes study (efficacy), whereas it is if observed in a large community cohort (effectiveness). Symptoms have both efficacy and outcomes influences. Applications are indicated in italics and may emanate from either clinical trials or outcomes research. See text for further details. (From Lee SJ, Earle CC, Weeks JC. Outcomes research in oncology: history, conceptual framework, and trends in the literature. J Natl Cancer Inst 2000;92:195–204.)