Contrary to a commonly held belief, medical biometry does not only pertain to the evaluation of data, but above all, has to do with the methodological aspects of research. These include the planning of trials, their interpretation, and methodological criticism of the study results. In recent years it has increasingly been concerned with meta-analyses of trial outcomes, a necessary component in evidence-based medicine. These tasks have more to do with theoretical reasoning about problems related to medicine than merely with mathematical or statistical procedures.

In this contribution emphasis is placed on questions concerning the methodology of therapeutic research in oncology. A specific focus is given to those aspects that play a role in the evaluation of efficacy of conventional as well as complementary therapies.

The general epistemological principles that govern medical research also apply to the field of oncology. However, clinical trials in oncology take place in a very special research environment which in turn affects the details of their planning and execution.

•   Cancer is a serious, progressive disease that leads to death if left untreated.

•   Cancer is a common disease.

•   In many instances, the available therapies are of limited efficacy.

•   Most of the conventional therapies have many side effects.

•   Most of the established therapies are expensive.

•   Disease prognosis depends on many factors, especially the initial therapy (operation) and the quality of diagnosis.

The circumstances outlined above have a multitude of consequences that characterize the current situation of therapeutic research in oncology:

•   In general, one-sided testing for treatment effects is inadequate. One-sided testing in the usual superiority studies comparing a new therapy to a standard regimen has consequences: If the therapy to be tested appears to be inferior, one is no longer allowed to clarify and publish whether this inferiority is treatment-related or merely a random effect. One-sided testing is generally only acceptable if inferiority of the tested therapy is biologically impossible.

In the context of what will be discussed below, and specifically the last two points just mentioned, the following should always be kept in mind: No matter how plausible the active principle of a cancer therapy may be, how strong the immediately observed antitumor effects are, and how harmless the side effects may appear, one should never exclude a priori the possibility that the therapy may actually shorten the patient’s life.

If one takes a closer look at the difficulties we have just described, it becomes evident why well-designed trials of complementary cancer treatments are infrequent. Often, proponents of complementary therapies are concerned about the “experimental situation” created by clinical trials and, in particular, by randomized allocation of patients to the treatment groups. Sometimes, they may be reluctant to do the study in the first place, fearing that a (well-founded) null-result might harm their interests. Sometimes it is the fear of a well-founded negative result. Another point to consider is that research in complementary oncology mostly does not take part within traditional university research centers. Therefore, research culture may be less developed in comparison to conventional medicine, in which high-quality publications play a decisive role in career advancement. Finally, it should also be taken into consideration that certain nonmedical procedures are not under any regulatory pressures.

On the other hand, there are multiple reasons for the dearth of well-designed studies that proponents of complementary procedures cannot be held accountable for. According to the experience of the author, it may be impossible to implement a study, even with the best intentions of all participants involved. Here, four major reasons will be given: For a more detailed discussion see reference 2:

•   Research environment and infrastructure. Research outside of university settings usually does not have access to established collaborations with methodological centers. Also few, if any, students are available to keep the trial going by working with little or no pay. Many institutions applying complentary medicine are companies that have to count the work and time dedicated by members of the staff to a clinical trial as expenses. Additionally, the available equipment may be insufficient to meet the standards required by a protocol of a multicenter trial.

•   Access to research funding. Sponsors of respective therapies are often small companies that are unable to come up with the necessary funding for carefully conducted studies. Access to public grant funding is difficult. Regulators of studies often harbor negative feelings toward these procedures, or they have doubts (sometimes well founded) that the trial will be conducted successfully at the applicant’s institution.

•   Patient numbers. Well-designed studies are large, in particular when it comes to complementary therapies where expected treatment effects may be small. Patient numbers required for such trials can greatly surpass the capacities of the mostly humble and small institutions of complementary oncology.

•   Fragmentation of therapeutic approaches. The therapeutic approaches in complementary oncology are extremely heterogeneous. Since doctors usually swear by their treatment, even if it is only marginally different from another approach, it will be difficult to unite a group of investigators from different institutions who are prepared to apply identical therapies. This makes multicenter trials difficult. However, heterogeneity of treatment is a problem for monocenter trials, as well, namely for the relevance of the study results: for if a treatment evaluated at one institution is only one of many variants used in other places, then a result will say little about these variants. From the point of view of an external sponsor, the study may not seem to be of much value.

A study is nothing else but structured observation and experience. Clinical studies aim at obtaining data on efficacy and tolerability of therapies under transparent, predetermined, circumstances. We will limit our focus to the aspect of efficacy.

The goal of therapies in oncology is ultimately the prolongation of survival time and the improvement in quality of life of patients. These are the most important primary endpoints in therapeutic studies, gathered through the use of adequate tools. Apart from that, there are other outcome measures that are used in therapeutic studies in oncology, of which the following are the most common:

•   tumor response rate and duration of tumor response

•   immune response (rate)

•   disease-free interval/survival

•   progression-free interval/survival

•   local recurrence-free interval/survival.

The two categories “disease-free interval” and “disease-free survival” differ in one point: the latter accounts not only for tumor recurrences, but also for deaths, both of which are counted as failures. The same is true for the last two outcome measures listed above.

Which endpoint is most relevant in a therapeutic trial depends on the type of cancer, the stage, and the type of therapy and questions posed by the study. It should always be realized, however, that increase in quality of life and survival time are ultimately of predominant importance to the patient, and all other outcome measures are secondary efficacy parameters.

Some of the methodological problems of the efficacy criteria will now be discussed.

While survival time does not present any specific problems, the outcome measurement quality of life does prove problematic, starting with its definition. The difficulties of defining quality of life are similar to those encountered when defining “intelligence”: most people have only intuitive and nebulous ideas as to its meaning, and are unable to give a precise definition. As is the case with “intelligence,” one is forced to content oneself with operational definitions, in which quality of life is defined by the tools and instruments used to measure it.

The deaths of patients can bias the true picture of the chronological development of quality of life and it can lead to biased comparisons of the groups of a clinical trial. Usually, cancer patients who die mostly had very low quality of life ratings prior to their death. Their death (and removal from their treatment group) will, therefore, lead to an immediate improvement of the average score of their group, which is surely an unwanted consequence that leads to inaccurate portrayal of the results of a trial.

Tumor response will be discussed as an example of a secondary efficacy parameter (surrogate variable). It serves as the main measure of outcome in phase II trials, most of which consist of only one treatment group, that try to evaluate the antitumor activity of a therapy. When discussing the benefits of cancer therapy this particular outcome measure is not very useful; it represents a soft variable. Observed response rates depend on the type and quality of diagnosis, and show considerable oberserver variability. They are also not safe from attempts at embellishment. In comparative studies, the assessment of tumor response should therefore be blinded, i.e., the judgment should be made without knowing the patient’s treatment assignment. This is especially important in those cases where response rate forms the basis for deciding whether the therapy should be used as a routine treatment.