When is it time to abandon older, established treatment protocols in favor of newer, evidence-based standards?

That question was given greater resonance with the publication of a new study in the International Journal of Radiation Oncology, Biology and Physics, the journal of the American Society for Therapeutic Radiology and Oncology (ASTRO).

Quality of evidence has a strong effect on shaping clinical practice and needs to be continually assessed, according to the study, led by Seymour H. Levitt, MD, DSc, a professor in the Department of Therapeutic Radiology at the University of Minnesota, Minneapolis. Evidence from older trials that were not guided by well-developed guidelines needs to be reviewed, especially when those results are continually updated and used to generate evidence on which to base current clinical practice.

"Physicians increasingly are becoming reliant on the outcome of randomized clinical trials when making treatment decisions," says Levitt. "They are less dependent on anecdotal evidence. This movement toward evidence-based medicine means that physicians now routinely engage in a clinical learning strategy that involves finding, appraising, and using current research findings as the basis for clinical decisions."

Levitt and his research team examined how the role of radiotherapy in treating breast cancer has changed over the years as the quality of evidence evolved from anecdotes based on expert opinion to randomized clinical trials and meta-analyses.  After searching the medical literature for key randomized trials dating back to the 1950s that have influenced the use of postmastectomy radiation, Levitt and his team assessed how clinical practice changed on the basis of trial outcomes. They found that many of the dozens of these studies they looked at had significant design flaws, and many do not meet today's evidence-based standards. As a result, it is important for patients and physicians to continually assess the quality of evidence from these and other trials.

"Keeping abreast of the latest research findings and learning new treatment approaches take time and effort," says Levitt. "In addition, evidence-based medicine must be tempered by clinical judgement. It is important for doctors to use both their individual clinical expertise and the best available clinically relevant research when making decisions for individual patients."

The research team's criteria for assessing the trials were: Consolidation of the Standards of Reporting Trials (CONSORT) guidelines, which are a checklist of items for reporting a randomized trial; and Quality of Reporting of Meta-Analyses (QUORUM) guidelines, a checklist of standards of reporting meta-analysis.

"The study was prompted by my concern about the quality of evidence reported and used to determine treatment," says Levitt. "Postoperative irradiation trials have been performed for over 50 years, and the use of the older trials is unjustified."

Looking at early evidence (pre-1950s), Levitt notes that the prevailing view of the natural history of breast cancer was that it began locally and spread in a sequential fashion through the lymph system. Control of the disease was thought to be necessary, and radiotherapy improved local disease control. The first randomized clinical trials and meta-analyses (1950s-1970s) cast the first doubts about the efficacy of radiotherapy. Those trials resulted in a changed view of the disease from localized in some patients to systemic in all patients. Under that new perception, Levitt points out, local disease control through radiotherapy was not deemed important to survival.

Levitt then looked at meta-analyses conducted in the 1980s that are considered quality evidence on the lack of value of radiation therapy in breast cancer treatment, since they include most of the randomized controlled trials that compare mastectomy with or without irradiation. Updates of these meta-analyses, which included the Early Breast Cancer Trialists' Collaborative Group (EBCTCG) analysis of nearly 20,000 women, have shown that the initial conclusions that radiotherapy does not improve survival may be questionable based on cause-specific analyses. In addition, several individual randomized clinical trials from the 1980s had results showing a significant positive advantage for adjuvant radiation in high-risk premenopausal women treated by mastectomy and chemotherapy.

Also, a 2000 meta-analysis that examined 6,000 pre- and post-menopausal women with node-positive breast cancer confirmed the benefit of adjuvant radiotherapy in high-risk women treated by surgery and chemotherapy.

CRITERIA NOT MET

Levitt says that, based on the current and more-developed criteria for good evidence, many of the older trials from the 1950s to 1980s would not meet the level of quality deemed "good evidence." For example, one stipulation of the QUORUM guidelines is that all trials included in a meta-analysis must be published. But in the EBCTCG meta-analysis, several trials have never been published or are difficult to find. Many of these trials showed the greatest negative effect, in this case cancer deaths, on the patients treated with radiotherapy. Due to this finding, access to the appropriate information about these trials is critical for an adequate analysis of the quality of the studies. But the information cannot be accessed.

"I think any trial that is more than 10 years old should be looked at critically, especially in those instances where the disease affects many patients, and in which the methods of treatment are changing radically," says Levitt. Or, as he writes in the final sentence of the study: "Although many consider meta-analyses the gold standard' for clinical medicine, they need to be carefully evaluated to be sure we are not dealing with fool's gold.'"

Ben Van Houten is associate editor of Decisions in Imaging Economics.