Health Care’s Trick Coin
By BEN GOLDACRE
Published: February 1, 2013
For Op-Ed, follow @nytopinion and to hear from the editorial page
editor, Andrew Rosenthal, follow @andyrNYT.
THIS month, Johnson & Johnson is facing more than 10,000 lawsuits over an artificial hip that has been recalled because of a 40 percent failure rate
within five years. Mistakes happen in medicine, but internal documents
showed that executives had known of flaws with the device for some time,
but had failed to make them public.
It would be nice to imagine that this kind of behavior is exceptional,
but in reality, the entire evidence base for medicine has been
undermined by a casual lack of transparency. Sometimes this is through a
failure to report concerns raised by doctors and internal analyses, as
was the case with Johnson & Johnson. More commonly, it involves the
suppression of clinical trial results, especially when they show a drug
is no good. These problems would be bad enough on their own, but they
are compounded by a generation of “fake fixes” that have delivered false
reassurance, and so prevent realistic public discussion.
The best evidence shows that half of all the clinical trials ever
conducted and completed on the treatments in use today have never been
published in academic journals. Trials with positive or flattering
results, unsurprisingly, are about twice as likely to be published — and
this is true for both academic research and industry studies.
If I toss a coin, but hide the result every time it comes up tails, it
looks as if I always throw heads. You wouldn’t tolerate that if we were
choosing who should go first in a game of pocket billiards, but in
medicine, it’s accepted as the norm. In the worst case, we can be misled
into believing that ineffective treatments are worth using; more
commonly we are misled about the relative merits of competing
treatments, exposing patients to inferior ones.
This problem has been documented for three decades, and many in the
industry now claim it has been fixed. But every intervention has been
full of loopholes, none has been competently implemented and, lastly,
with no routine public audit, flaws have taken years to emerge.
The Food and Drug Administration Amendments Act of 2007
is the most widely cited fix. It required that new clinical trials
conducted in the United States post summaries of their results at clinicaltrials.gov within a year of completion, or face a fine of $10,000 a day. But in 2012, the British Medical Journal published the first open audit of the process,
which found that four out of five trials covered by the legislation had
ignored the reporting requirements. Amazingly, no fine has yet been
levied.
An earlier fake fix dates from 2005, when the International Committee of Medical Journal Editors made an announcement:
their members would never again publish any clinical trial unless its
existence had been declared on a publicly accessible registry before the
trial began. The reasoning was simple: if everyone registered their
trials at the beginning, we could easily spot which results were
withheld; and since everyone wants to publish in prominent academic
journals, these editors had the perfect carrot. Once again, everyone
assumed the problem had been fixed.
But four years later we discovered, in a paper from The Journal of the American Medical Association,
that the editors had broken their promise: more than half of all trials
published in leading journals still weren’t properly registered, and a
quarter weren’t registered at all.
Even if these fixes had been successful, we would still be decades away
from knowing the full truth about our medical treatments, because
today’s decisions are informed by the trials of the past, on drugs that
were first researched and approved in 2007, 2002, 1998 and earlier. None
of the reforms has even tried to ensure public access for these
results, and so they remain buried in dry storage archives, deep
underground.
All of these problems are perhaps best illustrated by the case of
Tamiflu, which governments have spent billions of dollars stockpiling,
in the belief that the drug will reduce the rate of complications from
influenza. But roughly half the trial results have never been published,
and researchers trying to obtain the full Clinical Study Reports have
been stonewalled by the manufacturer, Roche.
This cannot be acceptable. Withholding data not only misleads doctors
and patients; it’s an insult to the patients who have participated in
clinical trials, believing that they were helping to improve medical
knowledge.
Medicine routinely overcomes enormous technical challenges, and there is
nothing complicated about the changes needed to prevent Johnson &
Johnson, or Roche — or anybody — from withholding information. The
F.D.A. has proposed reforms to its systems for assessing artificial hips. And a campaign, supported by the National Physicians Alliance, has begun at alltrials.net
demanding that results be publicly reported for all trials, dating back
to at least the 1990s, on all treatments currently in use. We need
competent legislation, enforcement and leadership from medical academic
bodies, all clearly stating that nonpublication of trial results is
nothing less than research misconduct.
This will take place against a clamor from industry stakeholders. They
have worked hard to silence discussion on these problems, by pretending
that the flaws have already been fixed. Why? Because this strategy is
their only hope. There is no defense for withholding information on
treatments used by patients around the world.
