As another major scientific journal criticizes the methodology behind a study in the New England Journal of Medicine (NEJM) that prompted a nationwide panic over a popular diabetes drug, it turns out that the doctor who authored the study, Steven Nissen MD, delivered a blistering attack on those very methods seven years ago, when they were used by another researcher to argue that a different drug posed an equally significant health risk.
These developments can only spell embarrassment for Congressional Democrats, who, with Rep Henry Waxman leading the charge, summoned the Food and Drug Administration to oversight hearings on June 6 on the basis of Nissen's supposed evidence of a dramatic 43 percent increase in the risk of hearth attack among diabetics taking Avandia. But they are also an indictment of the media, which promoted Nissen's warning without noting its obvious statistical limitations. (See here and here for earlier analyses on the Huffington Post of the methodological problems.)
In the July issue of Nature Clinical Practice, Valentin Fuster, Editor-in-Chief of Nature Clinical Practice Cardiovascular Medicine and Michael E Farkouh, Associate Professor of Medicine at Mount Sinai in New York, take NEJM to task in an editorial for rushing into print incomplete data derived from a flawed meta-analysis of studies that were not designed to be pooled in such a way.
The NEJM paper suffers from several serious limitations, three of which deserve specific attention. First, the 42 trials included did not have the same protocol. Patients excluded in one trial could have been included in the next. Some trials compared Avandia with placebo; others tested it against an active comparator. Furthermore, the trials were of different durations and tested different dosing regimens. Second, this analysis used published data only. This lack of first-hand source data means outcomes could not be verified, double-checked or examined closely. Third, as admitted in the meta-analysis, the conclusions are made on the basis of a small number of events 'that could be affected by small changes in the classification of events.Many studies included were designed to assess end points other than cardiovascular disease.
Given that the claim that Avandia increased the risk of heart attack was not supported by studies conducted under more rigorous methodologies, the authors conclude:
Neither publication deadlines or congressional schedules should serve as the catalyst for releasing incomplete data; the public will not understand the difference between a limited retrospective meta-analysis and a well-designed prospective evaluation. The treatment of type-2 diabetes is already a daunting task. Unfortunately when patients, many of whom already do not comply with medication regimens, hear that a prescribed drug has harmful effects, it's hard for doctors to regain their trust. Faster and incomplete publication isn't always better, particularly when the public and political implications may be huge, as demonstrated after publication of the NEJM report.
To add to the growing chorus of criticism, doctors Peter McCullough and Norman Lepor, in the spring issue of Reviews in Cardiovascular Medicine, call for "higher quality methods" in conducting meta-analysis on the basis of the flaws in the NEJM study - flaws that "worked to find statistical significance when in fact it was not there."
But the most telling criticism of all comes from statements made by the NEJM study's lead author seven years ago, when responding to the alarmist claims made in a similar meta-analysis presented at a cardiology conference.
"The meta-analysis was inappropriate,"Steven Nissen, MD, told Cardiology Today. "It mixed and matched studies that cannot be combined in a rational way. It tried to paint an entire class of drugs with a broad label that can't possibly make any sense."
Nissen, vice-chairman of the department of cardiology at The Cleveland Clinic Foundation, said news of [Curt] Furberg's research 'has been all over the planet and is going to hurt patients. Patients are going to stop taking their medicines which are working for them, so it creates an enormous problem for those of us who want to do the right thing. I'm upset because this does direct harm to the practice of medicine.'
Nissen, a Cardiology Today editorial board member on the Noninvasive Imaging section, said he was alarmed by Furberg's 'inflammatory' statement to the mainstream press prior to the European meeting that calcium antagonists cause 85,000 needless deaths annually. 'To paint an entire class of drugs with this kind of killer label is scientifically inappropriate and creates a lot of problems for practitioners and for the public who can't fully interpret the meaning of this study,' he said."
Nissen, it should be noted, described the risks of Avandia as possibly dwarfing that of 9/11 on ABC News, despite taking a methodological approach similar to Furburg - and, QED, adopting a similar "inflammatory" style.
The disconnect between Nissen's criticism and then later employment of meta-analysis, his views on what was appropriate to say to the media and what he ended up saying to the media, cannot but raise questions about his motives and credibility - questions that should, in light of the mounting criticism of the NEJM study, give reporters and editorial page writers pause when dealing with his future testimony (The New York Times and USA Today both wrote damning editorials on the FDA over Avandia, drawing from Nissen's data and the perfervid editorials NEJM commissioned to accompany it.)
Of course, if those very reporters and editors had paused over the statistics in the NEJM study, this debacle may not have come to pass. Seven million people take Avandia: it is the height of irresponsibility to pronounce it deadly one week, watch prescriptions collapse, and then, on sober reflection, to say "well, maybe not," the next.