Medical Journal Malpractice?

By rushing to publish a statistically compromised study, NEJM has put at risk serious, rigorous clinical research at establishing the efficacy and safety of diabetes treatments.
This post was published on the now-closed HuffPost Contributor platform. Contributors control their own work and posted freely to our site. If you need to flag this entry as abusive, send us an email.

Among the many febrile statements that followed the publication of a study in The New England Journal of Medicine (NEJM) contending that a diabetes pill carried an unacceptable risk of heart attack, one comment stood out, not so much for the scale of prognostication (deaths from Avandia may "dwarf 9/11," said study author Dr. Steven Nissen on ABC), or the degree of vitriol over the way drugs are regulated in the U.S., but because it managed the remarkable feat of sounding both stupifyingly naive *and* disingenuous at the same time. "We are a scholarly journal, not a news outlet," NEJM executive editor Gregory Curfman told BioCentury, a publication covering the drugs industry. 'What happens in the media is beyond our control."

The problem with that statement is that it looks as if NEJM did everything it possibly could to ensure that the study by Nissen and Kathy Wolski made headlines: It was given expedited peer-review; it was accompanied by an editorial commissioned from two longstanding critics of drug regulation which questioned whether Avandia (rosiglitazone) provided any actual health benefits to patients; it was posted online on the morning of May 21, hours ahead of its scheduled press embargo of 5pm; and -- in a miraculous feat of congressional speed -- it managed to elicit a flurry of press releases from Congress by noon of the same day, which included scheduled oversight hearings on the drug on June 6. (Sen. Max Baucus, for instance, released a statement calling the Food and Drug Administration's monitoring of Avandia "appalling and unacceptable.")

What happened in the media that evening and in the following day's coverage was utterly predictable: the combination of an embargo (which meant that there was hardly any time to discuss Nissen's paper with potential critics), an extremely critical editorial (which posited that Avandia was of no real medicinal benefit anyway), and a concerted mass of congressional fury (which suggested a major failure in drug oversight that could have resulted in thousands of deaths), produced the perfect scare. Doctors were deluged with calls from anxious patients; within days, sales of Avandia collapsed.

This would have all been heroic if Nissen and NEJM hadn't published a study that was as delicately suggestive as it was brutally flawed. As I originally noted on the Huffington Post, the statistical problems involved in pooling data from multiple studies (a technique known as meta-analysis) require extreme caution when it comes to interpreting the conclusions.

As it turned out, many of the examples of myocardial infarction driving the statistical increase in risk came from one study where the diabetic participants already had congestive heart disease. Worse, Nissen excluded studies where there were no events, something which, reportedly, caused a great deal of concern at a recent meeting of the Endocrine Society, as one participant managed, in a quick search of PubMed, to come up with numerous zero-outcome studies that ought to have been factored into Nissen's analysis.

If you want to look at the safety of a drug, studies that don't find adverse outcomes are as statistically important as studies that found adverse outcomes. Nissen also wasn't able to control for the amount of time patients were exposed to Avandia relative to when they had a heart attack, which further undermines the certainly of his result.

As Brian Strom, chair of the Department of Biostatistics and Epidemiology and director of the Center for Clinical Epidemiology and Biostatistics at the University of Pennsylvania told BioCentury, Nissen and Wolski "have raised a question, they have not provided an answer. You should not draw any conclusion based on this analysis. I would not recommend taking patients off of Avandia because of this."

(I also asked experts with statistical grounding from other medical fields to look at Nissen's study to make sure there wasn't a cardiology vs endocrinology turf war going on, and they, too, were shocked by the fragility of the data and the methods.)

The other disturbing charge that NEJM propelled into the media spotlight was the idea, articulated in the accompanying editorial, that controlling for blood sugar was a troubling measure for treating diabetes and approving diabetes drugs. This is something of a controversial viewpoint given that there appears to be a consensus in endocrinology that "glycemic control" is, in fact, critically important for treating microvascular problems such as blindness, kidney damage and amputation. It may not be the only endpoint or the cardinal endpoint, but it can't be dismissed as a fatal limitation in the approval of Avandia.

As one endocrinologist put it, the problem is how to balance two different kinds of risk in controlling type 2 diabetes. What if we save 20 out of 100 people from going blind, but increase the risk of heart attack for four out of 100? Is this acceptable? No-one really has a good answer.

Fortunately, there was little immediate risk of either for patients who suddenly stopped taking Avandia, as the level of the drug didn't subside immediately. Unfortunately, as the head of one endocrinology department at a leading university medical school put it to me, an alarming number of patients are dropping out of ongoing clinical trials involving rosiglitazone and other thiazolidinedione drugs thanks to the media and congressional circus over Avandia. This has been a huge disservice to patients in clinical care and research projects. And it's also a distraction from the deeper issue with type 2 diabetes, which is that we are medicalizing a condition largely driven by lifestyle.

NEJM cannot wash its hands of its role in this by invoking scholarship and blaming the media. By rushing to publish a statistically compromised study and frame it as being of critical importance to public health, NEJM has put at risk serious, rigorous clinical research at establishing the efficacy and safety of diabetes treatments.

Indeed, an "unplanned" interim analysis of the major randomized study on the cardiovascular impact of rosiglitazone (RECORD), was published (ironically in NEJM) because "concern raised by the meta-analysis of Nissen and Wolski could well compromise the study's integrity through an increase in the dropout rate and potential biases in reporting events." (The results of RECORD are, so far, "inconclusive.")

It is not quite enough to say, "oh, how ironic if NEJM have actually undermined the one study currently in progress that could actually answer the question raised by Nissen!" One can observe that in the vulgar world of daily journalism, reporters and editors endure much greater scrutiny and suffer much greater consequences for so much less.