CORONAVIRUS

What That Sturgis COVID-19 Paper Doesn't Show — And What It Does

Scholars are paying attention to new research on the motorcycle rally's effects on coronavirus spread, but they have lots of questions about it, too.

A new research paper on the spread of COVID-19 from a South Dakota motorcycle rally last month has generated a lot of attention ― and a lot of skepticism.

In both cases, it’s easy to see why.

The paper, by four economists from three U.S. universities, claims that the 10-day event in Sturgis led to nearly 267,000 cases around the country. That figure is dramatically higher than the number of cases public health departments have linked directly to the festival ― 124 in South Dakota, according to state officials, and 290 across the country, according to a tally from The Associated Press.

The study prompted splashy headlines and has lit up social media all week. Even Bette Middler tweeted about it. “Thanks assholes,” she wrote. 

South Dakota officials have rejected the paper’s findings, with Gov. Kristi Noem calling it “fiction.” Noem, a Republican, said the researchers “took a snapshot in time, and they did a lot of speculation, did some back-of-the-napkin math, made up some numbers and published them.”

Models are very valuable tools when we can’t have our ideal, but there are limitations. Keri Altoff, Johns Hopkins University Bloomberg School of Public Health

Noem’s attitude may not be surprising, given the relatively lax approach she’s taken to the pandemic ― eschewing mask mandates, for example, and allowing the Sturgis rally to go forward despite warnings that it would become a “superspreader” event. 

She isn’t the only one questioning the paper, however. A number of public health specialists raised concerns about its assumptions and findings. They have noted that the paper has yet to undergo peer review, which refers to the process of using outside scholars to vet a paper before it’s published in an academic journal.

At the same time, plenty of outside experts are taking the paper seriously ― noting that the authors used a widely accepted methodology and were transparent about their assumptions. That’s is a far cry from making up the numbers, as Noem suggested. 

Whatever the merits of that 266,000 figure, several outside scholars told HuffPost, the paper offers more reason to think that Sturgis significantly increased COVID-19 spread well beyond what official figures suggest.

Warnings Before The Rally, COVID Cases After It

The Sturgis rally took place over 10 days in early August and ultimately attracted an estimated 460,000 visitors from all over the country. It was one of the few mass public gatherings to proceed as planned in the last few months, as COVID-19 has forced cancellations of everything from professional sports to rock concerts. 

Many public health scholars warned beforehand that Sturgis would cause outbreaks, not just near the festival site but also in communities around the country once riders returned home.

During the event, images showed attendees crowded together, sometimes indoors, creating precisely the conditions in which the virus spreads most readily. Most people were not wearing masks.

Concertgoers attend a performance by Saul at the Iron Horse Saloon on Aug. 14,  2020, during the 80th annual Sturgis Mot
Concertgoers attend a performance by Saul at the Iron Horse Saloon on Aug. 14,  2020, during the 80th annual Sturgis Motorcycle Rally in South Dakota.

Since the festival, COVID-19 cases have spiked in South Dakota, North Dakota and neighboring Iowa, and public health workers in those states and others have identified cases originating from people who likely got sick while attending the event. That includes one Minnesota man who died last week.

Still, the official tally is not much more than 100, and it is not clear how much those big spikes in the Dakotas and adjacent states are byproducts of Sturgis. Other factors, starting with the return of college students to campus, could be playing a big role.

To address these questions, four economists affiliated with the Center for Health Economics and Policy Studies at the San Diego State University ― Dhaval Dave, Andrew Friedson, Drew McNichols and Joseph Sabia ― used mathematical modeling, based in part on anonymized cell phone data, to compare outbreaks in communities with Sturgis attendees to those without. 

That is how the scholars got their 267,000 caseload figure ― which, they said, would account for nearly 1 in 5 COVID-19 cases at the time. 

The findings, in a working paper published by the Iza Institute of Labor Economics, provide “strong evidence that the Sturgis Rally appears to have been a superspreader event for COVID-19,” the authors wrote. “We find significant case increases within the state of South Dakota as well as increases extending to counties from which relatively more residents attended the event.”

The authors also said evidence suggested lower spread in states that were taking active mitigation steps, like requiring masks and restricting business activity.

The Difficulties Of Pinpointing A Number

One reason the number of coronavirus cases the working paper links to Sturgis is so much higher than the official numbers is that public health departments rely on contact tracing ― that is, interviewing everybody who tests positive for the virus to get their known contacts, then getting in touch with those people, and so on. It’s labor-intensive work that can strain even well-funded public health departments. 

“At the scale we’re talking about here, contact tracing is just not going to be effective, at least as it’s currently implemented,” Elizabeth Stuart, a biostatistician at the Johns Hopkins Bloomberg School of Public Health, told HuffPost on Wednesday.

Keri Althoff, a professor of epidemiology also at the Hopkins Bloomberg School, said she, too, found the paper’s general finding “plausible.” She also said that’s not the same thing as saying the finding is right. 

“Models are very valuable tools when we can’t have our ideal, but there are limitations,” Althoff said.

Among the key issues, she and other scholars said, were the assumptions the authors plugged into their models.

“What the paper doesn’t tell us is what would have happened in these communities if the rally had been cancelled,” Lauren Ancel Meyers, director of the University of Texas COVID-19 Modeling Consortium, said in an email. “Would the folks who attended the rally still have gathered in large groups or engaged in other activities that exacerbated COVID-19 spread?”

Stuart added, “I would have liked to have seen more acknowledgment of the uncertainty, the noise in there.” 

What the paper doesn’t tell us is what would have happened in these communities if the rally had been cancelled. Lauren Ancel Meyers, University of Texas

Other scholars weighed in on Twitter, with Kevin Griffith of Vanderbilt urging “caution” before taking results at face value and Jimi Adams from the University of Colorado Denver saying, “Author’s summary of the assumptions left me less convinced of their methods, rather than moreso.” The paper, he said, “likely vastly overstates the impact.”

But Stuart said the general approach of the paper was “reasonable” and “widely accepted.” Although she might not vouch for the 266,000 figure, she said, all current estimates of COVID-19 spread are going to be highly imprecise. 

Meyers also saw the paper as a contribution: “Even though the analysis makes key assumptions that need further validation,” she said, “it provides initial evidence that the pandemic spread faster in communities that hosted or had large numbers of residents that attended the rally than communities that largely stayed away.”

It’s entirely possible that, as the paper continues to get attention and scrutiny, a consensus among scholars will form ― that it’s mostly right, that it’s mostly wrong, or somewhere in between. That’s the nature of academic research, especially for hastily written working papers.

“I think it’s really important to do these sorts of studies, but it’s also incredibly hard to do,” Stuart said. “I’m a big fan of building a body of evidence, knowing that any one study is not going to be definitive.”

 

CORRECTION: A previous version of this article misidentified the institution associated with the Center for Health Economics and Policy Studies as the University of California; rather, it is San Diego State.