By Ed Yong
In 1986, the social psychologist David Sears warned his colleagues that their habit of almost exclusively studying college students was producing a strange and skewed portrait of human nature. He was neither the first to make that critique, nor the last: Decades later, other psychologists noted that social sciences tended to focus on people from WEIRD societies — that is, Western, educated, industrialized, rich and democratic. The results of such studies are often taken to represent humanity at large, even though their participants are drawn from a “particularly thin and rather unusual slice” of it.
The same concerns have been raised in virtually every area of science that involves people. Geneticists have learned more about the DNA of people in Europe and North America than those in the rest of the world, where the greatest genetic diversity exists. The so-called Human Microbiome Project was really the Urban-American Microbiome Project, given that its participants were almost entirely from St. Louis and Houston.
“Geneticists have learned more about the DNA of people in Europe and North America than those in the rest of the world, where the greatest genetic diversity exists.”
Neuroscience faces the same problems. When scientists use medical scanners to repeatedly peer at the shapes and activities of the human brain, those brains tend to belong to wealthy and well-educated people. And unless researchers take steps to correct for that bias, what we get is an understanding of the brain that’s incomplete, skewed, and, well, a little weird.
Kaja LeWinn, from the University of California, San Francisco, demonstrated this by reanalyzing data from a large study that scanned 1,162 children ages 3 to 18 to see how their brain changed as they grew up. The kids came from disproportionately wealthy and well-educated families, so LeWinn adjusted the data to see what it would look like if they had been more representative of the U.S. population. That’s called “weighting,” and it’s a common strategy that epidemiologists use to deal with skews in their samples. As an easy example, if you ended up recruiting twice as many boys as girls, you’d assign the girls twice as much “weight” as the boys.
When LeWinn weighted her data for factors such as sex, ethnicity, and wealth, the results looked very different from the original set. The brain as a whole developed faster than previously thought, and some parts matured earlier relative to others.
Natalie Brito, from New York University, says that this study “clearly shows how our interpretation of brain development changes based off who is being represented within the sample.” She adds that most neuroscientists would acknowledge or agree that representative samples are a good thing, but that there are practical reasons why such samples are hard to get. Most obviously, brain-scanning studies are very expensive, so most of them are small and rely on “samples of convenience”—that is, whoever’s easiest to recruit.
RELATED: When Freud Meets fMRI
“Neuroimaging research is also complex and difficult to conduct, and because of this, I think there is a tendency to focus on its technological aspects,” says Duke Han, from the University of Southern California. That’s symptomatic of a problem in neuroscience that I’ve written about before: an inclination to focus on the technological innovations that allow us to study the brain, and to forget about the people whose brains are being studied.
Perhaps the brain itself invites this lapse. We intuitively understand that our thoughts and behavior vary considerably from person to person. But when it comes to the lump of gray tissue behind those behaviors, it’s easy to forget that variation. “To a degree, I think there’s a sense that a brain is a brain is a brain,” says LeWinn. “That’s problematic. Everyone’s brain is shaped by their experiences, and we want to capture the diversity of people’s experiences rather than just a few kinds.”
For example, in the study she reanalyzed, around 35 percent of the children had parents with college backgrounds, and around 38 percent had parents who earned more than $100,000 a year. If the sample had been truly representative of the U.S. population, those proportions would have been 11 percent and 26 percent, respectively. And weighting the data to account for these biases produced a different picture of brain development.
RELATED: Mapping Creativity in the Brain
Brains get bigger as we get older, before shrinking again during later childhood. In the unweighted data, the brains hit their peak volume at 6 years on average, and their peak surface area at around 12 years. But in the weighted data, the brains hit those milestones 10 months and 29 months earlier, respectively. The pattern of development across the brain also changed. In the unweighted data, three of the brain’s four lobes hit their maximum area from the ages of 12 to 13, with only the parietal lobe peaking earlier at around 10 years. But the weighted data showed more of a wave of maturation, from the back of the brain to its front, and going from 9 years to 11.
“These results shine a much-needed spotlight on an issue that does not receive enough attention in neuroimaging research — the lack of diversity among study participants,” says Han. “Unless these issues are adequately addressed, it would be wise to show temperance in discussing the implications of a study.”
Jim Coan, from the University of Virginia, learned the same lesson in his own work. A decade ago, he put 16 women in a brain scanner, promised to give them an electric shock, and looked at parts of their brains that respond to threats. He found that these areas are less active if the women held the hand of a stranger, even less active if they held their spouse’s hand, and less active still if they were in an especially happy relationship. “I had to raise $30,000 to do that experiment and everyone was white, wealthy, well educated. And yet, we thought: Here’s the story,” he says. “By yourself, you’re maximally responsible for meeting the demand of the threatening situation so you have more of a threat response. If you’re with your trusted romantic partner, you’re minimally responsive because you outsource.”
Years later, he got more money to do a bigger and more representative study of racially and socioeconomically diverse people drawn from the local community. “And the findings changed,” he says. The romantic partners still reduced the threat response, but a stranger’s hand had no effect at all. Why? Perhaps it’s because, as he showed in another study, the wealth of the neighborhood you grow up in affects the way your brain weighs up rewards and threats. “This shouldn’t surprise anybody,” he says. “The context in which you develop shapes the way your brain functions and probably the way it’s structured.”
Neuroscientists are increasingly grappling with this self-evident truth. Brain-scanning studies are getting bigger, and researchers are making more of an effort to recruit samples that are at least representative of the local community—if not America as a whole.
The Adolescent-Brain Cognitive Development study — the largest study of childhood brain development in the United States — is perhaps doing the best job. It’s looking to recruit around 11,500 children, ages 9 and 10, and follow their brain development over the next 10 years. The plan is to get a truly representative sample, and to deal with any small skews with the same weighting approach that LeWinn uses. That has many advantages, says Wes Thompson, from the University of California, San Diego, who is involved in the study. You not only can see what the average American brain looks like as it grows up, but you can see how different subgroups differ from this population-wide norm, and how individuals differ from their particular subgroup.
“Since bigger studies are now coming online, this is the time to think about the sampling issue,” says LeWinn. “We’re finally doing studies that are large enough to get representative samples.”
This story originally appeared on TheAtlantic.com.
More from The Atlantic: Death at a Penn State Fraternity, Google X and the Science of Radical Creativity