Note: Indiana University Prof. Russell Skiba co-authored this piece, along with Kevin Welner.
Science is additive. Even the most startling new discoveries are built upon years of research findings reported by others. Sir Isaac Newton said he saw further because he stood on the shoulders of giants.
So researchers are rightly wary when one of their own claims to have overturned years of established scientific findings in a single paper. Such claims tend to crumble under scrutiny. That is what happened this past week in a paper published by Paul Morgan and his colleagues in the journal Educational Researcher.
The over-representation in special education of students of color, particularly Black students, has been studied extensively for over 40 years. This over-representation has been documented in countless research studies by individual researchers, in comprehensive collections of studies, and even by the prestigious National Academy of Sciences. So it created sharp ripples in the special education community last week, when Morgan and his colleagues reported almost completely contradictory findings, in both the journal article and in an op-ed in The New York Times, titled, "Is Special Education Racist?" The commentary claims that minority students are in fact under-represented in special education. If these findings were to hold up, it would have serious practical consequences--it would mean that almost 20 years of federal policy, based on the evidence of over-representation of students of color, was just plain wrong.
Are these researchers on to something truly new, something that would overturn some of the basic assumptions of a whole field? Or, as happens far too often, were their results in error, out of synch with the existing mass of research simply because their data are wrong, their analyses are flawed, or both?
This is a case of the latter. As researchers around the country quickly noted upon the publication of the Morgan et al article, it is grounded in a host of suspicious procedures and questionable interpretations. These errors suggest that the study on which their findings are based is deeply flawed. The trumpeted results in no way support their sweeping claims.
The data on which Morgan et al base their study come from a longitudinal sample of 20,000 students first collected when the students were in kindergarten in 1998. The key data come from surveys of special education teachers about their students' special education status--data that may or may not be an accurate representation of actual numbers. During the same time period, the federal government collected actual enrollment data for the millions of students who were being served in special education.
If fact, the authors admit in a footnote that the use of survey data could result in different estimates than established national figures. Their estimate of the incidence of Intellectual Disability, for example, is significantly lower than the actual incidence in the actual national data around that time.
This cannot be glossed over. For the same years that Morgan and his colleagues claim to find minority under-representation in special education, analyses of actual counts by race of students in special education, conducted by the National Academy of Sciences (NAS) and others, found Black students to be significantly over-represented in the categories of Intellectual Disability, Emotional Disturbance, and Learning Disability. The NAS also reported higher rates for American Indians in Learning Disabilities, but Morgan's decision to lump data for American Indians with other groups prevented a test of this within their study.
Using these apparently erroneous data, the authors go on to make a strong claim that racial disparities in special education are due to the effects of poverty (including lead poisoning--which is an odd claim, since the study doesn't measure lead exposure). Their claim that poverty explains special education disproportionality is simply unsupported by their own data. In multivariate statistics of the type used in their study, one variable is said to explain another when it is found to be statistically significant. But poverty significantly predicts disability in only one of the five disability categories Morgan and colleagues studied, and then in the opposite direction of their prediction. This is hardly a strong demonstration that special education over-representation is due to poverty.
Like most issues involving race, over-representation in special education is highly complex and nuanced. For instance, while Black students have been found to be over-represented in at least three categories, Asian American students have never been found to be over-represented, and Latino students are found sometimes to be over-represented and sometimes under-represented.
Disparities in special education also vary widely by state. While some states in the Northeast have shown little or no Black over-representation, others in the South and Midwest show Black rates of special education enrollment four or five times that of White students. These discrepancies may be even greater at the district level.
Fortunately, the complexity in this area is already reflected in federal policy. Although Morgan and colleagues use their conclusions about under-representation to challenge federal policy on disproportionality that has been in place since first enacted by Congress in 1997, that policy doesn't require states to go chasing willy-nilly after every possible example of over-representation. Rather, it requires states to identify only those districts that show high and statistically proven levels of over-representation. Federal policy implicitly recognizes the complexity and nuance of special education over-representation. Morgan and company do not.
Faced with such massive discrepancies between their own findings and the vast body of research that came before, many researchers would make cautious claims at best. The phrase often seen in scholarly publications--"further research is necessary"--is not simply to guarantee professors more work; it is a crucial and serious acknowledgement that research findings, especially novel ones, must be tested further, and hopefully replicated by others, before we recommend important changes in policy or practice.
Morgan and colleagues, however, show no such caution. They have no compunction in asserting throughout their article and op-ed that their single study has overturned 40 years of research showing over-representation. They show no hesitancy in arguing that vital federal policy based upon that research is severely misguided.
This may be their greatest error. In the face of a highly complex issue, it is wrong to make simplistic claims about federal policy involving race, based on a suspect analysis of data that appear to be inadequate to the task. Hasty and ill-considered recommendations based on sweeping generalizations and questionable evidence are likely to be at best unhelpful, at worst harmful to the field.