Visit msnbc.com for breaking news, world news, and news about the economy
ATLANTA — Hundreds of school systems nationwide exhibit suspicious test scores that point to the possibility of cheating, according to an investigation by The Atlanta Journal-Constitution.
The analysis doesn't prove cheating. It reveals that scores in hundreds of cities followed a pattern that, in Atlanta, indicated cheating in multiple schools.
"This encourages further exploration of these unusual patterns, but it doesn't necessarily mean that there's widespread cheating," Aaron Pallas, professor of Sociology and Education at Columbia University's Teachers College, told NBC News.
The AJC reported in 2008 and 2009 about statistically improbable jumps in test scores within the 109-school Atlanta Public Schools system. Those reports led to an investigation by Georgia officials, which found that at least 180 principals, teachers and other staff took part in widespread test-tampering in the 50,000-student district.
In Sunday's editions, the AJC reports that 196 of the nation's 3,125 largest school districts had enough suspect test results that the odds of the results occurring naturally were less than one in 1,000.
For 33 districts nationwide, the odds of their test scores occurring naturally were worse than one in a million.
Standardized test scores have been at the forefront of national and local efforts to improve schools. Test performance was the centerpiece of the federal No Child Left Behind Act of 2001, which demanded higher classroom accountability. Tougher teacher evaluations that many states are rolling out place more weight than ever on the tests.
But the AJC report found that the sweeping policy shifts rely on test results that may be unreliable.
While the federal government requires states to use standardized testing, it does not require educators to screen scores for anomalies or investigate those that turn up.
"If we are going to make important decisions based on test results – and we ought to be doing that – we have to make important decisions about how we are going to ensure their trustworthiness," said Daria Hall, director of K-12 policy with the nonprofit Education Trust.
"That means districts and states taking ownership of the test security issue in a way that they haven't to date."
In nine districts –- Atlanta, Baltimore, Dallas, Detroit, East St. Louis, Ill., Gary, Ind., Houston, Los Angeles and Mobile County, Ala. –- scores careened so unpredictably that the odds of such dramatic shifts occurring without an intervention such as tampering were virtually zero, the newspaper found.
In Houston, test results for entire grades of students jumped two, three or more times the amount expected in one year, the analysis showed. When children moved to a new grade the next year, their scores plummeted – a finding that suggests the gains were not due to learning.
"These findings are concerning," U.S. Secretary of Education Arne Duncan said in a statement after being briefed on the AJC's analysis. He added that "states, districts, schools and testing companies should have sensible safeguards in place to ensure tests accurately reflect student learning."
In a statement to community members friday, Houston Independent School District schools chief Terry Grier acknowledged AJC's flagging of several schools in the district for improbable fluctuations in test scores. Grier notes that HISD had hired outside law firms to conduct investigations of possible cheating at schools that resulted in a number of educator terminations. HISD teacher are also not allowed to administer state exams to their own students, and storage rooms where testing materials are kept are monitored by security cameras.
Many school district officials contacted by the AJC disputed any conclusion that cheating was to blame for the swings.
Some school leaders attributed steep gains to exemplary teaching. But experts said instruction isn't likely to move scores to the degree seen in the AJC's analysis.
Cheating is one of only a few plausible explanations for such dramatic changes in scores for so many students within a district, said James Wollack, a University of Wisconsin-Madison expert in testing and cheating who reviewed the newspaper's analysis.
"I can say with some confidence," he said, "cheating is something you should be looking at."
A contrary example resides in Indiana. The Project School, a K-9 charter school in Bloomington, has started to teach standardized testing as literary genre, dedicating two weeks out of the year to teach students test strategy and how to take the state exam with a different approach.
"In our world, you have to be able to write to a prompt you have to be able to do what somebody else expects of you in the moment they expect it of you --- going to a job interview, going to write your college applications, your essays to get into college. So we want them to know whats expected of them and be able to perform in any setting," Baron-Caudill told StateImpact. "We analyze testing with them and learn test-taking strategies... We want them to know how to be smart about taking a test. But we're not going to spend time and energy teaching to the test."
In each state, the newspaper used statistics to identify unusual score jumps and drops on state math and reading tests by grade and school. Declines can signal cheating the previous year. The calculations also took into account other factors that can lead to big score shifts, such as small classes and dramatic changes in class size.
The newspaper also developed a statistical method to identify school systems with far more unusual tests than expected, which could signal endemic cheating similar to what occurred in Atlanta. In its approach, the score analysis used conservative measures that highlighted extremes. The methodology is more likely to overlook possible indications of cheating than to suggest problems where none exist.
The newspaper's methodology was reviewed by outside experts.
The AJC's analysis suggests that tens of thousands of children may have been harmed by inflated scores that could have kept them from getting the academic help they needed.
In 2010 alone, the grade-wide reading scores of 24,618 children nationwide – enough to populate a mid-sized school district – swung so improbably that the odds of it happening by chance were less than 1 in 10,000.
Experts said the findings warrant deeper investigation at the local level.
Statistical checks for highly improbable scores are like medical tests, said Gary Phillips, a vice president and chief scientist for the large nonprofit the American Institutes for Research, who advised the AJC on its methodology.
"This is a broad screening," he said. "If you find something, you're supposed to go to the doctor and follow up with a more detailed diagnostic process."
Districts and independent researchers have also questioned the AJC's methodology, and caution drawing premature conclusions. The analysis does not take into account erasure rates or other direct evidence of cheating, and analyzing "cohorts," or groups of students without access to individual students' test scores create room for large margins of error.
A statement issued Saturday by Nashville Metro Schools lists a number of concerns officials have with AJC's analysis. Variants like high student mobility rates, zoning changes and high proportions of English learners, among others, were not taken into consideration in analyzing the data, Nashville officials say.
Students absent from testing were assigned a "zero" in the data, rather than being excluded from the analysis, resulting in average scores that were below the minimum possible score. Flagged schools also tended to be those with large fluctuations in class size and frequent student transfers as well as those at the top and bottom of the economic scale.
To be sure, education leaders point out that the AJC data's bottom line is likely less about where the cheaters are but that educators feel a need to cheat at all -- directing the discussion to the contentious value of standardized testing. In a statement Saturday, the National Education Association responded to the AJC report by reaffirming its concern with high-stakes tests, adding that it strongly opposes cheating.
"The overuse of standardized tests for high-stakes decisions has shortchanged students, teachers and our education system in too many ways for far too long. We've lost sight of the reason tests were designed -- to help gauge students' comprehension and progress," NEA President Dennis Van Roekel said. "By continuing to focus on these tests results and placing yet more blame on teachers, we are diverting attention from what really matters -- making sure all our teachers have the resources they need to provide our students with the world class public school education they deserve."
AJC's prior reports revealed one of the largest cheating scandals in American school history, in which teachers and school administrators said they were pressured to maintain high scores under the federal No Child Left Behind law, as student performance on standardized exams is tied to school funding and teacher performance assessments. A number of school districts offered bonuses to teachers who had students with significant gains to testing scores. An investigation into the scandal found that Atlanta Public Schools officials created a culture of "fear, intimidation and retaliation."
But the overhead pressure wasn't unique to Atlanta. School districts from Pennsylvania to Texas to California saw similar problems, often identified by test erasure analyses, as investigations launched in systems across the country.
A Detroit Free Press survey last July reported that nearly 30 percent of public school educators say pressure to cheat on standardized exams is a problem at their schools, particularly at schools that don't meet federal standards, where 46 percent say cheating is an issue.
To lessen the strain of a one-size-fits all approach to student assessments, the Education Department has issued waivers to 11 states, allowing them more freedom from No Child Left Behind -- the Bush-era law that requires annual testing, results of which are tied to consequences for low-performing schools. States that seek waivers from the Obama administration are required to adhere to a measurement, curriculum and assessment plan proposed during the application process. An additional 26 states have applied for waivers.
Even so, some states are still trying to further lessen the emphasis on standardized tests. Virginia's state Senate voted to pass a bill in January that scales back statewide tests for 3rd graders -- cutting history and science from the list and only requiring English and math exams to allow teachers to focus on improving proficiency in those subjects.
The move by the Virginia Senate comes after a draft of a Republican bill would eliminate the federal requirement for statewide science testing. The draft legislation, introduced by House Republicans led by Rep. John Kline (R-Minn.), chair of the House Education Committee, marks a reversal of provisions under the current No Child Left Behind Law, which requires science testing at least three times -- once each during elementary, middle and high school.
In Texas, a Dallas elementary school that was given "exemplary" status for academic achievement was discovered to have only taught its third graders reading and math last year -- fabricating scores for every student in other subjects like social studies and science.
The Dallas Morning News reported in November that to propel the school's status, Field Elementary School Principal Roslyn Carter "directed and caused false school records to be created" so that teachers could focus on student excellence in reading and math.
Shocking cheating scandals in America: