On September 09, 2015, California State Superintendent of Public Instruction Tom Torlakson released California's Smarter Balanced results, which comprise the largest component of the California Assessment of Student Performance and Progress (CASPP).
As San Jose University professor Roxana Mariachi notes, a major issue with California's Smarter Balanced results involves the apparent inability of the California Department of Education (CDE) to produce evidence that the Smarter Balanced tests are reliable and valid:
At the September 2nd, 2015 State Board of Education meeting, you heard public comment from Dr. Doug McRae, a retired test and measurement expert who has for the past five years communicated directly and specifically to the Board about validity problems with the new assessments. He has submitted the following written comments for Item #1 [CAASPP Update] at the latest meeting and spoke again about the lack of evidence for validity, reliability, and fairness of the new assessments.
"My name is Doug McRae, a retired testing specialist from Monterey.
"The big question for Smarter Balanced test results is not the delay in release of the scores, or the relationships to old STAR data on the CDE website, but rather the quality of the Smarter Balanced scores now being provided to local districts and schools. These scores should be valid reliable and fair, as required by California statute as well as professional standards for large scale K-12 assessments. When I made a Public Records Request to the CDE last winter for documentation of validity reliability and fairness information for Smarter Balanced tests, either in CDE files or obtainable from the Smarter Balanced consortium, the reply letter in January said CDE had no such information in their files. I provided a copy of this interchange to the State Board at your January meeting. There has been no documentation for the validity, reliability, or fairness for Smarter Balanced tests released by Smarter Balanced, UCLA, or CDE since January, as far as I know.
"Statewide test results should not be released in the absence of documented validity reliability and fairness of scores. Individual student reports should not be shared with parents or students before the technical quality of the scores is documented." [Emphasis removed.]
The absence of evidence that Smarter Balanced tests are reliable and valid means that there is no evidence that the tests measure what they purport to measure (i.e., "college and career readiness"), nor is there any evidence that the tests measure anything consistently.
I wrote about Smarter Balanced in November 2014. At that time, the Smarter Balanced Assessment Consortium (SBAC) had decided that "college ready" meant "Common Core content ready," and it had no clue how to operationalize "career ready."
That was only two months prior to McRae's request for reliability and validity evidence, which does not seem to exist.
Still, Torlakson and the CDE are pitching Smarter Balanced results as though they are reliable and valid. From the September 09, 2015, CDE press release:
SACRAMENTO--State Superintendent of Public Instruction Tom Torlakson today unveiled the results of new online assessments administered to about 3.2 million students last spring to gauge their progress in learning new, more rigorous academic standards designed to prepare them for college and careers in the 21st century. ...
CAASPP includes a number of assessments, but the most widely given are the Smarter Balanced Summative Assessments, which evaluate student progress on the California standards in mathematics and English language arts/literacy, often referred to as the Common Core.
"The results show our starting point as a state, a window into where California students are in meeting tougher academic standards that emphasize critical thinking, problem solving, and analytical writing," Torlakson said. "California's new standards and tests are challenging for schools to teach and for students to learn, so I am encouraged that many students are at or near achievement standards. However, just as we expected, many students need to make more progress. Our job is to support students, teachers, and schools as they do."
Now, it's not that the Smarter Balanced tests are necessarily capturing nothing. It's just that neither the Smarter Balanced Assessment Consortium (SBAC) nor the CDE has bothered to establish what exactly the Smarter Balanced tests are capturing, or with what degree of consistency.
I would like to point out one comparison reported in the CDE press release, which states that 3.2 million students took what amounts to the mystery Smarter Balanced tests:
Almost 1.9 million students were classed as "economically disadvantaged" (defined by the CDE as "students eligible for the free and reduced priced meal program, FRPM, foster youth, homeless students, migrant students, and students for whom neither parent is a high school graduate").
The remaining 1.3 million were classed as "not economically disadvantaged."
The CDE divided the test results by percentage into four categories, which it termed "exceeding standards," "meeting standards," "nearly meeting standards," and "not meeting standards."
From "exceeding" to "not meeting, the economically disadvantaged percentages were 8, 23, 28, 41- which means that the economically disadvantaged tended toward lower scores.
Compare the above to the "not economically disadvantaged," whose corresponding scoring percentages were 29, 35, 21, 15- which means that students who were not economically disadvantaged tended toward higher scores.
Given the results above, the onus rests upon the CDE and SBAC to present to the public empirical evidence demonstrating that the Smarter Balanced tests do more than simply mirror the socioeconomic status of test takers.
In addition, it is the responsibility of CDE and SBAC to demonstrate that the shaky operationalizing of "college and career readiness" as referenced in the November 2014 SBAC information above has some solid empirical backing. If no such evidence is available, then it is reasonable to assume that both CDE and SBAC are knowingly perpetrating a colossal fraud with the supposed "college and career ready" SBAC assessment product.
Originally posted 09-10-15 at deutsch29.wordpress.com
Schneider is a southern Louisiana native, career teacher, trained researcher, and author of the ed reform whistle blower, A Chronicle of Echoes: Who's Who In the Implosion of American Public Education.
She also has a second book, Common Core Dilemma: Who Owns Our Schools?, published on June 12, 2015.
