On August 26, 2016, Reuters reported that the FBI raided the home of former College Board exec Manuel Alfaro, in connection with a leak of over 400 SAT test items to Reuters earlier the same month.
According to Reuters, the FBI “seized computers and other material” from Alfaro’s residence.
Alfaro has been publicly posting his concerns about the redesigned SAT in detail for months on LinkedIn, and he continues to post on LinkedIn the day following the raid, August 27, 2016.
Below is an excerpt from Alfaro’s August 27th post, which he begins with a bold declaration:
Shining a Spotlight on the Dark Corners of the College Board: SAT has a Critical Technical Flaw
Manuel Alfaro
August 27, 2016
The FBI raided my home yesterday at the request of the College Board. It appears that Coleman wants to raise the stakes. Challenge accepted!
Note: Whenever “SAT” is used below, it refers to the “redesigned SAT.”
The SAT test specifications require that, for items in context, 10% are Heavy (more than 60 words), 50% are Medium (40-60 words), and 40% are Light (less than 40 words). On its website, the College Board states that the SAT Practice forms were constructed using the same specifications as the SAT Operational forms. The College Board, however, did not use these specifications to build the SAT forms. And we found out about this after the SAT forms were built.
On January 2015, I received a high priority email from Sherri Miller requesting a report of the item distribution across the three categories (Heavy, Medium, Light)—she needed to provide confirmation, that night if possible, to Cyndie Schmeiser that the SAT forms were built to specifications. After analyzing the form metadata, I let her know that all the forms had about 45% Heavy items. “Wow,” she replied, “We had changed that to 10% heavy in the specs given the timing studies. How did we get to 45% of them being heavy?”
Months earlier, at a meeting with the SAT’s Technical Advisory Committee (TAC), the committee recommended that the SAT specifications be changed to address the timing problem—the formal timing studies conducted by the College Board had shown that the SAT test was speeded (students did not have enough time to complete the test). The other option was to increase testing time to ensure that students had enough time to complete the test. After the meeting with the TAC, Sherri Miller changed the specifications as follows:
Old SAT Specifications (Number of Items)
Heavy 13; Medium/Light 13; Total 26
New SAT Specifications (Number of Items)
Heavy 3; Medium 13; Light 10; Total 26
The new specifications, however, were not used to build the forms. I don’t know where the breakdown in communication occurred, as I was not involved in communications with the team that built the algorithm to select the forms. The only thing I know for sure is that the algorithm used the old SAT specifications to build the forms. Additionally, even if they wanted to use the new specifications, the operational item pool would not have been able to accommodate the new requirements: the item pool was built to support the old specifications, so there were not enough Medium and Light items in the pool to build the forms.
There is more to the above post. Alfaro clearly wants to expose the College Board.
I will continue following Alfaro’s story as it unfolds at deutsch29.wordpress.com.
(Note: The above post was originally posted on August 27, 2016, and can be accessed here. Too, additional information from Alfaro is already available and can be read here.)
***
