The New SAT: What Are We Really Testing? (Not What You Think)

In the spring of 2016, the SAT will change. Short version: Some math sections won't allow calculators. An optional essay will require analyzing a provided source document. Esoteric vocabulary will be replaced with words actually used in college courses. More free prep resources for everyone!

The New York Times is currently in a publishing frenzy, explaining the changes, exploring the history of the exam, and -- in one editorial -- suggesting that the test be dropped entirely.

I'm supportive of or agnostic about most of the changes to the SAT, and fully in favor of the free prep for all.

And now, full disclosure: I aced the SAT as a teenager (without paid test prep), and that got my ass out of the evangelical backwater I grew up in. That was pretty life-altering. I'm actually grateful to the SAT, a feeling shared by very few other students who have ever wielded a #2 pencil. And for more than a decade since, I've taken and retaken the exam, contributed to SAT manuals, and coached thousands of teenagers. I've worked for the big names in test prep, on my own, for a Korean-American study academy in Queens, and for nonprofits working with underserved populations. I had a 1600, and then a 2400. I enjoy beating tests, but I also think the United States would be better off with a completely different set of college admissions requirements that don't unduly privilege personalities like mine. (After all, no one is more annoying than adults who brag about their SAT scores.)

So, biased though I may be, I do have a certain insider's view about what's being tested on the SAT.

Let's first dispense with the notion that the exam has much to do with memorization (antonyms were dropped in 1994, folks!). Vocabulary-based questions account for only about one-ninth of the current test, and all of the vocab is tested in context, so you need reading comprehension skills beyond rote memorization to answer the harder questions. There are a handful of math formulas and geometry rules to memorize, but these are well covered in school, and there's usually way more going on in SAT math problems than simply applying a formula.

This, in fact, is the main complaint I've heard from thousands of students over ten-plus years: The questions are too tricky! They are wordy and the students get confused about what is being asked. It takes too long to figure out what is required for each question. Oh, and the wrong answers are really intuitively attractive. I sometimes bond with my students by saying, "Yes, this question was written by assholes."

So, in a way, a lot of SAT questions are about not being tricked by assholes.

This is a REALLY IMPORTANT SKILL IN LIFE. Surely more important than finding the slope of a line perpendicular to another line.

I have to say, I use the skill of "not being tricked by assholes" probably every day of my professional life. Never once has someone managed to slip something into a contract without my noticing. Or, just you try to substitute a mean for a median in a news article and then tell me that half of values are necessarily above the mean. I will spot that shit in an instant and call you out in the comments!

Here is a short list of some skills being tested by the SAT:
  • Speed
  • Adapting quickly to the unexpected
  • Not getting tricked
  • Performing under pressure

I don't think this list itself is terribly controversial. Love or hate the SAT, I think everyone agrees that the exam requires you to answer tricky questions quickly under pressure.

The question is, how important are these skills, and for what?

That's an open question. The above sounds perhaps more like a list of qualities I would want when hiring a stockbroker than when selecting a student who will go on to major in English.

I would call this list reflective of one kind of person I want at universities. But I also want slower, deeper thinkers who take their time to come to well-researched conclusions. I want all intellectual kinds.

Of course, SAT scores are also correlated with income, although not as much as you would think. Here, the New York Times published a chart entitled, "A test of knowledge or income?" Kids whose parents make under $20,000 are getting an average of 442 per section. Kids whose parents make over $200,000 are getting an average of 571 per section -- nowhere near high enough for top universities. The income effect is actually nowhere near as large as I expected. There are other, major factors at work. (And, of course, income also affects every other aspect of school performance, not just SAT scores.)

It is also the case that the effect of income on SAT scores is well known, and that universities can and do adjust for it. Every prep school kid knows she's expected to get 200-300 points higher on the exam, since colleges "know" she's had tutors. The Wall Street Journal also reports that "on average, prep courses yield only a modest benefit" and that "SAT coaching resulted in about 30 points in score improvement on the SAT, out of a possible 1600..."

Whatever other effects -- personality type, a particular type of intellectual orientation -- account for test score variation are perhaps not being adjusted for by universities in a systemic way.

ETS, for business reasons, has a vested interest in convincing us that a time-limited standardized test can be made fair (and "school-like") enough that students, parents, and teachers will all find it a satisfactory measure of student readiness for college.

I doubt that entirely. Change the test however you want, people like me will beat it. I beat tests. I've also beaten the GRE and the GMAT. It's fun for me. But I think people like me are not necessarily the kind of people who are going to write books that will still be read in 100 years, or create a diverse campus and a flourishing environment of sincere and lively intellectualism.

On the other end of the spectrum, some people want to abolish the SAT entirely. Jennifer Finney Boylan suggests as much in the Times. wants all universities to drop the SAT or make it optional.

I don't think either of these -- SAT for everyone, or SAT for no one -- is the answer.

You know what would be a lot more progressive than any possible changes to the SAT? (Other than a serious War on Poverty, of the kind that hasn't been on the table since LBJ? But that's another article.)

A college admissions system in which students are offered even more radical options as to how they can demonstrate their learning, intelligence, and competence. For instance:

Most of the selective SAT-optional universities want some additional proof of learning in lieu of the SAT or ACT -- for instance, a graded essay from a high school course, or an entire writing portfolio. That's reasonable.

What about students who don't do well on timed exams but also aren't writers? I'm not sure what a math, science, or foreign language "portfolio" would consist of, exactly, but surely universities could accept evidence of outstanding achievement and dedication in fields other than writing. That, plus transcripts, extracurriculars, and admissions essays would make a complete package.

But I also want an option that's testing ONLY. Some kids don't show up for class not because they're slackers, but because they're obsessed with their work in the lab, or they have better things to read, or their teachers are awful -- or they just cannot stand to be condescended to for one more minute. Let's acknowledge that, just as some kids just crumble under test pressure, some kids cannot maintain the finely orchestrated, four-year-long social performance that is required to conform, make nice, and fulfill the arbitrary requirements of dozens of adult strangers, some of whom frankly should not be teaching, or who like to penalize students who challenge them.

Let colleges rescue those kids. Let colleges say, "This kid's grades are mediocre because he's a genius and he's bored out of his mind." To determine that, you would actually need a lot MORE tests, and harder ones. A battery of tests that could be considered instead of a transcript. If there's a kid who can skip her teacher's boring lectures and pick up all the material from a textbook in half the time, or who aces an SAT like it's nothing, I want that kid doing research at a university. I want that kid starting companies, curing diseases, inventing.

So let's get real. The changes to the SAT are probably salutary. But changes to the SAT don't solve the fundamental problems of how we educate students in high school, and how we select them for higher education.

Jennifer Dziura runs, an organization that provides "aggressive lady-advice" on careers and other topics, and is author of or contributor to over a dozen test prep and educational books.