Nearly every day of my professional life I'm engaged in some manner of discussion about the "quality" of education, either at the Calhoun School or elsewhere. Whether as seen in broad educational policy research, in independent school admissions or in U.S. News and World Report college rankings, our society grapples constantly with the elusive process of assessing schools.
Perhaps the greatest fallacy in educational assessment is the nearly unchallenged idea that you can judge the quality of the process by the product. For example, conventional wisdom holds that the median SAT scores and colleges attended by graduates of a particular secondary school serve as an accurate barometer of the quality of the education. Quite to the contrary, SAT scores and college acceptances have almost nothing at all to say about the quality of education in secondary schools.
Consider this analogy: Let us imagine that the most important measure of success in school is not SAT scores or colleges attended, but foot speed. Most parents with the luxury to choose will naturally seek a school that seems to produce fast runners. One way a school might achieve this end is to begin with an admissions process that seems a fair race: Line up 1,000 young applicants at one end of a football field and have them sprint to the other. Admit the 50 who finish first. Next year do the same.
Furthermore, at the end of each subsequent year, line up the students in each grade and have them race again. Kick out (politely, of course) the five or 10 who finish last. Perhaps they lagged in growth, gained a few pounds during puberty or peaked early and really weren't born to run fast. Hold races among new applicants to fill the vacated slots. This process will inevitably produce one of the fastest graduating classes in town.
I hope the parallel is obvious, even if the variables are more complex. Many of the supposed "best" private schools admit students with the highest demonstrated test-taking ability, usually the Independent School Entrance Examination. That they subsequently score well on SATs is hardly surprising. Other predictors of high SATs and Ivy League admission also are carefully analyzed -- things like colleges attended by the parents and a family's social and economic standing (the most reliable predictor of all).
Having thus selected students with the most powerful predictors for the predetermined measures of success, all a school must thereafter do is line them up and reassess from time to time, culling out those whose predictors lag as they grow. At the end of this process, pop the champagne and celebrate the inevitable. This does not mean, of course, that schools with 50 percent Ivy acceptances and whose graduates boast very high SATs are bad schools. Many of them are very fine schools. But the self-fulfilling results don't prove a cause and effect relationship. Those students' scores and college acceptances would be likely whether the education provided was indifferent and unimaginative or inspirational and dynamic.
The same fallacious reasoning distorts college rankings and the assessment of public education. The reputations and rankings of schools like Harvard, Yale and Stanford soar on the achievements of their graduates: Elite graduate schools attended, earning power, subsequent professional achievement and so on. Again, no disparagement intended, but such schools are even more fastidious in their selection of only those who are the very most likely to succeed in the precise way that success has been pre-defined.
And the college ranking systems that use criteria like the average SAT scores of entering freshmen or the number of applicants they reject are patently ridiculous. Neither of these things says anything at all about the quality of education the college offers.
It's rather like a great violinist (a Juilliard graduate) friend of mine who said this about the Juilliard School. "It's not like the students learn how to play there. The greatest thing Juilliard can do is to not ruin them."
Suburban public schools in wealthy neighborhoods are "good" because 90 percent of graduates go to college. Urban public schools are "bad" because 90 percent don't go to college. Powerful social dynamics produce these self-fulfilling results just as do private school admissions, by sorting America's families into neighborhoods by the very same selection: well-educated parents with advantageous social and economic standing. The selection perpetuates itself circularly. The families with all the advantages are drawn to communities, in part, because of the alleged quality of the schools, which are deemed good primarily by virtue of the families who were drawn to the advantaged community.
Nothing about this has anything to do with good educational practice and we should stop pretending it does.