The Obama administration's efforts to implement policies to address access and cost issues in higher education are well-intended. Yet in truth, they put the cart before the horse. Many of these ideas are drawn from the health policy world, and a significant amount of work must be done first to understand the impact of these proposals. Otherwise, they have the potential to do serious harm if put into place prematurely.
Let's start with the idea that federal dollars should be tied to educational outcomes. This is the administration's most ambitious proposal, though how it would work remains unspecified. In explaining its plans, the administration has borrowed terminology such as "Pay for Performance" from the health policy sector, an area in which I worked before moving to higher education. The higher education community could learn a lot from health care's efforts to control costs and increase quality. But the administration must err on the side of caution before proposing crude policies without knowing more about how they would work and whether they would achieve intended goals.
Tying "pay" to "performance" sounds good, but how would it work in practice? What outcomes would be used as proxies for college? The Obama administration has proposed three: completion rates, graduate earnings and advanced degrees of graduates. For the sake of argument, let's ignore the conceptual drawbacks of taking any of these as a valid measure of whether a college is doing a good job. An incentive system only works if the outcome is truly a measure of quality -- of how well the college performs for its students.
There are many problems with simplistically attributing these outcomes to the institution only. Consider two hypothetical institutions: Silver Spoon College and Bootstrap College. Both have the same outcomes except one: Silver Spoon's graduates have higher salaries five years after graduating than those from Bootstrap. Silver Spoon's students are also more likely to come from upper class families with large reserves of social and financial capital. Its legacy admissions policy, which favors applicants born to alumni, leads to an overrepresentation of those students at the college. Many Silver Spoon graduates use their social and financial capital to secure higher paying jobs after graduation.
Here lies the problem. A federal system of "pay for performance" would attribute Silver Spoon College's better outcomes, measured by graduate earnings, to the college alone. This increases its federal dollars, while decreasing the flow of dollars to Bootstrap College. This policy would reduce overall social mobility by penalizing the institution (Bootstrap) that provides opportunity to students with less social and financial capital -- those needing an education to improve their economic condition.
How can we correct this? In considering outcomes, we need the educational equivalent of the valid, risk-adjusted mortality rates leveraged in health care. These would be sophisticated risk adjustments that tease out the effect of the education the college provides, as opposed to the effect of factors outside of its control. Using mortality rates as a measure of hospital care quality is comparable for many reasons. Gross outcomes like mortality result from how risks directly associated with the patient (e.g., smoking history or chronic disease) interact with the quality of care provided -- just as college outcomes result from how student-specific factors interact with the quality of education the college provides. Risk-adjusted mortality rates help us better assess hospital quality and have informed government policies and interventions to improve quality of care at underperforming hospitals. Yet this is not a perfect solution. Concerns still remain in the health care industry about the problems of using risk-adjusted mortality rates as a proxy for quality and tying them to reimbursement rates.
So why not use a set of valid, well-tested risk-adjusted college outcomes tools? Because it does not exist. The Gates Foundation has funded important work in this area, but much of the policy discussion has focused on students and institutions at the low end of the performance range. Little attention has been devoted to differences among institutions at the middle and high end of the performance distribution, where hard-to-measure factors like social class may play a greater role.
We are far from a consensus about tools to help adjust outcomes for variations in input risk. And we lack the evidence base needed to reasonably deploy the type of "Pay for Performance" system the administration has proposed -- a fact that should be obvious in Washington.
Instead of criticizing colleges with neat sound bites, the administration should be working with educators to build the evidence base that our education system needs. This is the only way that institutions can tackle the hard task of ensuring that every dollar is used in the most effective, efficient way for future generations.