Don't Like Value-Added? Cool. So Pick Your Poison
As regular readers know, much of my writing on value-added dings would-be reformers for getting waaaay ahead of themselves. They're busy trying to build whole systems around tools that are crude, limited, and relevant for only a portion of what teachers and schools do. That's why I find it troubling that "reformers" are in a headlong dash to use these primitive systems to measure everything they can, or to validate everything else (observations, student feedback, etc.).
But--and here's the crucial "but"--value-added is a tool for measuring performance that may be useful in holding adults accountable for how well they do their job. Here, the question is not "is it perfect?" but "if, when, why, how, where, and why is it better than the alternatives?"
I was reminded of this simple point on Tuesday, when my discussion of Doug Harris's book on the strengths, limitations, and proper uses of value-added drew a scorching wave of furious comments insisting that any attempt to use value-added is dumb, inane, and probably malicious. The problem for these anti-value-added vigilantes is that there are only a limited number of ways to hold educators and schools accountable for their work, and it's not clear that they like any of them.
I'm writing for those who believe employees should be responsible for how well they perform their chosen work (if you reject this premise, so be it; no need to read any further). In that effort, value-added is one possible tool. Those who find it useful are obliged to address its problems and explain how to use it smart. Those who don't think it usefuI are obliged to offer serious, viable alternatives. What are the alternatives? Beyond value-added, I'd suggest five main ones.
There's principal evaluation. This asks accountable supervisors to take responsibility for their employees, though research has shown that principals usually punt, rating 99 percent of their teachers as terrific, come hell or high water. In principle, this is an attractive tool. Of course, the same folks who denounce value-added also tend to reject this one, arguing that supervisors may be dumb, biased, subjective, or eager to fill the teacher's job with a friend or relative.
There's level-based accountability, upon which value-added attempts to improve. As critics of NCLB have noted, the problem with holding teachers or schools responsible for achievement levels is that the result is only a faint reflection of their work; it also captures everything else in that child's life, and all previous years of schooling. Level-based accountability has the virtue of helping us see how kids are faring, but it's a troubling tool when used to evaluate individual teachers or schools.
There's student and/or parental feedback, which presumes that student surveys or parental information can provide valuable insight into teacher or school performance. The Gates Foundation's "Measures of Effective Teaching" project, for instance, has been working intensively with student surveys. These seem like potentially useful tools, though there are serious questions about reliability, validity, how these data are collected, and the rest.
There's choice-based accountability, where one leans less on supervisor judgment or on measuring particular dimensions of performance and more on the marketplace. This is accountability that results from families making choices about where they want to send their kids, with those dollars following along. This is decentralized, market-driven accountability. It is the least driven by bosses, policymakers, or simple measures, and in that sense it is particularly democratic. But questions immediately arise about whether families value the "right" things or are making informed choices, whether this can work in rural communities or those with a dearth of good schools, whether there are unintended consequences (like social fragmentation), and so on.
Finally, there's peer review, which many self-styled teacher advocates tend to like. They see it as sensible, professional, and fair. And I'm all for peer review, so long as it's identifying excellence and mediocrity and providing tough-minded accountability. However, few peer review efforts have lived up to their billing. For instance, as Steven Brill has reported, the lauded Toledo peer review program--which has been credited with aggressively weeding out bad teachers--turned out, when studied for The New Teacher Project's "Widget Effect," to have removed just one tenured teacher (in a fair-sized, low-performing system) during the two years studied. If peer review is providing toothsome accountability, then it's a swell option. But if teachers engage in peer review and nothing much happens, that doesn't cut it. This means that those who think peer review is the answer need to explain how parents and taxpayers know when peer review is really working and what happens when it's not.
My experience is that the same folks who lash out at value-added also pooh-pooh each of the alternatives (except a weak sauce version of peer review). Rather than recognizing that each approach has strengths and weaknesses, and that smart accountability is designed accordingly, they attend only to the potential flaws--and use those to reject each in turn. The result? What they're ultimately rejecting is not just the tool of value-added but the notion that public educators who are paid with public funds to serve the public's children ought to be responsible for how well they do their jobs. And I, along with the "reform" community, find that an unacceptable stance.
Bottom line: It's fair to note the limits of value-added and insist that this crude tool not be overused. It's also fair to critique and reject any of these other approaches in isolation. But serious critics can't serially reject every means of accountability without ever putting forward an actionable alternative. That adds up to nothing more than a recipe for public sector entitlement.
(Note: For anyone who's interested, I sketched my own broad stance on all this in 2004's Common Sense School Reform; today, I'm still pretty much where I was then.)