During the 1990s, voucher advocates formed a number of organizations to provide privately funded vouchers to mostly urban, low-income students. The foundations supporting these groups also funded research conducted by Paul Peterson and colleagues in some other cities. The most prominent report to come out of this research to date has been the two-year study of private vouchers in New York City, Washington, D.C., and Dayton, Ohio released in August, 2000.34 More recently (2002), Peterson and colleagues completed and published results of a third year evaluation of private voucher programs in the same three cities.35
In both evaluations, Peterson and his colleagues found that African American students using vouchers to attend private schools scored significantly higher on combined reading and math tests than the control group students who had applied for vouchers but remained in public schools. However, the studies found no statistically significant positive or negative effects for Hispanic or white students using vouchers in spite of the numbers. Fifty-one percent of students in the New York City study were Hispanic and 24% in Dayton were white. In fact, the New York City data reveals that reading scores actually declined for all groups of Hispanic voucher students, except for those in 7th grade.36
The 2000 study was immediately criticized by researchers who pointed out a number of flaws. These included the presence of bias in the study. “Disappointment effects” among some parents who lost in the voucher lottery, as well as “selection bias” due to the fact that significant numbers of students dropped out of the study in subsequent years were cited as potential bias culprits. Another problem undermining study findings were the unexplained dramatic inconsistencies in achievement gains, where some groups do very well while others do poorly.37 Responding to his critics, Peterson was forced to use precisely those statistical corrections he claimed to be avoiding—thus undermining his claim that his research adhered to a ‘gold standard’ of social science research.38
Similarly, some researchers also questioned the validity of the 2002 study. Specifically, researchers pointed to data from Washington, D.C., which showed no achievement gains among black voucher students after three years. Additionally, there were large fluctuations in the Peterson-Howell study both among subgroups, and years of the study. Education Week reported that the data “raise as many questions as they answer” about students who receive vouchers.39 Even Peterson and Howell were less than enthusiastic about their results, concluding that there is “no overall private school impact” on the test scores of voucher students.40
Unexplained, Erratic Test Results
One of Peterson-Howell 2000 study’s partner agencies, Mathematica Policy Research, Inc. took the extraordinary step of releasing its own press statement entitled “voucher claims of success are premature in New York City.” Mathematica revealed without explanation, that, in New York City, all of the test score gains for African American students were concentrated in one of the four grades studied.41 Yet the improvement shown by this one subgroup was so marked that it demonstrated a statistically significant improvement for all grade levels of African American students when the four grades were averaged together. The truth is that most African American voucher students in New York did not do better than their public school peers.
Similarly erratic results were evident in Dayton in 2000, where students studied were in grades 3 through 9 in the second year of the program. African American students in three of the grades (3rd, 5th & 7th) showed large gains of between 14.5 and 19.7 national percentile points in combined math and reading scores while those in the other four grades (4th, 6th, 8th & 9th) actually declined nearly 15 percentile points.42
Back to Top
No Research Basis for Expanding Vouchers
Despite Peterson’s claim that this three-city study gave indications that vouchers might lead to the eradication of the achievement gap in this country, offers no convincing evidence that would support larger scale, publicly funded voucher programs. The authors themselves caution that generalizing results beyond this small, three city, privately-funded voucher program is problematic writing, “A much larger [voucher] program could conceivably have quite different program outcomes.”43
One reason that generalizations may not be drawn is the presence of what are known as “peer effects.” Much of whatever academic advantage private schools appear to have may likely be due to the fact that private schools can—unlike public schools—select and choose their students, thus affording some control over the peer environment in the schools. As few poor, urban students with vouchers enter these private schools—made up predominantly of children of higher socioeconomic status—there may be a positive “peer effect” at work that helps to boost academic achievement. To the extent this is true, it would suggest the possibility of potential gains for those few voucher students able to gain admission to established private schools and might explain the lower achievement scores produced by students attending “new” private voucher schools.44 Metcalf’s findings in Cleveland point to exactly this problem: voucher students in already established private schools did as well as their public school peers—better in one subject, science. But those who attended new schools—the ones opened precisely to accept voucher students—did worse in all subjects than their peers in both the older private schools and public schools.
Back to Top
Research on the Impact of Vouchers on Public Schools
In a 2001 study widely touted by advocates of private school vouchers, researcher Jay Greene claims that the threat of vouchers caused significant improvement in Florida public schools rated ‘F’ in 1999 on the state’s Florida Comprehensive Assessment Test (FCAT) exams.45 Greene established that F-rated schools showed greater gains than schools with higher grades, and went on to argue that it was the threat of vouchers that motivated these schools to do better. In fact, as several researchers have shown subsequently, the study has serious flaws and does not demonstrate that the improved test scores of students at these ‘F’-rated schools was caused by or in any way related to the threat of vouchers.
Significant methodological flaws in Greene’s analysis have been identified by Rutgers University researchers Gregory Camilli and Katrina Bulkley, including the fact that he used school-level rather than the more conventional student-level statistics ones yet asked that his resultant “effect size” data be interpreted as if student level statistics were used. This leads to greatly inflated estimates of the gains made by the ‘F’-rated schools. Camille and Bulkley found Greene’s conclusions to be “implausible.”46
A second study of Greene’s findings conducted by Haggai Kupermintz of the University of Colorado at Boulder, demonstrated that the “gains” made by Florida’s ‘F’-rated schools may have been more apparent than real. Kupermintz’s analysis supports what had become the conventional wisdom in Florida at the time, shortly after the FCAT test results were announced in June 2000.47 Specifically, schools knew what was required to avoid an ‘F’ grade: they needed only to pass any one of the three sections on the FCAT, reading, math or writing. Of the three test sections, writing was an easy target for specific test preparation. Because schools did not want to again be branded as “failing schools,” they took this route to escape failure, and they succeeded—unfortunately at the expense of actual classroom learning. As Kupermintz concludes, Greene’s report provides a “a false sense of a dramatic success” with respect to vouchers.48
Greene failed to pursue and investigate other plausible explanations and causes for the improved (FCAT) scores in ‘F’-rated schools. For example, Greene overlooked the significant amount of extra resources, both state and local, that were directed towards the ‘F’-rated schools. These enabled schools to reduce class size and extend the school day and year in order to help their students improve their performance, and may well have been one of the real causes of improvements in these schools.49
Back to Top