KALAMAZOO, MI – A Western Michigan University professor, who was asked to review an investigative report by The Atlanta Journal-Constitution claiming to find suspect math or reading scores in school systems across the country, is saying the journalists didn’t do enough homework.
Atlanta Journal-Constitution’s affiliate Dayton Daily News asked Gary Miron, a WMU education professor known for his evaluation of education policy and reforms, to review Ohio test data last week. He said the journalists had only reached the ‘the first step’ of their investigation when they published the report on Saturday, March 24.
Miron shared his concerns with the journalists but after the story was published, he sent a letter explaining his concerns about the lack of erasure and student-level data in the report to The Washington Post’s Valerie Strauss, who published it the day after the article ran.
Miron was an external advisor for an award-winning USA TODAY investigation last year, which found that Washington D.C.’s Crosby S. Noyes Education Campus test scores drastically improved from 2006 to 2010 was because school administrators were erasing wrong answers to improve their school rankings.
“It’s sensationalism,” Miron said. “That’s not responsible journalism and after seeing what was done with USA TODAY; this is night and day. This was the first step, but USA TODAY went back and got student level data and erasure data.”
The Atlanta Journal-Constitution report does admit that their methods of analysis did not “prove cheating but reveals that test scores in hundreds of cities followed a pattern that, in Atlanta, indicated cheating in multiple schools.”
Miron says the analysis may accurately detect large swings in test scores from year to year but the cause of the irregularities “are less likely due to actual cheating than due to mobility in student population and other factors.”
He says by not making sure the same students were being tested year after year, the report assumes the wide variations in test data roots from cheating.
"The analysis shows that in 2010 alone, the grade-wide reading scores of 24,618 children nationwide — enough to populate a midsized school district — swung so improbably that the odds of it happening by chance were less than one in 10,000," the AJC article reads.
The Atlanta Journal-Constitution's Editor Kevin Riley said their report did consider mobility by removing small classes and classes with large enrollment changes and checked if there were sudden population shifts that could explain scores.
"Mobility is the reason Atlanta initially cited for highly suspicious score changes. In that case, of course, state investigations later determined widespread cheating was to blame," Riley said. "Not all states perform erasure analysis and not all states are willing or able to share student-level data, so those conditions would rule out a national analysis."
"Basically without the erasure and student-level data, the report is likely comparing the test results of different students," Miron said. "They are alleging or leading readers to believe that cheating is rampant and I don’t think it is and they don’t have the evidence to prove it."