KALAMAZOO, MI -- Results from the latest round of the Michigan Assessment Program tests were recently released to the public, and the reaction was completely predictable.
School communities with strong scores are patting themselves on the back. Those with disappointing results are downplaying the importance of standardized test scores.
"MEAP is a complete joke!!! The absolute only thing this test proves is the districts with involved parents have better scores," one commenter posted on one of my MEAP stories on MLive.
"So what have we learned since we started MEAP testing that has made a positive effect on student learning?" posted another.
To be sure, there is value to standardized tests. "It's hard to improve what you don't measure," a school administrator once told me, and there's truth in that.
Personally, I do think the MEAP has had a positive effect on student learning. When the state began emphasizing MEAP results in the 1990s, it forced much more consistency in curriculum from district to district, school to school, classroom to classroom.
Because of the MEAP writing test, schools have improved their writing instruction. Because of the science test, there is more emphasis on critical-thinking skills. Because of the sanctions associated with No Child Left Behind, educators and the general public are paying more more attention to the achievement gaps between middle-class and low-income students.
But while the MEAP is a useful tool, its importance also can be overstated. As measures go, it's inexact, a snapshot in time. People can, and often do, read too much into results.
A few years ago, I wrote a column on reasons to take MEAP scores with a grain of salt, and those points are worth repeating.
1. Sociodemographics. The biggest factors that influence academic outcomes are outside of teachers' control: Family income, race, whether the mother went to college, how many books are in the home, a family's child-rearing practices, the neighborhood where the child resides, and a family's social circle and the child's friends.
On the MEAP tests administered in 2013, 81 percent of middle-class students in grades 3-8 scored as proficient in reading compared to 55 percent of Michigan students who qualify for the subsidized lunch program, a 36-point gap. No surprise, then, a district's demographics have a huge impact on MEAP scores.
2. The idiosycrasies of a pass/fail test. The MEAP results reported to the public are "proficiency" rates -- i.e. the percent who pass the test.
But there's definitely a margin of error in this, especially in regard to the number of kids who barely pass vs. barely fail -- not to mention the arbitrary nature of determining what score determines "proficiency."
Consider, for instance, that it's entirely possible for school districts to see their average test score rise and their proficiency rate fall, or visa versa.
Take this year's results on the seventh-grade reading test for Portage Public Schools. The passage rate dropped from 80 percent in 2012 to 76 percent in 2013. Yet the district's mean score actually inched up slightly, from 741.5 to 742.3.
The pass/fail nature of the test also means teachers don't get credit for improving the achievement of students who start out way behind, and they get credit for students who are already advanced.
Consider the fifth-grade teacher who has a class of advanced readers; those students are likely to pass the MEAP regardless of the quality of instruction. By contrast, a teacher with a kid who is multiple grade-levels behind has little chance of getting the child to proficiency. The latter point is particularly true in the secondary grades, where struggling students have more ground to make up.
3. Small sample size. More than half of the school districts in the Kalamazoo Gazette's circulation area test fewer than 100 children per grade; in 10 of the 36 districts, it's less than 50. In systems that size, it doesn't take much to move the numbers up or down.
For instance, Burr Oak students scoring as proficient in fifth-grade math went from 37 percent in 2012 to 56 percent in 2013 -- a whopping increase. In actuality, it was a difference of three more kids who passed the test.
4. Timing of the test. The MEAPs are administered in early October, and that's to provide test results to schools in time for them to adjust their curriculum for the next school year.
But that also means the test is really a reflection of what kids learned in the previous grade — and if children attended a different school, it reflects last year's building compared to where they are now.
5. Comparing different groups of kids. The standard practice is to compare MEAP scores by grade — i.e., how did this year's fourth-graders do compared to last year's fourth-graders?
The problem, of course, especially in small districts, is that the difference in ability levels of this year's students compared to last year might account for a rise or fall in MEAP numbers — yet the school gets all the credit or the blame.
For instance, Climax-Scotts' proficiency rate in seventh-grade math took a dive from 51 percent in 2012 to 28 percent this year. Yet last year's seventh-graders had the best math scores in the district. Meanwhile, only 17 percent of this year's seventh-grade passed the MEAP math test as sixth-graders, which means that cohort actually improved this year.
6. Importance of curriculum alignment -- i.e. teaching to the test.
"Teaching to the test" has a bad rap, but it's not necessarily a bad thing. After all, classroom teachers use tests all the time to assess whether students have mastered material, and one would expect the instruction to match the tests.
The difference with the MEAP is that it's a external exam. And sometimes poor MEAP results are a result of a mismatch between classroom instruction and the MEAPs -- students haven't learned the material yet or it's not being taught in the same way it's being tested, so students get confused by the exam.
The good news for districts: Improving curriculum alignment is an excellent strategy for raising MEAP scores. The downside: Curriculum alignment can be a challenge, especially in small districts that lack a curriculum director or another person with the time and expertise to figure out and implement an improvement plan.
Bottom line: The MEAP is one slice of a much bigger pie. It tells us something about a district or a school or about instruction statewide, but it's far from a complete picture.
Search tips: - You can search by one or any combination of the following categories - "District Wide" in the school name results is the total number of students in the district, and may include buildings/ student counts that were too small to be counted on their own. It is a true representation of total - The symbol: < indicates less than 10 students were tested - A blank indicates no tests were administered, or less than 10 students were tested - Non-public schools are associated to school districts that operate within the geographical area these schools reside in.
Please leave any questions in the reader comments below.