Universities are falling down on the job, a new study attests, failing to teach America's college students the crucial skills they need to succeed throughout their lives.
The study, summarized in the new book "Academically Adrift" and in a paper out this week from the Social Science Research Council, tracked several thousand college students' performance on a critical-thinking test from their first year on campus through their fourth year. Some 45% of students showed no significant improvement after two years of college, and 36% didn't improve after four years—troubling numbers that made headlines this week.
That message about America's universities plays into beliefs that the U.S. education system isn't preparing students for an increasingly competitive world. But other researchers point out that the study relied on limited data that might offer an incomplete snapshot of the caliber of U.S. undergraduates. In addition, the findings are rooted in questionable assumptions about what constitutes a successful education.
The test only sought to gauge critical-thinking ability, which the study's authors said was a crucial indicator of college achievement. It didn't evaluate how much students learned about their area of study.
The study also had no way of requiring students who took the critical-thinking evaluation as freshmen to sit for the follow-up test as seniors. And, in fact, more than half of the first-time test takers didn't participate in the second go-round.
Catharine H. Beyer, a research scientist in the office of educational assessment at the University of Washington, says a quantitative analysis of progress in critical thinking isn't the best barometer of how colleges are faring. A "generic test can't capture" the kind of learning unique to each subject, she says. Rather than use a single metric, she says each college department needs to set goals and determine whether they are being met.
The researchers counter that while every study, and every standardized test, has its flaws, they took pains to measure college learning as accurately as possible. The researchers say that 24 participating colleges, which aren't named in the study, make up a representative sample of colleges nationwide. Researchers used an open-ended written test, rather than multiple-choice, to assess a broader range of abilities.
The researchers also say their findings are corroborated by their survey of test takers that found a lack of academic rigor in most students' schedules. For example, they determined that half of U.S. college students didn't have to write a 20-page paper in a typical semester. One-third of undergraduates studied alone five or fewer hours per week.
"Given those levels of effort, are the test-score results at all surprising?" asks Richard Arum, co-author of the study and book, and a sociologist at New York University. He notes that several studies suggest students are spending less time studying, reading and writing than they have in the past. And he cites a recent analysis conducted by a consortium of universities that showed no significant improvement in critical thinking after four years—in line with Prof. Arum's findings.
Even if the numbers on student learning are valid, some critics question whether they really implicate undergraduate education. The researchers based their conclusion in part on comparisons with college students a generation or more ago. For instance, researchers found that an average student, who entered college at the 50th percentile in critical thinking among incoming freshmen, would have risen to just the 68th percentile—when compared to his classmates' first-year scores—after four years in school. In the 1980s, the new study points out, researchers found that incoming freshmen improved to the 84th percentile in critical thinking by their fourth year.
But what if students today are entering college with sharper critical-thinking skills to begin with? Or perhaps their inferior skills were caused by inadequate high schools, rather than faulty colleges.
Prof. Arum acknowledges that the current study doesn't have a direct historical analogue. "It is indeed quite possible that there have always been significant numbers of students who have made it though college with little effort and small learning gains," he says.
What the new study does tell us is how scarce hard numbers are on undergraduate achievement.
The new study made use of an exam known as the Collegiate Learning Assessment, which made its debut in 2004. It asks students to write analytical essays and also to perform critical-thinking tasks; the latter was the basis for the study. Some education researchers say the test shows promise, but shouldn't be used as the sole basis to compare colleges.
Without any widely available data on college teaching success, college rankings generally are based on reputation, incoming test scores such as SATs and other factors. "Colleges should do more to show what they know about outcomes" such as learning and employment, says Alexander C. McCormick, director of the National Survey of Student Engagement, another college study. "The discourse about quality is unhelpfully dominated by reputation and resources."
Write to Carl Bialik at firstname.lastname@example.org