Psst! Want to hear about the latest fiasco? Everywhere you look is another book, article, or blog post reminding us of the dubious success of meritocracy in the United States. Unending scandals—MF Global, Knight Capital, Madoffgate, Liborgate, Peregrine—suggest that the system ostensibly selecting and training society’s current leaders is seriously flawed. Even universities are not immune, as the recent scandal at Penn State has shown.
What’s wrong today is not meritocracy, per se, but rather our meritocracy. Our society uses an outdated and inadequate notion of merit. America, which relies so heavily on standardized tests as a means of entry into opportunity-expanding educational institutions, is at best a fractured meritocracy. The selection tests we use are based on too narrow a band of skills to provide a basis for a true meritocracy.
Research by Douglas Detterman and his colleagues at Case Western Reserve University has shown that tests like the SAT and ACT are IQ tests by another name. Such tests measure largely knowledge base (which is learned) and abstract analytical skills (also largely learned). They do not measure, or claim to measure, the additional kinds of skills that lead to true meritorious achievement in society, like common sense, creativity, wisdom, ethics, emotional intelligence, motivation to achieve, intellectual curiosity, persistence in the face of obstacles, and resilience in the face of failure. High-school grade-point average reflects at least some of those additional variables, but standardized tests do not adequately reflect any of them.
At Oklahoma State University, where I am provost and senior vice president, statistical analyses revealed that the widely used standardized test we rely on heavily in admissions, the ACT, adds virtually nothing statistically over high-school record in predicting first-year college grades. Moreover, we found that students with high scores on the test but low high-school grades are one of the groups most at risk for dropping out during the first year of college.
The problem is not with such tests themselves, but that we have asked too much of them. What we really need are tests that tell us what the IQ tests tell us, but also much more. But can that “more” be measured? Yes.
For example, here at Oklahoma State, the undergraduate admissions office, led by Vice President Kyle Wray, has a project, Panorama, that measures in our applicants, at least to a first-order approximation, analytical, creative, practical, wisdom-based, and ethical skills.
For instance, an ethics question is: “After submitting a class project, you realize one of your partners committed plagiarism. Your teacher previously announced that if he or she learned that cheating had occurred, all members of the work group would receive an F. How would you handle the situation and what would be your ideal outcome?” An example of a practical (common-sense) question is: “If you were able to open a local charity or business of your choice, what type of organization would it be and whom would it benefit? Describe your start-up process.” And an example of a creative question is: “Write a story or poem that includes one of the following sets of words: Purple, panic, panda, petunia, and popcorn; a stick, a light bulb, the Great Wall of China, and water; a bicycle, a clock, the Wild West, and duct tape.”
For Panorama, we drew on Tufts University’s experience in the Kaleidoscope Project, on which I collaborated with Lee Coffin, dean of undergraduate admissions, when I was dean of arts and sciences there. As summarized in my book College Admissions for the 21st Century (Harvard University Press, 2010), we found at Tufts that items similar to Oklahoma State’s Panorama prompts improve prediction of academic and extracurricular leadership beyond what we can derive from standardized-test scores and grades. The Kaleidoscope approach also has greatly reduced differences among results from various ethnic groups. Moreover, applicants just like it better.
In a national study, my colleagues and I, when I taught at Yale, found that what we then called “rainbow” assessments, measuring analytical, creative, and practical skills, essentially doubled prediction of freshman-year grade-point averages over conventional standardized tests and greatly reduced ethnic-group differences. Moreover, applicants and their parents feel better about the process because students can show skills beyond the narrow ones measured by conventional standardized tests. We also found that analytical or “academic” intelligence as measured by conventional tests is only minimally correlated with practical intelligence—that is, common sense.
America has found itself with a fractured “meritocracy” based on attributes that are a significant but small part of what people need to succeed and, more important, make the world a better place. People with high IQ’s on average may well be more successful in some respects, but they are no more likely than anyone else to use their abilities to help others. To create a real meritocracy, we need a broader range of tests that assess real merit.
Robert J. Sternberg is provost and senior vice president of Oklahoma State University, where he is also a professor of psychology, education, and leadership ethics.Return to Top