A central tenet of the student learning outcomes "movement" is that higher education institutions must articulate a specific set of skills, traits and/or dispositions that all of its students will learn before graduation. Then, through legitimate means of measurement, institutions must assess and publicize the degree to which its students make gains on each of these outcomes.
Although many institutions have yet to implement this concept fully (especially regarding the thorough assessment of institutional outcomes), this idea is more than just a suggestion. Each of the regional accrediting bodies now requires institutions to identify specific learning outcomes and demonstrate evidence of outcomes assessment as a standard of practice.
This approach to educational design seems at the very least reasonable. All students, regardless of major, need a certain set of skills and aptitudes (things like critical thinking, collaborative leadership, intercultural competence) to succeed in life as they take on additional professional responsibilities, embark (by choice or by circumstance) on a new career, or address a daunting civic or personal challenge. In light of the educational mission our institutions espouse, committing ourselves to a set of learning outcomes for all students seems like what we should have been doing all along.
Yet too often the outcomes that institutions select to represent the full scope of their educational mission, and the way that those institutions choose to assess gains on those outcomes, unwittingly limit their ability to fulfill the mission they espouse. For when institutions narrow their educational vision to a discrete set of skills and dispositions that can be presented, performed or produced at the end of an undergraduate assembly line, they often do so at the expense of their own broader vision that would cultivate in students a self-sustaining approach to learning. What we measure dictates the focus of our efforts to improve.
As such, it’s easy to imagine a scenario in which the educational structure that currently produces majors and minors in content areas is simply replaced by one that produces majors and minors in some newly chosen learning outcomes. Instead of redesigning the college learning experience to alter the lifetime trajectory of an individual, we allow the whole to be nothing more than the sum of the parts -- because all we have done is swap one collection of parts for another. Although there may be value in establishing and implementing a threshold of competence for a bachelor’s degree (for which a major serves a legitimate purpose), limiting ourselves to this framework fails to account for the deeply held belief that a college experience should approach learning as a process -- one that is cumulative, iterative, multidimensional and, most importantly, self-sustaining long beyond graduation.
The disconnect between our conception of a college education as a process and our tendency to track learning as a finite set of productions (outcomes) is particularly apparent in the way that we assess our students’ development as lifelong learners. Typically, we measure this construct with a pre-test and a post-test that tracks learning gains between the years of 18 and 22 -- hardly a lifetime (the fact that a few institutions gather data from alumni 5 and 10 years after graduation doesn’t invalidate the larger point).
Under these conditions, trying to claim empirically that (1) an individual has developed and maintained a perpetual interest in learning throughout their life, and that (2) this lifelong approach is directly attributable to one’s undergraduate education probably borders on the delusional. The complexity of life even under the most mundane of circumstances makes such a hypothesis deeply suspect. Yet we all know of students that experienced college as a process through which they found a direction that excited them and a momentum that carried them down a purposeful path that extended far beyond commencement.
I am by no means suggesting that institutions should abandon assessing learning gains on a given set of outcomes. On the contrary, we should expect no less of ourselves than substantial growth in all of our students as a result of our efforts. Designed appropriately, a well-organized sequence of outcomes assessment snapshots can provide information vital to tracking student learning over time and potentially increasing institutional effectiveness. However, because the very act of learning occurs (as the seminal developmental psychologist Lev Vygotsky would describe it) in a state of perpetual social interaction, taking stock of the degree to which we foster a robust learning process is at least as important as taking snapshots of learning outcomes if we hope to gather information that helps us improve.
If you think that assessing learning outcomes effectively is difficult, then assessing the quality of the learning process ought to send chills down even the most skilled assessment coordinator’s spine. Defining and measuring the nature of process requires a very different conception of assessment – and for that matter a substantially more complex understanding of learning outcomes.
Instead of merely measuring what is already in the rearview mirror (i.e., whatever has already been acquired), assessing the college experience as a process requires a look at the road ahead, emphasizing the connection between what has already occurred and what is yet to come. In other words, assessment of the learning that results from a given experience would include the degree to which a student is prepared or “primed” to make the most of a future learning experience (either one that is intentionally designed to follow immediately, or one that is likely to occur somewhere down the road). Ultimately, this approach would substantially improve our ability to determine the degree to which we are preparing students to approach life in a way that is thoughtful, pro-actively adaptable, and even nimble in the face of both unforeseen opportunity and sudden disappointment.
Of course, this idea runs counter to the way that we typically organize our students’ postsecondary educational experience. For if we are going to track the degree to which a given experience “primes” students for subsequent experiences -- especially subsequent experiences that occur during college -- then the educational experience can’t be so loosely constructed that the number of potential variations in the order of a student experiences virtually equals the number of students enrolled at our institution.
This doesn’t mean that we return to the days in which every student took the same courses at the same time in the same order, but it does require an increased level of collective commitment to the intentional design of the student experience, a commitment to student-centered learning that will likely come at the expense of an individual instructor’s or administrator’s preference for which courses they teach or programs they lead and when they might be offered.
The other serious challenge is the act of operationalizing a concept of assessment that attempts to directly measure an individual’s preparation to make the most of a subsequent educational experience. But if we want to demonstrate the degree to which a college experience is more than just a collection of gains on disparate outcomes – whether these outcomes are somehow connected or entirely independent of each other – then we have to expand our approach to include process as well as product.
Only then can we actually demonstrate that the whole is greater than the sum of the parts, that in fact the educational process is the glue that fuses those disparate parts into a greater -- and qualitatively distinct -- whole.
Mark Salisbury is director of institutional research and assessment at Augustana College, in Illinois. He blogs at Delicious Ambiguity.
In many respects, higher education in the United States – with credits awarded on time a student sits in a chair – remains trapped in the 19th century and has been slow to embrace technology.
Online education from traditionally accredited colleges has been available since at least 1999, but almost always at the same high tuition cost as the traditional “physical” courses. New ideas, such as tuition-free massive open online courses (MOOCs), are now emerging, but are generally not accredited.
For a true revolution to occur, regulation will need to change along with the technology. The key advance would be to establish a new private sector accrediting body, the “Modern States Accrediting Agency,” that would ensure the quality and reputation of the innovative courses, make the credits transferable into the traditional system and which would be recognized by the U.S. Department of Education as an approved accreditor in order to qualify students for federal student aid.
The Department of Education began its first online education pilot program in 1999. In 2006, it allowed institutions to offer all of their courses online. However, these courses were offered by institutions accredited in the traditional way, with student enrollment in the courses kept limited, and with tuition set as high (or even higher) than tuition for the physical alternative.
The paradigm began to shift in 2011 when Stanford University offered three of its courses online, free of charge, to any person anywhere who chose to take them.
Since then other innovators have continued to introduce MOOCs, notably led by the nation’s most respected traditional universities, such as MIT and Harvard. MIT, for example, is creating MITx, which is intended to offer a great number of MIT’s courses free of charge or nearly so; taught by MIT’s renowned faculty; and with graded assignments, tests, online discussion groups, online professor “office hours” and other quality advances.
The problem with MOOCs, though, is that there is usually no mechanism for obtaining accreditation and, in U.S. higher education, accreditation is the “coin of the realm,” which gives a degree its value. As a result, most MOOCs can offer students only a letter of completion, a pat on the head and no degree. Few other institutions or graduate schools will recognize completion of a MOOC course for credit, and employers do not know how to judge the student’s level of accomplishment.
Schools like MIT should not be forced to dilute the power of their brand by being forced to give their regular degree to students who simply take some of their tuition-free online courses. However, it is equally inappropriate to give no value to the online learning that occurs in a MOOC, particularly if a student can complete a high-quality, rigorous course and then prove mastery of the material on a separate, proctored, certifying exam.
In the traditional system, a degree is accredited because the degree-granting institution is itself accredited by an agency recognized by the U.S. Secretary of Education. The accrediting agencies (such as the Middle States Commission on Higher Education or the New England Association of Schools and Colleges) are private sector, self-regulatory groups which, in most cases, were created nearly a century ago by the member institutions themselves. The best of these agencies were later recognized by the Department of Education and included on its list of approved accreditors.
Today students can only qualify for federal financial aid if their institution has been accredited by one of these recognized agencies, and these accreditors’ decisions control access to the more than $150 billion in federal aid paid out to students each year. The traditional accrediting agencies, which were founded long ago to serve the needs of the traditional institutions, are not well-suited to lead technological and social innovations that are alternatives to the traditional system. A few experiments with traditionally accredited MOOCs are under way.
However, for the most rapid and effective progress, America needs a new, innovation-focused accreditor, Modern States, which would also be recognized by the Department of Education and which could accredit providers of emerging technologies and ideas in order to drive down costs, drive up quality and to shape federal aid programs in new and effective ways.
Unlike traditional accreditors, Modern States would be able to accredit specific courses, not just the degree-granting institution as a whole. For example, it could recognize that the freshman physics MOOC from MITx is of high quality, and then develop a widely available, proctored test for students who complete that course, similar to an SAT exam or CPA exam.
Students who complete the preapproved, tuition-free MOOC and also pass the confirmatory Modern States assessment would earn accredited course hours from Modern States itself. Enough such courses in the right scope and sequence (say physics from MITx, poetry from Harvard, theology from Notre Dame and so on) could lead to a fully accredited Modern States degree. Modern States would also approve courses and develop tests in vocational areas, in career training fields and at the two-year and community-college level, in order to serve all types of students.
The creation of Modern States could then enable a whole field of academic innovations to bloom, including blends of “bricks and clicks” and new types of federal financial aid models. For example, students might take their core lectures tuition-free and online from a nationally renowned professor in a MOOC, and then attend supplementary weekly study groups with a live professor and other students in their home towns, all at a lower overall cost than a traditional course today.
Similarly, students might take their first year of introductory courses all online for free, but then transfer to a traditional college for the last three years, lowering their total educational costs by 25 percent. Students who complete their educations at a low cost or no cost to the federal government might even be paid a federal bonus upon completing their degree and successfully entering the work force. In this way, the self-motivated Abe Lincolns of tomorrow could finish their higher educations debt-free and with cash in the bank. Meanwhile, U.S. taxpayers would save money.
Modern States could also lead the way in areas unrelated to MOOCs, such as competency-based exams and on-the-job skills training. For example, if there is a national shortage of skilled welders, and if an employer trains a worker in welding who then passes the Modern States assessment, the worker could earn a course credit in welding while the employer itself might be paid some stipend as an “educational institution of one.”
The traditional accreditors were founded by their member institutions, but – with a few exceptions – traditional institutions are not likely to be the best champions for low-cost alternatives to themselves. Modern States should be chiefly formed by a voluntary association of philanthropies and nongovernmental organizations concerned with increasing access to high-quality education while lowering its cost -- groups such as the Bill and Melinda Gates Foundation, the Ford Foundation, the World Bank and so on. Employer and labor groups could join as well, as could providers of the innovative courses, student consumer groups and others.
Membership contributions could fund a staff that would develop a starting catalog of approved courses from the already-existing universe of MOOC and related offerings. The staff would then work with testing organizations to develop the independent assessments needed to prove student mastery of the material. The catalog would cover a range of academic and vocational fields, and grow and evolve over time.
The ultimate path to success for Modern States would be to keep its testing standards high and rigorous so that employers, traditional institutions and the world at large will see that the students are truly deserving of the degree credits. In this way, Modern States could function like a universal version of the CPA exam developers, who have become an accepted standard of testing and quality in the accounting field.
Once Modern States is formed, it would write criteria for granting accreditation that would be aligned with the Secretary of Education’s criteria for recognition of accrediting agencies. After applying these criteria, Modern States could petition the Department of Education for recognition. The petition would be reviewed by staff and by a federal advisory committee that advises the Secretary on whether to recognize accreditors. If the Department grants recognition, then students at the institutions and programs accredited by Modern States would be eligible to participate in federal student aid programs.
Transferability of credits to more traditionally accredited programs would be negotiated by Modern States through reciprocity agreements with other accreditors, as supported by the Department of Education. Modern States would be a private-sector organization, not a government organization. However, political leaders in both parties could help achieve educational goals by expressing clear support for the Modern States approach as outlined here.
In its best form, traditional higher education is one of America’s great treasures, and no online program is ever likely to equal the experience of four years on campus at a great school. However, the high cost and limited availability of such traditional best-in-class programs have placed them increasingly out of reach to many striving students in America and around the world. By unleashing the power of technology and social innovation, Modern States could be the key regulatory mechanism to make education more accessible and affordable, and to bring higher education more fully into the 21st century.
David Bergeron, the former acting assistant secretary for postsecondary education at the U.S. Department of Education, is vice president of postsecondary education policy at the Center for American Progress.
Steven Klinsky, a New York-based businessman and philanthropist, has been active in education reform since 1993.
With great interest, I read the recent news announcing that the American Council on Education (ACE) had evaluated five Coursera MOOCs and recommended them for credit. But I had hoped for something different.
Having traditional prestigious institutions making their online content open to the world – of course without their prestigious credit attached – was an exciting development. A race to post courses ensued. On the surface, it’s an altruistic move to make learning available to anyone, anywhere for free.
Dig deeper and we are left to ask, how many MOOC courses will really be worth college credit, where will the credits be accepted, and for how long will college credits even be the primary measurement of learning?
Now that ACE has evaluated a few courses, MOOC providers will see how their process goes as students start actually finding proctors and taking tests -- or finding other methods of assessment -- to prove they learned the material. But a few courses will not be enough to really help students earn degrees, and with MOOC courses and providers continuing to proliferate, this does not seem like a viable way to keep up with demand.
Regardless, it is more than likely that the universities that agreed to the ACE CREDIT review are never going to accept an ACE CREDIT transcript themselves. The students with ACE CREDIT transcripts will need to present those transcripts to “lesser known” schools that are not among the elite players – colleges with much lower tuition and a willingness to serve post-traditional students.
More troubling is the fact that the ACE process for credit review is still course-based. Will this really be flexible enough in the future? Will it measure competencies and individual learning outcomes? Even if it seems scalable, will it mean all MOOC evaluations have to run through ACE and only ACE? Will students have to wait until ACE has evaluated a MOOC course before they can get credit?
Moreover, this raises the question: Are course evaluations and testing really the best or only way to deal with this new era of learning? What about experiential learning? If someone has college-level learning from their life experience is it invalid unless they take a course?
As Inside Higher Ed points out in its article, this was a fast move in an industry that moves at a glacial pace. But when ice really begins to melt, it can quickly turn into a waterfall. Students have more options for learning, and can get more information, from a variety of sources. So the question for education becomes, how can we best accommodate that?
I would assert that a portfolio assessment of students’ learning is the best way. Just as an artist shows a portfolio to a prospective employer, students should be able to demonstrate learning from wherever they have learned -- work, MOOCs, informal training, military service, volunteer service, and more -- all in one place. And much of this learning will not involve a course at all.
If MOOCs are to be truly disruptive, they must link to competencies, credentials, degrees and/or ultimately jobs. Using a course-by-course, credit hour-by-credit hour approach to do this will not dramatically change the way people earn degrees. And dramatic change that allows for individual demonstrations of competencies is the only way to provide the education quality and agility necessary to truly recognize learning derived from free resources on the web. By focusing on competencies, we can align and accept learning experiences from everywhere.
Pamela Tate is president/CEO of the Council for Adult and Experiential Learning.