Members of the GAISE Group:
Martha Aliaga, George Cobb, Carolyn Cuff, Joan Garfield (Chair), Rob Gould, Robin Lock, Tom Moore, Allan Rossman, Bob Stephenson, Jessica Utts, Paul Velleman, and Jeff Witmer
I Executive Summary
The American Statistical Association (ASA) funded the Guidelines for Assessment and Instruction in Statistics Education (GAISE) Project, which consists of two groups, one focused on K-12 education and one focused on introductory college courses. This report presents the recommendations developed by the college group. The report includes a brief history of the introductory college course and summarizes the 1992 report by George Cobb that since that time has been considered to be a generally accepted set of recommendations for teaching these courses. Results of a survey on the teaching of introductory courses are summarized along with a description of current versions of introductory statistics courses. We then offer a list of goals for students, based on what it means to be statistically literate. We present six recommendations for the teaching of introductory statistics that build on the previous recommendations from Cobb’s report. Our six recommendations are:
4. Foster active learning in the classroom;
5. Use technology for developing conceptual understanding and analyzing data;
The report concludes with suggestions for how to make these changes, and includes numerous examples in the appendix to illustrate details of the recommendations.
The GAISE project was funded by a Strategic Initiatives Grant from ASA in 2003 to develop ASA-endorsed guidelines for assessment and instruction in statistics in the K-12 curriculum and for the introductory college statistics course.
Our work on the college course guidelines included many discussions over email as well as in-person small group meetings. Our discussions began by reviewing existing standards and guidelines, relevant research results from the studies of teaching and learning statistics, and recent discussions and recommendations regarding the need to focus instruction and assessment on the important concepts that underlie statistical reasoning.
History and Growth of the Introductory Course
The modern introductory statistics course has roots that go back a long way, to early books on statistical methods. R. A. Fisher’s Statistical Methods for Research Workers, which first appeared in 1925, was aimed at practicing scientists. A dozen years later, the first edition of George Snedecor’s Statistical Methods presented an expanded version of the same content, but there was a shift in audience. More than Fisher’s book, Snedecor’s became a textbook used in courses for prospective scientists who were still completing their degrees: statistics was beginning to establish itself as an academic subject, albeit with heavy practical, almost vocational emphasis. By 1961, with the publication of Probability with Statistical Applications by Fred Mosteller, Robert Rourke, and George Thomas, statistics had begun to make its way into the broader academic curriculum, but here again, there was a catch: in these early years, statistics had to lean heavily on probability for its legitimacy. During the late 1960s and early 1970s, John Tukey’s ideas of exploratory data analysis brought a near-revolutionary pair of changes to the curriculum. First, by freeing certain kinds of data analysis from ties to probability-based models, so that the analysis of data could begin to acquire status as an independent intellectual activity, and second, by introducing a collection of “quick-and-dirty” data tools, so that, for the first time students could analyze data without having to spend hours chained to a bulky mechanical calculator. Computers would later complete the “data revolution” in the beginning statistics curriculum, but Tukey’s ideas of exploratory data analysis (EDA) provided both the first technical breakthrough and the new ethos that avoided invented examples. Two influential books appeared in 1978: Statistics, by David Freedman, Robert Pisani, and Roger Purves, and Statistics: Concepts and Controversies, by David S. Moore. The publication of these two books marked the birth of what we regard as the modern introductory statistics course.
The evolution of content has been paralleled by other trends. One of these is a striking and sustained growth in enrollments. Two sets of statistics suffice here: (1) At two-year colleges, according to the Conference Board of the Mathematical Sciences, statistics enrollments have grown from 27% of the size of calculus enrollments in 1970, to 74% of the size calculus enrollments in 2000. (2) The advanced placement exam in statistics was first offered in 1997. There were 7,500 students who took it that first year, more than in the first offering of an AP exam in any subject at that time. The next year more than 15,000 students took the exam, the next year more than 25,000, and the next, 35,000. In 2004 over 65,000 students took the AP statistics exam.
Both the changes in course content and the dramatic growth in enrollments are implicated in a third set of changes, a process of democratization that has broadened and diversified the backgrounds, interests, and motivations of those who take the courses. Statistics has gone from being a course taught from a book like Snedecor’s, for a narrow group of future scientists in agriculture and biology, to being a family of courses, taught to students at many levels, from pre-high school to post-baccalaureate, with very diverse interests and goals. A teacher in the 1940s, using Snedecor’s Statistical Methods, could assume that most students were both quantitatively skilled and adequately motivated by their career plans. A teacher of today’s beginning statistics courses works with a different group of students. Most take statistics earlier in their lives, increasingly often in high school; few are drawn to statistics by immediate practical need; and there is great variety in their levels of quantitative sophistication. As a result, today’s teachers face challenges of motivation and exposition that are substantially greater than those of a half-century ago.
Not only have the “what, why, who, and when” of introductory statistics been changing, but so has the “how.” The last few decades have seen an extraordinary level of activity focused on how students learn statistics, and on how we teachers can be more effective in helping them to learn.
The 1992 Cobb Report
In the spring of 1991 George Cobb, in order to highlight important issues to the mathematics community, coordinated an email Focus Group on Statistics Education as part of the Curriculum Action Project of the Mathematics Association of America (MAA). The report was published in the MAA volume, Heeding the Call for Change (Cobb, 1992). It included the following recommendations:
Emphasize Statistical Thinking
Any introductory course should take as its main goal helping students to learn the basic elements of statistical thinking. Many advanced courses would be improved by a more explicit emphasis on those same basic elements, namely:
The need for data
Recognizing the need to base personal decisions on evidence (data), and the dangers inherent in acting on assumptions not supported by evidence.
The importance of data production
Recognizing that it is difficult and time-consuming to formulate problems and to get data of good quality that really deal with the right questions. Most people don't seem to realize this until they go through this experience themselves.
The omnipresence of variability
Recognizing that variability is ubiquitous. It is the essence of statistics as a discipline and it is not best understood by lecture. It must be experienced.
The quantification and explanation of variability
Recognizing that variability can been measured and explained, taking into consideration the following: a) Randomness and distributions; (b) patterns and deviations (fit and residual); (c) mathematical models for patterns; (d) model-data dialogue (diagnostics)
More Data and Concepts, Less Theory and Fewer Recipes
Almost any course in statistics can be improved by more emphasis on data and concepts, at the expense of less theory and fewer recipes. To the maximum extent feasible, calculations and graphics should be automated.
Foster active learning
As a rule, teachers of statistics should rely much less on lecturing, much more on the alternatives such as projects, lab exercises, and group problem solving and discussion activities. Even within the traditional lecture setting, it is possible to get students more actively involved.
The three recommendations were intended to apply quite broadly, e.g., whether or not a course has a calculus prerequisite, and regardless of the extent to which students are expected to learn specific statistical methods. Although the work of the Focus Group ended with the completion of their report, many members of the Group continued to work on these issues, especially on efforts at dissemination and implementation, as members of a joint ASA/MAA Committee on Undergraduate Statistics.
Current Status of the Introductory Statistics Course
Over the decade that followed the publication of this
report, many changes were implemented in the teaching of statistics. In recent
years many statisticians have become involved in the reform movement in
statistical education aimed at the teaching of introductory statistics, and The
National Science Foundation funded numerous projects designed to implement aspects
of this reform (Cobb, 1993).
In 1998 and 1999,
The results of this survey suggested that major changes were being made in the introductory course, that the primary area of change was in the use of technology, and that the results of course revisions generally were positive, although they required more time of the course instructor. Results were surprisingly similar across departments, with the main differences found in the increased use of graphing calculators, active learning and alternative assessment methods in courses taught in math departments in two year colleges, the increased use of web resources by instructors in statistics departments, and the reasons cited for why changes were made (more math instructors were influenced by recommendations from statistics education). The results were also consistent in reporting that more changes were to be made, particularly as more technological resources became available.
Today’s introductory statistics course is actually a family of courses taught across many disciplines and departments. The students enrolled in these courses have different backgrounds (e.g., in mathematics, psychology), and different goals (e.g., some hope to do their own statistical analyses in research projects, some are fulfilling a general quantitative reasoning requirement). As in the past, some of these courses are taught in large classes, some are taught in small classes or even Freshman seminars. Some students are taught statistics in computer labs, some students take the course using only a simple calculator, and some take the course via distance learning without ever seeing their classmates or instructor in person. Some classes are taught over a ten-week quarter some are taught over 15-week semester. Each of these classes might range from three to six hours per week.
Today’s goals for students tend to focus more on conceptual understanding and attainment of statistical literacy and thinking, and less on learning a set of tools and procedures. While demands of dealing with data in an information age continue to grow, advances in technology and software make tools and procedures easier to use and more accessible to more people, thus decreasing the need to teach the mechanics of procedures but increasing the importance of giving more people a sounder grasp of the fundamental concepts needed to use and interpret those tools intelligently. These new goals, described in the following section, reinforce the need to reexamine and revise many introductory statistics courses, in order to help achieve the important learning goals for students.
III Goals for Students in an Introductory Course: What it Means to be Statistically Educated
Some people teach courses that are heavily slanted toward teaching students to become statistically literate and wise consumers of data; this is somewhat similar to an art appreciation course. Some teach courses more heavily slanted toward teaching students to become producers of statistical analyses; this is analogous to the studio art course. Most courses are a blend of consumer and producer components, but the balance of that mix will determine the importance of each recommendation we present.
The desired result of all introductory statistics courses is to produce statistically educated students which means that students should develop statistical literacy and the ability to think statistically. The following goals represent what such a student should know and understand. Achieving this knowledge will require learning some statistical techniques, but the specific techniques are not as important as the knowledge that comes from going through the process of learning them. Therefore, we are not recommending specific topical coverage.
Students should believe and understand why:
Students should recognize:
Students should understand the parts of the process through which statistics works to answer questions, namely:
Students should understand the basic ideas of statistical inference:
Finally, students should know:
We endorse the ideas in the three original goals found in the Cobb report (Cobb et al., 1992) and have expanded them in light of today’s situation. The intent of these recommendations is to help students attain the list of learning goals described in the previous section.
We define statistical literacy as understanding the basic language of statistics (e.g., knowing what statistical terms and symbols mean and being able to read statistical graphs), and understanding some fundamental ideas of statistics. For readings on statistical literacy see Gal (2002), Rumsey (2002), and Utts (2003).
Statistical thinking has been defined as the type of thinking that statisticians use when approaching or solving statistical problems. Statistical thinking has been described as understanding the need for data, the importance of data production, the omnipresence of variability, and the quantification and explanation of variability (Cobb, 1992). We provide illustrations of statistical thinking in the following example and analogy.
The Funnel Example
Think of a funnel that is wide at the top, corresponding to a great many situations, and narrow at the bottom, corresponding to a few specialized cases. Statisticians are practical problem solvers. When a client presents a problem (e.g., Is there a treatment effect present?), the statistician tries to provide a practical answer that addresses the problem efficiently. Quite often a simple graph is sufficient to tell the story. Perhaps a more detailed plot will answer the question at hand. If not, then some calculations may be needed. A simple test based on a gross simplification of the situation may confirm that a treatment effect is present. If simplifying the situation is troublesome, then a more refined test may be used, capturing more of the specifics of the modeling situation at hand. Different statisticians may come up with somewhat different analyses of a given set of data, but will usually agree on the main conclusions and will only worry about minor pints if those points matter to the client. If there is no standard procedure to answer the question, then and only then will the statistician use first principles to develop a new tool. We should model this type of thinking for our students, rather than showing them a set of skills and procedures and giving them the impression that in any given situation there is one best procedure to use and only that procedure is acceptable.
The Carpentry Analogy
In week 1 of the carpentry (statistics) course we learned to use various kinds of planes (summary statistics). In week 2 we learned to use different kinds of saws (graphs). Then we learned about using hammers (confidence intervals). Later we learned about the characteristics of different types of wood (tests). By the end of the course we had covered many aspects of carpentry (statistics). But I wanted to learn how to build a table (collect and analyze data to answer a question) and I never learned how to do that. We should teach students that the practical operation of statistics is to collect and analyze data to answer questions.
Suggestions for teachers:
In the appendix we present examples of projects, activities, and assessment instruments that can be used to develop and evaluate statistical thinking.
It is important to use real data in teaching statistics, for reasons of authenticity, for considering issues related to how and why the data were produced or collected, and to relate the analysis to the problem context. Using real data sets of interest to students is also a good way to engage them in thinking about the data and relevant statistical concepts. There are many types of real data including archival data, classroom-generated data, and simulated data. Sometimes hypothetical data sets may be used to illustrate a particular point (e.g., the Anscombe data illustrates how four data sets can have the same correlation but strikingly different scatterplots) or to assess a specific concept. It is important to only use created or realistic data for this specific purpose and not for general data analysis and exploration. An important aspect of dealing with real data is helping students learn to formulate good questions and use data to answer them appropriately based on how the data were produced.
Suggestions for teachers:
Search for good raw data to use from web data repositories, textbooks, software packages, as well as from surveys or activities in class. If there is an opportunity, seek out real data directly from a practicing research scientist (through a journal or at one’s home institution). Using such data can enliven your class and increase the store of good data sets for other teachers by communicating the newly found data to others.
The Appendix includes examples of good ways (and examples of not so good ways) to use data in homework, projects, tests, etc.
Many introductory courses contain too much material and students end up with a collection of ideas that are understood only at a surface level, are not well integrated and are quickly forgotten. If students don’t understand the important concepts, there’s little value in knowing a set of procedures. If they do understand the concepts well, then particular procedures will be easy to learn. In the student’s mind, procedural steps too often claim attention that an effective teacher could otherwise direct toward concepts.
Recognize that giving more attention to concepts than to procedures may be difficult politically, both with students and client disciplines. However, students with a good conceptual foundation from an introductory course are well-prepared to go on to study additional statistical techniques in a second course such as research methods, regression, experimental design, or statistical methods.
Suggestions for teachers:
Perform routine computations using technology to allow greater emphasis on interpretation of results. Although the language of mathematics provides compact expression of key ideas, use formulas that help to enhance the understanding of concepts, and avoid computations that are divorced from understanding. For example, helps students understand the role of standard deviation as a measure of spread and to see the impact of individual y values on s, whereas has no redeeming pedagogical value.
Using active learning methods in class is a valuable way to promote collaborative learning, allowing students to learn from each other. Active learning allows students to discover, construct, and understand important statistical ideas and to model statistical thinking. Activities have an added benefit in that they often engage students in learning and make the learning process fun. Other benefits of active learning methods are the practice students get communicating in the statistical language and learning to work in teams. Activities offer the teacher an informal method of assessing student learning and provide feedback to the instructor on how well students are learning. It is important that teachers not underestimate the ability of activities to teach the material or overestimate the value of lectures, which is why suggestions are provided for incorporating activities even in large lecture classes.
Types of active learning include:
Suggestions for teachers:
· Ground activities in the context of real problems. Therefore, data should be collected to answer a question, not “collect data to collect data” (without a question).
· Intermix lectures with activities, discussions and labs.
· Precede computer simulations with physical explorations (e.g., die rolling, card shuffling)
· Collect data from students (anonymously).
· Encourage predictions from students about the results of a study that provides the data for an activity before analyzing the data. This motivates the need for statistical methods. (If all results were predictable, we wouldn’t need either data or statistics.)
· Do not use activities that lead students step by step through a list of procedures, but allow students to discuss and think about the data and the problem.
· Plan ahead to make sure there is enough time to explain the problem, let the students work through the problem, and wrap up the activity during the same class. It is hard to complete the activity in the next class period. Make sure there is time for recap and debriefing, even if at the beginning of the next class period.
· Provide lots of feedback to students on their performance and learning.
· Include assessment as an important component of an activity.
Suggestions on implementing active learning in large classes:
· Take advantage of the fact that large classes provide opportunities for large sample sizes for student generated data.
· In large classes it may be easier to have students work in pairs rather than in larger groups.
· Use a separate lab/discussion section for activities, if possible.
Technology has changed the way statisticians work and should change what and how we teach. For example, statistical tables such as a normal probability table are no longer needed to find p-values and we can implement computer-intensive methods. We think that technology should be used to analyze data, allowing students to focus on interpretation of results and testing of conditions, rather than on computational mechanics. Technology tools should also be used to help students visualize concepts and develop an understanding of abstract ideas by simulations. Some tools offer both types of uses, while in other cases a statistical software package may be supplemented by web applets. Regardless of the tools used, it is important to view the use of technology not just as a way to compute numbers but as a way to explore conceptual ideas and enhance student learning as well. We caution against using technology merely for the sake of using technology (e.g., entering 100 numbers in a graphing calculator and calculating statistical summaries) or for pseudo-accuracy (carrying out results to multiple decimal places). Not all technology tools will have all desired features. Moreover, new ones appear all the time.
Suggestions for teachers on ways to use technology:
Things for teachers to consider when selecting technology tools:
· Ease of data entry, ability to import data in multiple formats
· Interactive capabilities
· Dynamic linking between data, graphical, and numerical analyses
· Ease of use for particular audiences
· Availability to students, portability
See the Appendix for an example illustrating technology uses.
Students will value what you assess. Therefore assessments need to be aligned with learning goals. Assessments need to focus on understanding key ideas and not just on skills, procedures, and computed answers. This should be done with formative assessments used during a course (e.g., quizzes and midterm exams and small projects) as well as with summative evaluations (course grades). Useful and timely feedback is essential for assessments to lead to learning. Types of assessment may be more or less practical in different types of courses. However, it is possible, even in large classes, to implement good assessments.
Types of assessment:
Suggestions for teachers:
Suggestions for student assessment in large classes:
See the Appendix for examples of good assessment items and suggestions for improving weak items.
Making it happen
Statistics education has come a long way since Fisher and Snedecor. Moreover, teachers of statistics across the country have generally been enthusiastic about adopting modern methods and approaches. Nevertheless, changing the way we teach isn’t always easy. In a way, we are all teachers and learners alike, a bit like hermit crabs: In order to grow, we must first abandon the protective shell of what we are used to, and endure a period of vulnerability until we can settle into a new and larger set of habits and expectations.
We have presented many ideas in this report. We advise readers to move in the directions suggested in this report by taking small steps at first. Examples of small steps are
Your teaching philosophy will inform your choice of textbook, but the recommendations in this report are not about choosing a text, but about a way of teaching.
There are many resources available, including the MAA Notes volumes that deal with teaching statistics, the Consortium to Advance Undergraduate Statistics Education (CAUSE) (causeweb.org), the Isostat discussion list (http://www.lawrence.edu/fac/jordanj/isostat.html), the SIGMAA- Stat Ed group within the MAA (http://www.pasles.org/sigmaastat/), and the ASA website, especially the Center for Statistics Education (http://www.amstat.org/education/) and the Statistical Education Section (http://www.amstat.org/sections/educ/).
GAISEing into the Future
A good deal of progress has been made, but there is still plenty of room to improve the introductory statistics course. Moreover, this course must be flexible and adaptable to change as more students enter college having learned aspects of statistics in elementary and secondary school. The Advanced Placement course continues to change the statistics education landscape. Although we have been addressing the general introductory course, we must be mindful of other courses, such as business statistics and mathematical statistics, and of the content and goals of good second courses in statistics that build on the solid conceptual understanding developed in the first course.
Cobb, George. (1992). Teaching statistics. In Lynn A. Steen (Ed.), Heeding the call for change: Suggestions for curricular action (MAA Notes No. 22), 3-43.
Cobb, G. (1993), ‘Reconsidering Statistics Education: A National Science Foundation Conference’, Journal of Statistics Education 1 (1)
Garfield , J. (2000) Evaluating the Statistics Education Reform. Final Report to the National Science Foundation. http://education.umn.edu/EdPsych/Projects/Impact.html
Rumsey, D. J. (2002). Statistical Literacy as a Goal for Introductory Statistics Courses. Journal of Statistics Education [Online], 10(3).
Utts, J. (2003). What educated citizens should know about statistics and probability? The American Statistician, 57 (2), 74-79.