There are NAEP-related presentations and a training opportunity at the annual meetings of the American Educational Research Association (AERA) and the National Council on Measurement in Education (NCME). Below is a list of sessions from the AERA and NCME programs; you will find more detailed abstracts through their websites. At the conference, please check for schedule updates.
Be sure to stop by the NAEP booth, #421, on April 14 to 16, to learn more about NAEP, see demonstrations of the NAEP web tools, and browse the latest reports and publications!
8:00 a.m.–12:00 p.m. Fairmont Waterfront, MacKenzie 2
PDC11: Using NAEP Data on the Web for Educational Policy Research
Directors: Debbie Kline, Cathy Trapani, and Emmanuel Sikali
This course is for researchers interested in exploring the National Assessment of Educational Progress (NAEP) data using the NAEP Data Explorer Web tool. Participants will be guided through an examination of the data, focusing on: (1) student, teacher, and school variables; (2) relationships between student performance and characteristics of students, their teachers, and their schools; and (3) using NAEP data to supplement other education research. The audience will have the opportunity to work independently and share their findings with the group. The course will highlight 2011 math and reading data; 2010 civics, history, and geography data; and the most recent studies of science and writing. The course provides hands-on learning and active participation. A laptop or tablet computer with wireless Internet access is needed.
PAPER AND POSTER SESSIONS
8:15 a.m.–9:45 a.m. Marriott Pinnacle, Third Level/ Dundarave
National Databases, Science and Math Student Achievements, and a Methodological Assist.
One of five papers:
Reporting of Design Effects and Sample Weights: A Review of Published Early Childhood Longitudinal Study, Kindergarten Cohort and NAEP Articles
Mihaele Ene, Kshawna Askew, Bethany A. Bell
Secondary data analysis of nationally representative data is commonly used to address educational outcomes. As the frequency of these analyses continue to increase, omission of elements such as sample weights or design effects can lead to incorrect inferences. This study addresses the prevalence of researchers reporting the use of statistical techniques that take into account the complex sampling structure of survey data in published, peer-reviewed articles between 2000 and 2010. A total of 266 articles using data from two national education surveys (ECLS-K and NAEP) were selected for a systematic review. Preliminary results indicate that across the two data sources, 62.5% of the reviewed articles reported accounting for design effects and 75% reported using sample weights.
2:25 p.m.–1:55 p.m. Marriott Pinnacle, Third level - Dundarave
A Closer Look at the Hispanic–White Achievement Gaps Using NAEP Data
Chair: Cadelle Hemphill Discussant: Richard P. Duran
Overview of Achievement and Attainment Gaps Between Hispanic and White Students at the National Level
Examining Contextual Factors Related to the Hispanic White Achievement Gaps
Young Yee Kim, Chaturont Santrakul
The Hispanic White Achievement Gaps in the Five States with the Largest Hispanic Student Populations
Examining the Hispanic-White Achievement Gaps in Large Urban School Districts Using NAEP Data
The U.S. Hispanic student population has increased from 7 to 21 percent in the last 20 years. According to the National Assessment of Educational Progress (NAEP), the achievement gaps between Hispanic and White students have persisted since the early 1990s. These persistent achievement gaps have attracted educational policy makers and educational researchers’ concerns and interest. The papers in this symposium present different perspectives of the achievement gaps, using NAEP data: educational achievement and attainment gaps at the national level; the relationship between contextual factors and the achievement gaps; an in-depth look at the gaps in the five states with the largest Hispanic populations; and the gaps for 15 large urban districts participating in NAEP’s TUDA assessments.
2:15 p.m.–3:45 p.m. Sheraton Wall Centre, Third Level - North Junior Ballroom A
Discussing Educational Policy and Implications of Efficacy in the Arts as School Subject Areas: NAEP to Teacher-Made Assessment to Subject Area Growth
Chair: Read M. Diket; Discussant: Michael Sikes
Questions About Arts Learning That National Assessment of Educational Progress Can Help Answer
Read M. Diket
Policy Power: Teacher Assessment and NAEP Exemplars
How NAEP Data Explain Aspects of Arts Learning
The So Called Achievement Gap between White and Black Students on 2008 NAEP Visual Arts Assessment
Symposium panelists and discussant present statistical and qualitative analyses of NAEP Arts 2008 data from two arts subject areas, music and visual art. Due to financial limitations and program availability for national study, theatre and dance opportunities were queried by school questionnaires but not tested for art form achievement in 2008. The interval between art tests corresponds to major demographic changes in our society, and that, too, provides an opportunity to uncover how artistic understanding follows from affiliation within one’s micro culture interacting with expectations originating within a macro culture. Eighth grade represents a threshold, a tipping point, for adulthood understandings about a subject area and students’ proclivities to reflect on and create artistic forms.
8:15 a.m.–10:15 a.m. Marriott Pinnacle, Third Level, Dundarave
Achievement and Course Taking from HSTS; Engagement, Charter Schools, and SES Measure in NAEP
Chair: Edward Wiley; Discussant: Stacey S. Merola
Academic Achievement and Advanced Courses in High School: Do Schools Provide Access and Are Students Taking Advantage of It?
Stephen E. Roey, Judy H. Tang, Robert Colby Perkins, Janis D. Brown, Philip H. Morse
Not Being All They Can Be: A Comparison of Four-Year-College-Bound and Military-Bound High School Graduates
Robert Colby Perkins, Stephen E. Roey, Philip H. Morse
Examining Student Engagement and Motivation in Grade 12 NAEP Mathematics Assessment
Burhan Ogut, Ebony L. Walton, Enis Dogan, William Tirre
Comparing Public and Charter Schools Using Propensity Score Analysis
Reliability of Student-Reported Parental Education at NAEP Grade 8 Mathematics Assessment
Burhan Ogut, George W. Bohrnstedt
This paper session consists of five papers exploring various topics related to achievement by analyzing NAEP data. The first paper explores the relationship between achievement gaps and advanced courses in high schools from the 2009 NAEP HSTS data; the second paper analyzes course-taking and performance to examine high school graduates’ readiness for post-secondary success using the 2009 NAEP HSTS data; the third paper assesses the degree of student engagement and motivation on grade 12 NAEP mathematics assessment data; the fourth paper examines the differences between charter and public school students in grades 4 and 8 using NAEP assessment data; and the fifth paper investigates the reliability of student reported parental education on NAEP grade 8 student background questionnaire.
10:35 a.m.–12:05 p.m. Hyatt Regency, Floor Convention Level - Balmoral
Developing a NAEP Validity Research Framework
Chair: Cadelle Hemphill
Presenters: George W. Bohrnstedt, Andrew J. Kolstad, Frances Stancavage, Sharyn Rosenberg
Discussant: Peggy G. Carr
2:15 p.m.–3:45 p.m. Hyatt Regency, Plaza Level - Plaza C
Efforts to Improve Measurement Precision at the Lower end of the National Assessment of Educational Progress (NAEP) Scales
Chair: George W. Bohrnstedt
Presenters: Lizanne DeStefano, Kim Gattis, Andreas H. Oranje, Enis Dogan
Discussant: Lorrie A. Shepard
2:15 p.m.–3:45 p.m. Vancouver Convention Center, Second Level - West Room 205
The Unintended Impact of Education Policy Implementation
One of four papers:
Conditional Effects of Language Acquisition Policies on Academic Achievement of Hispanic English Language Learners
Francesca Lopez, Elizabeth McEneaney, Martina Nieswandt, Lara Geronime
NAEP (2009) achievement data in reading, math, and science is used to evaluate the effect of the bilingual emphasis of state language acquisition policy on academic achievement of 4th grade Hispanic English language learners who are eligible for federal lunch programs. Additionally, the analysis explores whether the impact of bilingual policy emphases (coded on a 0 to 7 scale) is conditional upon Hispanic presence in the state, namely the percent of Hispanic residents. Preliminary regressions with state-level data indicate that bilingual policy emphasis is negatively related to mean achievement, although the effect becomes significantly positive in states that also have a higher proportion of Hispanic residents. Multilevel models will be estimated when restricted-use data are available.
6:15 p.m.– 8:15 p.m. Marriott Pinnacle, Third Level - Shaughnessy II
NAEP Studies SIG Business Meeting: Future of NAEP
Program Chair: Young Yee Kim
10:35 a.m.–12:05 p.m. Vancouver Convention Center, First Floor - East Ballroom B
Poster Session 13:
Investigations of Science Learning
The quality of high school curriculum and academic success: Does coursework rigor translate into higher achievement?
Judy H. Tang, Stephen E. Roey, Robert Colby Perkins, Philip H. Morse
The present paper examines the extent to which high school graduates met course requirements for graduation by curriculum level with a closer look at current high school science curriculum related to student coursework by grade level, reasons for students not attaining more rigorous curriculum levels and possible relationships between curriculum levels and NAEP mathematics and science assessment scores.
2:15 p.m.–3:45 p.m. Marriott Pinnacle, Fourth Level - Ambleside
Validity of Accommodations for English Language Learners in NAEP
Chair and Discussant: Peggy G. Carr
Effectiveness, Validity, and Comparability of Accommodated and Nonaccommodated Assessments for English Language Learner Students in NAEP
Investigating the Effects of Direct Linguistic Support Accommodations for English Language Learner Students on a Mathematics Assessment
Mikyung Kim Wolf
Comparing the Performance of English Language Learners and Non-English Language Learners on Test Accommodations: A New Index to Evaluate Differential Boost
Maria J. Pennock-Roman, Charlene Rivera
Students Response Processes as Validity Evidence for the Accommodations for English Language Learners
Maria J. Martiniello
English language learners (ELLs) perform substantially lower in content-based assessments particularly those with higher level of language demand. Results of studies on the assessment and accommodations of ELL students clearly suggest that unnecessary linguistic complexity of assessments as a source of systematic error may negatively impact the reliability of assessments for ELL students and as a source of construct irrelevant affects validity of the assessments for ELLs (see, for example, Abedi, 2010). Therefore, the limited English proficiency may create unequal opportunities for students to fully demonstrate what they know. To level the playing field, legislations require accommodations—changes in the test process, in the test itself, or in the test response format.