Introduction: What is Data-Driven Decision-Making (DDDM) and what should be considered before selecting and implementing Electronic Learning Assessment Resources (ELAR)? The following questions should be understood and/or addressed prior to making the time and funding investment to use an ELAR in your school or district:
1. Establishing a district/school level commitment to DDDM.
2. Implementing the California curriculum content frameworks and related standards.
3. Using objective assessments to determine the extent to which students attain the standards.
4. Using local assessments and related data to plan for individual student needs.
5. Considering information about the demographic and instructional variables that influence student performance as determined from the academic assessment components of DDDM.
6. Developing instructional units and lessons based on DDDM.
7. Making relevant assessment data available to educators for planning.
8. Providing professional development and ongoing support for school administrators and teachers in using assessment information to inform instructional planning at the district, school, and classroom levels.
9. Providing for technology applications that will equip educators with the current student assessment data on demand and in a user-friendly format.
A. What are Data-Driven
Decision-Making (DDDM) and Data Literacy? A DDDM focus uses student assessment
data and relevant background information, to inform decisions related to planning
and implementing instructional strategies at the district, school, classroom,
and individual student levels. “Data literacy” means that a person possesses
a basic understanding of how data can be used to inform instruction.
Back to Top
B. Why is DDDM important?
Research shows that if instructional plans at the state, county, district,
school, classroom, and individual student levels are based on assessment information
relevant to the desired learning outcomes for students, the probability is
increased that they will attain these desired learning outcomes.
Back to Top
is data literacy important? Educators must understand that
assessment of student performance is integral in the planning, implementation,
assessment, and revision of instruction. Data literacy also implies that
the educator must be able to determine whether or not an assessment is a valid
and reliable measure of what is being taught and to know what types of assessments
are appropriate for district level vs. classroom or individual student level
planning. What is commonly referred to as a “cycle of inquiry” illustrates
the basic steps in the application of data to inform instructional decision-making.
Back to Top
D. What is necessary for an education agency to implement DDDM? Teachers and school administrators need to be data literate. This can happen only when the district/school leadership actively encourages and assists educators in understanding the importance of relating valid student assessment information to instructional practice. A culture must be created in which student information is considered essential to improving student results, and education policy makers, education leaders and practicing educators need to agree on the following:
Back to Top1. Establishing district level commitment to DDDM: The school district should adopt policies and strategies to ensure that all necessary resources and support services are funded and implemented to guide planning and related instruction through use of objective and relevant student assessment information. School leadership must understand that use of objective and relevant student assessment data will significantly increase the probability that learning will occur. This will result when cause and effect are understood, and when using student information to ask “Why?” and “What can we do better?” becomes standard in the school.
2. Implementing the California curriculum content frameworks and related standards: Implementation of the state content standards is a mandate at the state and federal level as dictated by No Child Left Behind. All school districts in California are adopting the California Content Standards and purchasing instructional materials that align to these standards. It may be assumed that the majority of instructional time is allocated to teaching to these standards.
3. Using objective assessments to determine the extent to which students attain the standards: The state has adopted the California Standards Test (CST) as a measure of the extent to which groups of students meet standards at the state, district, and school levels by grade level in English Language Arts, Math, and Science. The California High School Exit Exam (CAHSEE) is a proficiency assessment used to determine whether or not a student is eligible to graduate from high school. It is recognized that these assessments do not provide the information needed to determine specific instructional activities at the classroom or student level. However, the state assessments can be used to determine overall gaps in how a particular school compares with schools of similar demographics at a particular grade level in the subject area tested. The state assessments can be used to inform DDDM by assisting educators to identify a gap or area where students at a school are generally not meeting the state standards, and then moving to a deeper level of investigation. This means using alternative sources of data to help define the specific instructional needs at the classroom and student levels.
4. Using local assessments and related data to plan for individual student needs: Now that state assessment has helped pinpoint the school(s), grade level(s), subject area(s), and perhaps classrooms needing additional assistance, it is necessary to move to local and alternative assessments. Such assessments provide sufficient detail enabling educators to develop specific educational plans or plan revisions to address the general deficiencies identified by the state assessments. Following are some assessment strategies that school districts have adopted to enable DDDM for the purposes of classroom and student level instructional planning informed by assessment data.a. Locally developed performance tests based on the state content standards;b. Performance measures embedded in the adopted instructional materials used in the school;c. The use of standards-aligned rubrics to judge student performance;d. Selection of standards-aligned test items from published item-banks for the construction of locally relevant academic assessments;e. Teacher-assigned grades assigned based on guidelines that reflect state content standards;f. Other measures to be determined.
5. Considering information about the demographic and instructional variables that influence student performance as determined from the academic assessment components of DDDM: The use of student data alone is not sufficient to design instructional plans that effectively address the individual instructional needs of students. Information systems for DDDM must also provide data on student demographics: the amount of time in the district, socioeconomic status, achievement motivation and other behavioral indicators, parent involvement, attendance, and other factors as needed. Other considerations in DDDM are conditions under which instruction can be provided: availability of instructional resources, class size, access to technology, adequacy of school facilities, etc. These factors are often referred to as “opportunity to learn” variables.
6. Developing instructional units and lessons based on DDDM: In order for data to influence instructional practice, the school must institute a process whereby teachers continuously monitor assessment data and other information relevant to student academic performance, and translate that information into the delivery of curriculum and instruction. This component of DDDM requires the greatest effort on the part of educators, as the design and re-design of instruction based on student assessment data is time consuming and tedious. However, recent developments in technology applications can make this task less burdensome. Some basic considerations in preparing or revising instructional units and practice based on DDDM are:a. Are instructional unit objectives linked to the state content standards?b. Are technology-based and other supplements linked to the adopted text?c. Are assessments embedded into the instructional resources linked to the instructional unit objectives?d. Are the results of available state assessments considered and linked to any local assessments and are these referenced in the unit?e. Is periodic assessment of student performance provided, recorded and linked back to the unit objectives?f. Are the assessments valid in accurately assessing what students learn in relation to the standards?g. Is there a commitment to implementing units of instruction and administering the unit-prescribed assessments as scheduled?h. Do teachers collaborate on students’ progress and compare notes on using assessments in planning instruction?i. To what extent is technology used to help automate the design and monitoring of instructional units?j. Are teachers provided with the time and resources needed to plan, modify, and implement units based on DDDM?k. Is there significant value, evidenced by teachers and the school level administrators, in using student data and background information to inform instructional planning?
7. Making relevant assessment data available to educators for planning: The school district must be committed to using DDDM as a pre-requisite to the effective and sustained use of data to inform instructional planning at the school and classroom levels. This means the district will make the funding and staff time available to assist schools in the use of DDDM. It also means that district staff will make every effort to make the relevant assessment and demographic data and information available to schools when they need it to do planning. This includes the provision of longitudinal student data so that change in student performance over time can be determined and used in the local DDDM instructional planning.
8. Providing professional development and ongoing support for school administrators and teachers in using assessment information to inform instructional planning at the district, school, and classroom levels: As with any new initiative, the school district must provide the professional development necessary to build the capacity and desire of each educator to utilize DDDM. Professional development must extend beyond simply using data to inform others about student performance. It must ensure that data is used to design and revise instructional practice so that it more closely conforms to the state standards and results in increased student performance. Finally, educators need to be committed to DDDM as a strategy to help students improve their performance to the extent that the “performance gaps” or deficiencies documented by the CST and other state measures are eliminated.
9. Providing for technology applications that will equip educators with the current student assessment data on demand and in a user-friendly format: Staff and evaluators for the California Learning Resource Network (CLRN) have determined that a wide variety of technology applications are available, with many more emerging, that are intended to assist educators in making data-driven decisions as suggested. For this reason a new service has been added to CLRN. That is a “clearinghouse” of information that will help educators identify and select an Electronic Learning Assessment Resources (ELAR). These assessment tools provide assistance in accessing, analyzing, and making data-driven decisions using student data, and range from the district to the classroom levels.
E. How does CLRN’s ELAR information resource support DDDM? The ELAR component of CLRN provides an online “consumer guide” intended to assist educators in selecting data management resources which meet their local needs and state and federal reporting requirements. The ELAR component of CLRN also provides an emerging source of information resources to help educators become data literate and learn how to make the most effective use of an ELAR. The ELAR resource provides:
F. What types of Electronic Learning Assessment Resources are reviewed by ELAR? The ELAR staff and advisors have adapted the definitions established by CTAP 4 in collaboration with the Stupski Foundation to describe the categories relating to ELAR reviews. A few ELAR programs incorporate all five categories but most address fewer than the following five categories:
Back to Top1. Operational Data Systems (Student Information Systems): Operational systems are concerned primarily with day-to-day administration issues, such as tracking student attendance, grades, course enrollment, etc. SASI and Eagle are examples of operational data systems, which are transactional systems that update student data as soon as it changes, usually erasing older data. Although some analytical functions typically are built into operational data systems, they generally do not meet all of a district's data analysis needs. Many ELAR programs will import student demographic data from a SIS.
2. Direct Instructional Systems: Direct instructional systems, such as Accelerated Math, provide educational content directly to students and teachers in the classroom via electronic media. CLRN classifies these direct instructional systems as Electronic Learning Resources (ELR) as differentiated from Electronic Learning Assessment Resources (ELAR). CLRN has reviewed at least 40 ELR which have content-related embedded student assessment systems. In some cases those assessments can be linked to ELAR.
The ELAR website address is www.clrn.org/elar
For additional information regarding this report and/or implementation of ELAR contact:
John Cradler, Educational Support Systems and External Evaluator for the CLRN Statewide Educational Technology Service. email@example.com
Bob Beuthel, Learning Resources and implementation manager of the ELAR service component of CLRN. firstname.lastname@example.org
Brian Bridges, Stanislaus County Office of Education and
Director of the California Learning Resource Network (CLRN). email@example.com