VI. Writing Subjective Test Items
Most teachers find that it is relatively easy to write subjective test item prompts as contrasted to objective ones. The difficulty lies in clearly specifying the task for the student so that grading is fair and equitable to all students. Some teachers find that the best approach is to write a sample answer and then analyze the elements of that answer. Alternatively, it is useful to ask a colleague to write a sample answer and critique the prompt. Writing good subjective items is an interactive, negotiated process.
The F/SL literature generally addresses two types of writing: free writing and guided writing. The former requires students to read a prompt that poses a situation and write a planned response based on a combination of background knowledge and knowledge learned from the course. Guided writing, however, requires students to manipulate content that is provided in the prompt, usually in the form of a chart or diagram.
Guided Writing | Free Writing | Writing Calibration Process
Guided writing is a bridge between objective and subjective formats. This task requires teachers to be very clear about what they expect students to do. Decide in advance whether mechanical issues like spelling, punctuation and capitalization matter when the task focuses on comprehension. Some important points to keep in mind for guided writing are:
· Be clear about the expected form and length of response (one paragraph, a 250-word essay, a letter etc.).
· If you want particular information included, clearly specify it in the prompt (i.e. three causes and effects, two supporting details etc.)
· Similarly, specify the discourse pattern(s) the students are expected to use (i.e. compare and contrast, cause and effect, description etc.)
· Since guided writing depends on the students manipulation of the information provided, be sure to ask them to provide something beyond the prompt such as an opinion, an inference, or a prediction.
· Be amenable to revising the anticipated answer even as you grade.
back to top
All of the above suggestions are particularly germane to free writing. The goal for teachers is to elicit comparable products from students of different ability levels.
· The use of multiple raters is especially important in evaluating free writing. Agree on grading criteria in advance and calibrate before the actual grading session.
· Decide on whether to use holistic, analytical or a combination of the two as a rating scale for marking.
· If using a band scale, adjust it to the task.
· Acquaint students with the marking scheme in advance by using it for teaching, grading homework and providing feedback.
· Subliminally teach good writing strategies by providing students with enough space for an outline, a draft and the finished product.
· In ES/FL classrooms, be aware of cultural differences and sensitivities among students. Avoid contentious issues that might offend or disadvantage students.
back to top
Writing Calibration Process
For test reliability, it is recommended that clear criteria for grading be established and that rater training in using these criteria take place prior to marking. The criteria can be based on holistic or analytical rating scales. However, whatever scale is chosen, it is crucial that all raters adhere to the same scale regardless of their personal preference.
The best way to achieve inter-rater reliability is to practice. Start early in the academic year by employing the marking criteria in non-test situations. Make students aware from the outset of the criteria and expectations for their work.
Collect a range of student writing samples on the same task and have teachers evaluate and discuss them until they arrive at a consensus score.
back to top
back to table of contents