Looking back at this posting from 2010 – I wanted to highlight the key reasons why even close to three years on – these points are still very relevant.
- Students haven’t changed: Ask any academic – as soon as student leaves the class – the chance of getting them to respond to a survey is low. In class surveys are the best way to capture a high response rate.
- Using the right survey methodology: Course and module evaluations (50 students or less) requires a high response rate in order to avoid ‘non response bias’. In this scenario – paper is best. Whereas with 0verall satisfaction surveys, where thousands of students are responding – gaining a high response rate is not as important. In this instance the online survey approach is best. This is especially important if you are looking to identify the specific programme the student is on.
- Combating the threat from social media: With the advent of Twitter and sites such as ratemyprofessor.com, it is more important than ever for universities to protect the reputation of their academics with robust evidence based course evaluation based upon valid results. If students do not believe the institution is interested in their feedback and transparent in implementing changes – then there is real threat that their opinions enter the public domain through social media. Universities must take a lead in protecting their reputations and becoming the trusted source for their students regarding improving the quality of teaching and learning.
6 Key Things to Consider When Managing Paper Based Course Evaluations
Many higher and further education organisations are unable to provide computers in every class or provide online surveys in computer labs for logistical reasons. For that reason paper based ‘in-class’ surveys are the best option in driving response rates. For organisations choosing paper surveys there are some key points to consider.
1 – Gain stakeholder buy-in as the key incentive to participate.
The best way to gain active participation in the survey is to clearly articulate the vision and purpose of the survey and what actions will result. It is important to communicate this from the executive of the institution to both the students and of equal importance, to the instructors.
For students: They need to know that their participation is a key element of the overall quality enhancement effort and that decisions and actions will take place based upon the outcomes.
For Instructors: In paper survey administrations, the instructor’s play a big part in administering the survey to the students. If the Instructors are suspicious of the process or feel that the responses will be used against them, they have the opportunity to influence the results. Instructors need to understand that the results will not be used against them, moreover, results will be used to help instructors improve their teaching. If this message is clarified and supported by the executive vision statement and throughout the organisation, they will participate and assist in the process.
2 – Create surveys that make sense and report on what you need to know.
A well designed questionnaire that has clearly defined question group objectives and valid question content helps the student engage and provide the best feedback. In addition, the questions need to make sense and follow through to provide the key indicators that are unique to each institution’s reporting requirements. Unique to paper survey administration is the need to reduce the number of pages of a questionnaire in order to minimise the impact on the administrative staff tasked with scanning through results. Reducing the length of the questionnaire also makes it easier for the students to complete and again provides the best feedback results. Remember to avoid closed (yes/no) questions and create as many open ended scale questions as you can in order to gain a deeper understanding of the students attitudes and perceptions. Do not rely too heavily on open comments as they are difficult to quantify and students lose interest if asked to constantly comment.
3 – Reinforce the concept of anonymity to get honest student feedback
A good way to create stakeholder buy in from the students is to ask that the instructors read a short prepared statement that highlights the importance of the survey and the action that will be taken from the results. In addition the procedure should include the instructor leaving the room while the students complete the survey. A student representative should be asked to collect the completed surveys and deliver the surveys in the provided sealed envelope to the administration offices. By taking these additional steps the concept of anonymity is reinforced because the opportunity for instructor influence is minimised.
4 – Use in-class time
Educational institutions have an advantage in driving response rates owing to the opportunity they have to ask the survey respondents to complete the surveys during class. There are three main points to consider as to why this leads to increased survey responses.
1.Students are used to being asked to complete assignments in class so there is a natural tendency to comply with the request.
2. Administering the survey during class time will result in the students taking time in providing their feedback. Remember to remind the instructors to avoid handing out the survey at the end of class as this will lead to rushed responses and partially completed results.
3. Having student’s complete surveys in class takes advantage of the herd instinct which exerts tremendous pressure in a social collective to do what the majority of your peers do. Thus when handing out a survey in-class the natural tendency is to complete the survey.
5 – Standardising the course evaluation process will help avoid headaches later on.
It is understandable that many higher and further education organisations question the way they run their course evaluations. This tendency is influenced by the sometimes conflicting regulatory requirements of the various governing bodies. It is important not to lose sight of the value of quality enhancement and the need to drive overall internal quality improvement. The best way to achieve this is to aim towards ‘holistic’ strategic initiatives that are representative of the entire student experience. By taking a ‘holistic’ approach the executive at the institution can gain the most transparency and evaluate the outcomes strategically to make the best decisions.
6 – Survey administration should be a second job, not a day job.
It makes sense to look at the way course evaluations are run in higher and further education organisations as this process represents the best way to gain a better understanding of the perceptions and attitudes of the student. Many organisations have outdated technologies and significant efficiency gains can be made by deploying available ‘best in class’ technology An inordinate amount of energy is being wasted because of the challenges of paper based survey administration and reporting on course evaluations. This energy is better spent on driving quality to implement change and key strategic initiatives. Technology can be implemented to minimise the manual constraints and challenges surrounding both paper and online course evaluations. The decision criteria need not be centred on paper vs. Online rather on implementing technology that allows for the best use of either methodology.
Electric Paper works with universities to capture student feedback that will help to improve the future design and delivery of their courses. We work with more than 600 education institutions worldwide through web-based data capture and student survey management solutions to drive efficiency in capturing the student voice at course level. Our flagship product EvaSys Education Suite™ works to automate course and module evaluation and reporting, thereby saving staff time and costs, as well as generating cross-institution common best practice in assessing student feedback about their teaching and learning experience. In light of the need to place greater emphasis on the “student experience” in UK higher education, our role is increasingly significant. We also deliver surveys of alumni, employees and other members of the education community. For more information: www.electricpaper.co.uk