Calculating the response rate for an online survey is an important but sometimes difficult process. Here are some tips to help you calculate an accurate response rate.
For surveys in which potential respondents are invited by email, the traditional practice of calculating response rate as the number of completed interviews divided by the number of invitations generally understates the response rate because it assumes that all invitation were received and opened. Because of the prevalence of spam and spam filters, and the simple fact that many people are inundated with email messages, a more accurate way of measuring response rate is needed.
A truer measure of response rate is calculated as the number of completed interviews divided by a) the number of invitations sent minus b) the number of bounce backs. However, even this calculation can understate the response rate because:
- Not all failed emails generate a bounce back
- It still assumes that all emails that were received were seen by the recipients
Although it is possible to count the number of email invitations that were seen (not just received, but opened), the process by which this is done can be detrimental to the study.
The option of requesting a return receipt is included in many email clients, but is not a good option because it is so intrusive. Generally, when a sender requests a return receipt the recipient’s email client posts a message such as: “This user has requested a return receipt. Allow? Yes/No”. If the recipient clicks “No” (and many do as a standard response), that is the end of that.
Using an Image
This method is more effective but a lot sneakier. In this process, the survey invitation is formatted in HTML and includes a tiny image tag (often only one pixel) such as:
As soon as the email is opened or displayed in the preview pane, the website receives a page request and extracts the email address from the querystring to confirm that the email sent to that address was opened.
Sounds great, right? Well, maybe not. One problem is that not all recipients receive email in HTML format (many use text only). Another problem is that many email clients now default to not automatically downloading images. But the biggest problem is that this method seems very underhanded. If potential recipients discover that you are doing it, they are likely to feel you are trying to deceive them. As a result, they are less inclined to complete the survey or provide frank responses.
The bottom line: the best practice for calculating the response rate is to divide the number of completed interviews by the number of invitations sent minus the number of bounce backs. Although it is possible to achieve a more accurate calculation, the risks of doing so generally outweigh the benefits.