Internet surveys
January 2003
Is it time you changed your survey publishing method? In recent years Internet surveys have been steadily growing in popularity. In this article we consider when an Internet survey is appropriate and we hear from a number of Snap users about their experiences with Internet surveys. We look at some of the benefits Internet surveys offer to respondents and to those conducting surveys, as well as the restrictions Internet surveys can impose on data collection. Finally we consider which types of survey are most commonly conducted using the Internet and why.
How appropriate is an Internet survey?
Will an Internet survey be the most appropriate publishing method for your survey? Internet surveys lend themselves to certain types of survey because respondents know how to use e-mail or the web, have the technology to reply, give their permission to participate and have an interest in completing the survey.
Not everyone has the skills to use and has access to the Internet. The world population is 6.2 billion. The number of Internet users is estimated as 580.78 million (Nua study May 2002), approximately 10% of the world population. In some countries web usage is inevitably higher. Internet usage in Europe stands at 185.8 million, 61% of the total population of 302.6 million. In the US (population of 280.5 million) and Canada (population of 31.9 million) there are 182.8 million users, 59% of the population of the two countries.
John Cooper at Bath Spa University College has been using internal e-mail surveys to assess students ICT (Information and Communications Technology) skills and their need for further training for the past two years. Using the internal e-mail system and sending a copy to every student (all students in 2001, first years only in 2002) he has achieved a return rate of 23% in 2001, but only 17% in 2002. This lower return was due in part to an extended failure of the student e-mail system about six weeks into the survey. Even so an estimated 35% of students on the Postgraduate Certificate of Education (Secondary) course responded.
Analysis of both surveys shows that 90% of the returns arrived within 30 days of the start date. A single reminder attracted another 4% of the final return in 2001.
He is currently investigating the influence of ICT skills on the return rate, using data from this survey and from a parallel one using an identical paper-based questionnaire. As part of this research he is examining the use by students of multiple e-mail addresses (Hotmail, Yahoo mail, BSUC mail etc).
Even if your target respondents have the right skills and technology will they want you to send them your survey? Permission is not always a given. There is, for instance, still a great deal of uncertainty about how well unsolicited promotional e-mails will be received. The same can be said for unsolicited Internet surveys. To some people receiving an unexpected e-mail is reasonable, to others it is an outrage. The only way of ensuring that survey respondents are not outraged when they receive your Internet survey is to ask permission before you send it.
Equally an Internet survey - like all surveys - should be sent to an appropriate audience. As you are targeting the respondent, the onus is on you to find out whether there is a likelihood of him or her finding your survey relevant and worthy of completion. Will completing the survey be a valuable use of their time? Will they be able to voice their views on a subject that is in their interests, such as a service improvement, or if not, is there a chance to win a worthwhile prize? Is now an appropriate time to be asking them to complete your survey?
The benefits of Internet surveys
There are a number of real benefits to Internet surveys, which make them attractive to both the respondent and the person conducting the survey. Internet surveys are fun to complete because responses can be interactive. Perhaps most importantly data entry is automatic, however Internet surveys do have their limitations.
Benefits and limitations for respondents
Internet surveys appeal to respondents, as not only do they look impressive, they can be set up to be interactive.
Snap provides 'routing'. For example, a respondent replies to Question 1 that he does not travel to work by bus, so the survey routes around (skips over) Question 2, which asks how much the bus fare costs, and goes straight on to the next appropriate question. The respondent may not know it, but this has saved him time, as he has not had to read a question, that is simply not relevant to him. The survey is interacting with his responses.
Another feature of Snap is 'dynamic text substitution'. This means that key words can be taken from one question and dropped into another. For instance, say a respondent has replied to our Question 1, that he does travel to work by bus, Question 3 can then be automatically modified to ask 'How long does your journey to work take by bus?" Again the question is made to be relevant and the respondent gains the impression that his replies are being noted and incorporated into the survey in an intelligent way, just how a human interviewer would conduct a survey.
Validation in Internet surveys is a useful way of ensuring that only accurate data are received. For example, Snap can be set up to allow 'Yes' or 'No' responses and the respondent cannot key in an alternative. The survey cannot be submitted until these mandatory fields are completed.
Leslie Henry, Research Director at Book Marketing Limited says that validation can also be a limitation. "One of the design faults with many Internet surveys is that they do not allow people to answer outside the pre-specified parameters, and they force you to give an answer which isn't really correct or else you cannot proceed. This is one of the downsides of the validation process." Similarly Leslie says, "There are severe problems in obtaining qualitative data. We find that we get longer, more explicit, answers to open-ended questions on a postal study than we do with online studies.
On the plus side, using a 'cookie', your Internet survey can block multiple responses from individuals to prevent results from being skewed. Furthermore respondents can complete the survey at a convenient time. This could even be in the middle of the night. They are not restricted to filling it in the moment they receive it, giving them time to give the survey some thought.
Benefits to you
You can expect a much faster survey process as an Internet survey can be completed quickly and the respondent keys in the data, which is fed straight back to you for immediate analysis. You can expect lower costs, thanks to the absence of printing and postage, and the keying in.
However it is likely that survey data will be affected by the fact that your survey is published on the Internet.
Leslie Henry says, "When we have conducted surveys using both Internet and other methods - where all respondents have access to the Internet - and compared responses, we have, of course, discovered differences in the profiles of respondents - in terms of their comfort in using computers, accessing the Internet, and answering online. This has fed through to differences in responses to other questions, seemingly unrelated to use of the Internet."
Internet surveys also make it easy to give feedback. Results can be e-mailed to respondents, as their e-mail addresses have already been noted, or they can be published on a website as a news item, to attract visitors.
Most common types of Internet surveys
As mentioned, having Internet skills, and access to the Internet plays a key role in whether or not Internet surveys can be deployed. When these are in place it is a question of permission and then relevance of the survey to the respondent. Internet surveys are therefore most commonly used for internal staff surveys, surveys of websites and surveys of customers in remote locations.
The internal survey, for example a staff satisfaction survey, is increasingly becoming Internet based. The company HR department knows whether or not personnel have access to Internet technology, and can legitimately contact personnel by e-mail, although permission and buy-in may be needed from superiors to proceed with the survey. The survey is most likely to be well-received assuming that it offers employees an opportunity to address issues in which they have an interest, or it may be that a directive is issued obliging all employees to complete the survey.
At Arun District Council Internet surveys have proven themselves the best medium for staff surveys. Rachel Jupp, Assistant Policy and Research Officer says, "The main benefit of Internet surveys is time. As there is no need for data inputting, results are instantaneous. Mainly, we use Internet surveys for staff panels as most of our staff have Internet access. It's very successful for us and response rates are high."
Similarly, Internet surveys are commonly used to survey websites. Clearly respondents have the means to reply to the survey as they will have either been on the website when they saw a pop-up survey, or they actively chose to click on a link to the survey. Completing the survey is strictly voluntary, so permission is not an issue, and there is a strong likelihood that respondents will have an opinion on the design of the website. However unless they feel strongly about the site or need to use it for their work, respondents may need an incentive to reply.
Finally Internet surveys are useful where it is critical to ask opinions, but respondents are located remotely making face-to-face interviews impossible due to cost and time constraints, possibly on both sides.
At Apollo Fire Detectors in Hampshire, Internet surveys replace personal interviews with customers overseas and in the UK. Rowland Davies, Marketing Manager at Apollo, says, "We have to design products that are sold in over 75 countries overseas. Since our products are lifesaving products, i.e., fire detectors, they are subject to testing and approval under different standards in different countries. Local culture also affects design features. It is very important to understand these issues before developing new products. Personal interviews are simply not possible for many of our customers (even in the UK!) but an Internet questionnaire, which we put in a hidden area of our website and invite customers to respond to in personal e-mails, is the ideal solution. Response is good and the importing of responses into the survey data could not be simpler. This is now an established methodology in our company."
Summary
Internet surveys must be targeted at appropriate audiences, who have ICT skills and access to the technology to participate, who give permission and are willing to reply because doing so is in their interests. If these criteria are not met, then other methods of survey publication and data collection should be considered --such as paper, face-to-face interviewing, or telephone. Internet surveys are ideal for internal surveys, website surveys and reaching respondents in remote locations. They have their limitations, but they do bring real benefits in particular in terms of reduced time and cost