Surveys can be a great way to collect a large amount of data very quickly, particularly if you have mailing lists or customer databases at your disposal. They’re also a great way to recruit for telephone interviews for deeper insight.
The down side is that if your survey isn’t asking the right questions you might be making key decisions that affect people’s lives and businesses based on bad data. So it’s vital that your survey is delivering the right insight, which is harder than it sounds. As Erica Hall put it bad surveys have no smell. So to help give your survey the best chance of success I’ve put together some simple tips.
1. Ask yourself if you actually need a survey
We often use surveys because they’re easy. They can take just a few hours to create and once they’ve been sent out you just sit back while the data comes flooding in. However just because they’re easy doesn’t mean they’re the right thing to use.
A survey is useful if you need qualitative data, or are looking for general opinions, but they’re not a substitute for user interviews, usability testing or other qualitative methods. A survey can be a good place to start your research, but it only shines a light in the directions that you should be exploring further.
2. Work backwards from your goals
Like all good research methods you need to start by defining the questions that you want your survey to begin to answer. It’s no good just writing “what’s your opinion of x”, “how often do you y” if the answers don’t help you reach your overall goals. Start by jotting down three to five questions and then create your survey around extracting the answers to them. When you’ve finished go back and check every question in your survey and make sure that it relates to one of your goal questions before you send it out. If it doesn’t then ask yourself whether you really need to include it.
3. Use the Likert scale
If you’ve written a survey before the chances are you used a scale of some kind. Scales are great as they allow us to measure degrees of opinion instead of simple yes/no answers. So if your question is measuring things like agreement, frequency, importance or likelihood then you should be using the Likert scale. This PDF has some great examples.
4. Mind your metrics
When writing questions it’s crucial that you consider the outputs. “Think to yourself what am I measuring?” and stick to it. I’ve seen questions that ask people to rate their happiness with a product, but the question includes other options like I haven’t used it and I don’t find it valuable. While it may seem helpful to give extra options to cover all your bases, it just makes it more difficult to make sense of the data that you’ll get back. In this example we’re measuring happiness but then shoehorning in frequency of use and value too, and without a scale.
5. Add a Speed Check
A good way to get people to fill in your survey is to offer an incentive. We commonly give away vouchers or run a draw for an iPad to help boost responses, but the danger with this is that you’ll get freebie hunters clicking through the survey, polluting it with rubbish data. I’ve seen my surveys listed on HotUKDeals and had to take them offline, which can be a real pain. The best way to deal with this is to add a speed check question, a bit like a CAPTCHA, which helps you tell whether someone is paying attention. Jeff Sauro wrote a wonderful blog post How Many People Cheat In Online Surveys? which I highly recommend.
6. Be careful when asking personal questions
Sometimes asking for gender and age group can be a great way to segment data, and asking a few easy questions at the start of a survey is a great way to warm up. However be careful, as asking for too much personal data can turn people off and harm response rates. It’s a good idea to include a ‘prefer not to say’ option when asking personal questions so that people can still fill in the rest of the survey even if they’re not comfortable answering personal questions.
7. The shorter the better
People don’t always have a lot of time to spend filling in your survey, so the shorter you can make it the better the response rate you’re likely to see. As a general rule try to keep it to around 10 questions or less. It’s okay to go above 10 if you need to, but make sure that your questions are directly related to your goals. If they’re not, cut them.