Developing Surveys

Asking the Right Questions the Right Way

Keep your survey to a reasonable length.

If you want to know someone's view on a particular topic, you go and ask them, right? Getting their answer directly is much better than second-guessing what their response might be.

The challenge comes when you want to gauge the opinions of several groups of people. For example, you may need to assess the mood of your team or department in response to a new initiative you've introduced. Or you may want to test out an idea with your customer base before launching a new product or service.

Whether it's corporate culture issues, or customer satisfaction research, the best source of feedback and suggestions is usually the primary stakeholders themselves. We can try to guess what customers want, or what motivates workers – or we can simply ask them directly.

That's where surveys come in. Surveys give you the chance to ask a target group of people a set of questions about a particular subject, project, or issue. You can then use their views to tailor your strategy, or review your approach.

However, do bear in mind that effective surveys involve much more than writing down questions, and asking people to respond.

What, therefore, do you need to know about surveys before you implement one? And how do you maximize the value you gain from them?

This article outlines the strategies and techniques that will help your survey deliver the desired outcome.

Planning the Survey

If you simply sit down and start writing questions, you may risk missing critical elements of the data you want to capture. Take time to plan your approach, and decide what you want to achieve.

Consider the following when planning your survey:

Determine your survey goals – What do you want to learn from your respondents? The clearer you are about what you need to know, the more focused your questions and answers will be. Do you want satisfaction ratings, or people's opinions? Are you measuring attitudes, or predicting future needs?
Identify your target population – Whom do you want to survey? Respondents could be past customers, potential customers, job applicants, workers in a particular geographic location, and so on. Before you define your target audience, assess how their answers will help you achieve your objectives. Also, make sure that you have access to your desired respondents. Are you able to contact the right people to ask your questions?

Be aware of the potential bias associated with your target population. For example, if you survey only former workers, or people who have contacted your customer service department, you're likely to receive a higher proportion of negative answers. This is because former staff may have left the company because they were unhappy, and people tend to contact customer service when they have a complaint.

Decide your sample size – How many responses are enough to give you the information and insight you require? Larger samples usually allow you to reach more reliable conclusions from the audience's responses. However, you don't want to survey too many people. Ideally, you need a balance between the number of responses, and your ability to process and analyze them.
Choose your survey method – How will you deliver the survey? The time and budget you've got available will often determine how you carry out the research. Here are some options, and the common advantages and disadvantages of each:
- Person to person – In one-on-one interviews, it's easy to change or clarify your questions as needed. However, these interviews take a lot of staff time, so you have to decide if the investment is worth it. Some companies use telephone interviews to reduce costs, but this can also be expensive.
- Mail – This can be cost effective, and relatively easy to organize. This method also allows respondents to complete the surveys at their own convenience. However, there is a tendency for people to ignore this kind of survey, and there's no opportunity for you to explain the questions. However, there are ways to improve response rates, which we discuss below.
- Internet – Email and online surveys are growing in popularity, partly because they're inexpensive to deliver. Email surveys can be limited in the types of questions you can ask – but you can set up surveys on websites so that they're interactive, and you can even program them to react in real time to the responses given. However, there are disadvantages to Internet surveys. Respondents must have access to a computer and the Internet. And, you need to make sure that only the target audience has access to the survey, and that respondents answer only once. However, there are various survey tools available – such as Surveymonkey, PollDaddy, and Google Forms – that aim to make the whole process easier.

Your survey method may create biases. As we said, online surveys limit your respondents to computer users with Internet access. One-on-one interviews can be affected by the time of day – for example, if you're surveying office staff during work hours, they may be pressed for time. And interviewing people in one location may reach only one socioeconomic group, or have another limiting factor. Consider any potential biases when choosing your method.

Determine your survey length – How many questions will you ask? The key here is to be very focused on your goal. If you allow multiple stakeholders to contribute, you'll quickly end up with a list of questions that's far too long. It's important to be ruthless in challenging the value of each question, and to avoid including the vague 'nice to knows,' or the ones that do not contribute to your goal. Bear in mind that the survey length may influence the quality and quantity of responses you receive. When a survey is too long, people may get bored – and either fail to complete it, or answer too quickly to save time. If the survey takes more than five minutes to complete, people may not bother doing it at all. If the survey is too short, you may not get enough information to meet your research goals. Also, consider asking the same question in different ways to evaluate the reliability and integrity of the responses. KISS (Keep it Simple, Stupid) is a good guideline when determining your survey length.
Choose the types of questions – Will you use true/false, multiple choice, ranking, or open answers? The more customized your responses, the more work it takes to interpret the results. Simple multiple-choice or limited-choice questions can be scanned into a computer, and analyzed quickly. However, you may lose some of the detail by taking this approach, so consider the pros and cons of each option.

It's important to consider these basic planning elements, and be clear about your objectives and basic strategy, before you start writing the survey questions.

Developing the Questions

You can use two basic types of questions in a survey:

Multiple choice – Respondents choose from a series of answers that you provide. This category includes ranking and true/false questions.
Open ended – Respondents answer with a number, or with a written or verbal response.

Here are some useful guidelines for writing questions:

Allow for every alternative – To find out what kind of green vegetables a person regularly eats, it would be difficult to list every possibility. Instead, you could list the more common vegetables, and include a space for 'Other.' Be careful to allow respondents to include more than one choice. If people eat broccoli, spinach, and green beans, for example, don't force them to choose just one.
Allow for Other, None, Don't Know, and Not Applicable responses – Don't force a choice when it may not be relevant. This is also an opportunity to list only the choices most relevant to your objective, and then analyze the remaining 'Other' responses separately.
Ensure that options are mutually exclusive – If questions force a choice, make sure each option is clear and different. For example, when asking if workers are full time, part time, or seasonal, remember that seasonal workers can be full time or part time.
Make answer choices specific – You ideally want the most accurate response possible to your survey. For instance, if you ask people if they've ever listened to country and western music, they may answer 'yes' – even if they heard only one country song 20 years ago. It's therefore vital to be specific about what you want to know.
Ask one question at a time – Avoid questions that deal with more than one issue. For example, consider 'Do you think people should be allowed to vote early, and have you ever done so?' If people answer 'no,' does that mean they disagree with early voting, or they've never voted early?
Limit your choices – Too many options may overwhelm or frustrate respondents, and they may not read the whole list before making a choice.
Use an appropriate Likert scale (ordinal scale) – Your response scale should match the way your choices are worded. A Likert scale asks respondents to indicate how much they agree with a statement. Consider, for example, 'I often seek other opinions before making a decision.' It doesn't make sense to offer the choices Very Often, Fairly Often, Sometimes, Almost Never, or Never – because the statement already includes the word 'often.' More appropriate choices would be Strongly Agree, Agree, Disagree, Strongly Disagree. An alternative way of doing this is to keep the original responses, but change the statement to 'I seek other opinions before making a decision.'

Here are other common Likert scale choices:

Excellent, Very Good, Fair, Poor.
Completely Satisfied, Very Satisfied, Somewhat Satisfied, Somewhat Dissatisfied, Very Dissatisfied, Completely Dissatisfied.
Definitely True, True, Don’t Know, False, Definitely False.
None, Very Mild, Mild, Moderate, Severe.

Don't 'lead' the respondent to answer a certain way – Avoid wording that assumes you'll receive a particular response – and don't make your preferred answer obvious. Consider the question 'Isn't it best for animals to live in the wild instead of a zoo?' This will probably lead people to answer 'yes.' A better question would be 'Where is the best place for animals to live?'
Be aware of cultural issues – People in some cultures are more likely to be agreeable, or aim to create a positive image through the answers they give. Some cultures teach people to give the answer they think is 'correct.' Also, remember to take into account the issue of anonymity. You may want to reassure all respondents that their individual answers will be kept completely confidential.
Ask clear questions – Above all, questions must be clearly understood. Be aware of these potential problems:
- Avoid words that may cause certain emotions. For example, words like patriotic, communist, victim, feminist, and insurgent may cause bias in the responses.
- Avoid business jargon, acronyms, and technical terms.
- Avoid slang and idioms.
- Avoid double negatives, such as 'Should we not appeal the decision not to introduce a dress code?'
- Avoid ambiguity and vagueness, such as 'We should try to limit access to cigarettes most of the time.'

Designing the Survey Elements

Formatting and laying out the survey is important. Consider these tips to maximize the survey's effectiveness:

Give your survey a title.
Include a short introduction.
Include information about you and your objectives, where possible.
Guarantee anonymity and confidentiality, if possible – or describe how results will be shared, and ask for agreement.
Provide detailed instructions on how to return the survey. Supply pre-addressed, postage-paid envelopes for mailed surveys. Provide contact details where appropriate.
Ask for demographic information (age, education level, income level, and so on) at the end of the survey.
For written responses, provide enough space to answer open-ended questions.
Allow respondents to comment on the survey at the end. Provide space to write responses, or make sure interviewers ask for feedback.
Keep lots of white space (areas with no printed text).
DON'T USE ALL CAPITAL LETTERS – THIS IS HARD TO READ.
Ask pleasant and easy-to-answer questions at the beginning. Save the more sensitive or difficult questions for later in the survey.
For agree/disagree scales, offer the positive answers first.
Mix up the response choices. For example, make sure a 'disagree,' or other negative response to some questions, actually indicates a positive behavior or perspective. This keeps respondents from getting into the habit of always giving an 'agree' answer.

Improving Response Rates

The best-designed survey in the world may not generate the number of responses you need. This is because people who receive surveys often don't care enough to complete them. Some of us receive too many surveys – letters in the mail, phone calls during dinner, and pop-up screens on our computer when we're working. The smart way to carry out a survey involves a variety of methods to persuade people to respond.

Here are three ways that can help you improve your return rates:

Design the survey well – Follow the design tips above to improve your response rates dramatically. People are more likely to answer surveys that are well organized, and easy to understand.
Keep the survey length reasonable – Don't make your survey too long. Few people want to answer page after page of questions.
Create a perceived value to the respondent – One of the best ways to increase responses is to prove that the information provided will benefit the respondent. For example, if you ask for their input, tell them how their suggestions will be used to impact their work. When you link responses to a positive outcome, you help provide clear motivation for them to respond.
The other way to add value is to provide a tangible incentive or reward. Even a small token of appreciation may help increase your response rate. And incentives also tend to improve the value of the responses – you typically get fewer 'Other' and 'Don't Know' answers.
Finding an incentive that will appeal to your target audience is key to your success. Here are some of the more common ones:
- Tokens – You might include a small, inexpensive item with the survey, such as a pen or notepad. These are known as pre-survey incentives.
- Coupons – You could provide coupons with the survey package, or mail them out after a completed survey is received. Typically, coupons are for the next purchase from your company. However, you can send coupons for anything you think the respondent might want.
- Sweepstakes – Enter all completed surveys into a draw for a major prize. You can award trips, free merchandise, or even cash.
- Other post-survey tangibles – These may include tickets to shows, dinner certificates, spa certificates, and so on.
- Refunds or rebates – For customer satisfaction surveys, you may offer the chance to win back the purchase price of the product the customer has just bought.

Remember to issue reminders after sending out the initial survey. A postcard that arrives a week after the survey may be enough to motivate someone to complete it. Reminders may not be as effective as rewards, but they can significantly improve response rates.

Key Points

Surveys are a great way to discover what people think. If you need to know something, asking is more efficient than guessing – but you must ask the right questions in the right way. To develop an effective survey, you should be aware of many factors that affect your design and questions. Have a clear objective, then determine your target respondents, and make decisions about design and content. After your survey is planned, you're ready to develop the questions. This can be an 'art' – and there are many things to consider, including the types of questions, and the exact wording.

A well-designed survey will help improve response rates. But, to maximize responses, you could consider offering incentives. These can range from small tokens to prize sweepstakes. You'll probably invest a lot of time and effort in your survey, so plan carefully to get the most from your responses.

This site teaches you the skills you need for a happy and successful career; and this is just one of many tools and resources that you'll find here at Mind Tools. Click here for more, subscribe to our free newsletter, or become a member for just $1.

Add this article to My Learning Plan