When asked directly, customers often hedge the truth of their opinions. Maybe
they do not want to insult you. Or they've been brought up to accentuate the
positive. Or they talk your ear off. However, in a focus group with a good
facilitator the same customers often will talk in detail about how to improve
your product or services.
Online surveys can replicate this focus group effect. An online survey can
provide a safe place for your customers to share their responses, opinions, and
ideas, especially if they complete the survey in a comfortable place such as
their office or at home. The Internet provides anonymity and ease of use to help
customers speak freely.
Setting up an effective online survey does not have to be expensive or
complicated. In fact, if your goals are modest, it can cost you only your time.
Many businesses do not need the elaborate precision required for formal surveys.
It is enough to ask a chosen group of customers what they think about a product
or service. This article describes how to set up an online survey that will meet
the basic needs of most businesses.
The Survey Process
To create and run a survey, you will need to work through these steps:
- Define your goals
- Pick your tools
- Write your questions
- Test your survey
- Deploy your survey
- Measure your results
The rest of this article will describe these steps in greater detail.
Define Your Goals
When you sit down to plan your survey, your first task is to determine
whether or not you need help. If you need to ask hundreds or thousands of
customers about their preferences and you want accuracy within 5% or less, then
you should spend money to hire a good research firm. There is enough complexity
and skill involved in running this sort of survey that experts are needed. This
article will help educate you about the process but it is not a substitute.
However, if you sense that customers are responding poorly to some part of
your business, for example, customer service, and your goal is to get your
customers to vent in useful ways, then running your own survey probably will
work fine. The question is what level of precision you need from your results.
In many cases, extreme precision is not required. It is enough for you to offer
your customers the opportunity to speak out about existing or new products
Once you have an idea about whether to do your survey yourself or hire
experts, the next issue to address is what you want to achieve with your survey.
If you don't know, start by writing down what you would like customers to tell
you about your business, products, and services. Then prioritize that list. On
another sheet of paper, write down a list of questions you have about your
business and prioritize your list. If you have time, you also should write up a
list of customer ideas and complaints that you have heard in the past year. Put
these lists side by side and look for commonalities in subject and priority.
This list writing process often will pull out ideas and issues that lie just
under the surface. For example, a customer comment might give you an idea about
how to improve a product. If that product is extremely important to your
business, a survey might help you understand how to improve it. (So would paper
mockups as part of usability testing but that is another article.) Or you might
decide to use periodic surveys as a way to add a customer feedback channel and
so your list might be a series of small topics. Or you might realize that a
competitor product is your biggest worry and you wonder if your customers care
about their features (and which features, and with what priority).
Pick Your Tools
When you have picked one or two goals for your survey, the next step is to
consider tools. You either can host the software yourself or use a vendor. My
recommendation is to start with a vendor. This will reduce the complexity of
your first survey and help you to focus on your goals, the quality of your
questions, and other issues more important than technology.
There is at least one robust online survey vendor: eSurveysPro.com.
There are a number of services that offer polling but this
vendor offer a broad range of question types, data output, and ease of use for
small amounts of money. eSurveysPro is free.
The criteria for choosing a tool or vendor should depend on your goals. While
it is great to have a wide range of question types available for use, for
example, too many types of questions will overwhelm your customers. So a broad
range of question types does not mean you should use every one. It simply allows
you to pick the types you'll need when you set up your survey.
In addition to checking for a broad set of question types, you should ensure
that you can control your data by downloading it at any time day or night. You
should ask the vendor how they secure their data so that your questions and
results are not shared with others. Ideally your data should be stored on a
server separate from the survey application and it should be isolated through
firewalls and other techniques. The vendor also should have a daily backup for
your data and the ability to roll back to a previous version if needed.
You also might ask if your survey vendor software allows branching,
automatically leading your customer to the next question if they answer a
certain way on one question. If the customer answers "yes," for example, they
would be jumped down to the next question and not confronted with questions
related to answering "no." That said, while branching is useful and cool, it can
be a sign of poorly worded questions, as described below. Needless branching can
confuse your customers and waste their precious time.
Other criteria to consider with tools is the ability to define easily the
text that appears before, during, and after your survey. You also want to
control where your customers go when they complete your survey. Ideally, you
want to send them to a thank you page on your website rather than dump them at
the home page of the survey vendor.
Write Your Questions
How to write survey questions can be more complicated than you might think,
but not impossible to work through. You will need to consider at least three
Types of questions
Wording of questions
- Order of questions
Questions, for example, can be open ended or closed. You either ask your
customers to write a response (open ended) or select from a defined set of
possible answers (closed). Ideally, most of your questions should be closed
because the survey will go more quickly with less chance that a customer will
become tied down answering one question then abandon your survey. Closed
questions also yield more uniform responses when you measure your results.
There are also at least these types of questions to consider when you write
- Multiple choice questions offer a limited range of possible answers, either
one or multiple responses per question.
- Matrix questions that allow the customer to rate a range of products or
services by a single set of possible responses.
- Open ended questions that allow the customer to type in their response.
These three types of questions have numerous permutations. eSurveysPro,
for example, offers a dozen different question types based on these three basic
types. For example, you can have a multiple choice question that lets the
customer choose from a dropdown list, a vertical (up/down) list of choices, or
horizontal (left/right) choices. This may seem cosmetic but dropdown lists might
retard participation because the customer may not see the question. And a
horizontal list of choices might make your survey appear shorter than if the
same choices are laid out vertically across the page and under the question.
How you word survey questions is another important aspect to writing survey
questions. While common sense rules apply (short sentences with familiar words
are better than convoluted sentences with unusual words), here are some
guidelines to follow as you write survey questions:
Avoid double negatives, abbreviations, acronyms, and relax your grammar as
- Avoid leading questions, be neutral in how you phrase your questions.
- Look for questions that should be broken into two or more parts. If you have
the conjunction "and" in a question, chances are you can break the question into
two or more questions. Long questions are another candidate for breaking into
smaller, more pointed questions.
- Use consistent rating scales, for example, use 1-5 for bad to good on all
questions with rating scales instead of 1-4 for one question, 1-10 for another
question, and 1-6 for a third question.
- When a question begins with or uses "if," consider rewriting to avoid
branching (having customers skip down some number of questions to get to the
next relevant question). If branching cannot be avoided, be sure to use text
(headlines, questions, and descriptions) to clearly lead your customer to the
next point in the survey.
- Where possible, provide multiple choice responses instead of asking
customers to distribute their ratings across multiple categories. Asking "which
of these choices best describe your response" is more efficient and less
frightening to deal with than "tell us which of these choices is most important
to you in terms of percentages (must equal 100%)."
- For multiple choice responses, prune your list to 2-5 choices in most cases
and responses to no more than a half dozen words where possible.
- Look for questions that require recall and rewrite them to minimize how much
your customers have to recall since memories are often difficult to recall,
especially if people are in a hurry while completing your survey.
- Consider use of the third person for challenging and threatening questions.
For example, ask "How do your friends respond to losing this benefit?" instead
of "How do you respond to losing this benefit?" As with any rule, it depends
upon the expected response of your customers and the overall flow of your
- Use consistent phrase structures in your questions and answers to improve
the overall flow and tie together different elements of your survey. For
example, if appropriate, begin all responses with verb phrases instead of
alternating randomly between starting responses (or questions) with verb, noun,
adjective, and adverb phrases.
- Your last edit with each question should be to pose and answer two
questions: What will I learn from this question? Will what I learn be useful to
Here's one example that shows how to edit and polish your questions. Say
you want to know which day of the week works best to deliver product or services
to your customer. You could ask, "Which days work best for you to receive
Product X?" However, this is a leading question: it presumes your customer wants
Product X. It would be better to break this one question into two questions, at
least. Do you want to receive Product X? and If you want to receive Product X,
which days work best for you? Then you might add a third question, "If you do
not want to receive Product X, please give your reasons" with multiple choice
responses and/or an open ended question they can use to type a unique
Finally, in writing your survey questions, consider how you order the
questions in your survey. Placing your most difficult questions up top risks
turning off your customers which leads to them abandoning your survey. Here are
some guidelines to follow as you order your survey questions:
- Lead with interesting questions that entice your customers to respond.
- Demographic and other easy to comprehend questions are a scarce resource
that can help lead your customer into your survey. They also can be used to pace
the survey if placed between more difficult questions, as a way to help your
customer through your survey.
- Use logical order. For example, if questions are related, be sure they're in
an order that makes sense to your customer. Move from the general to the
specific. Be aware if answering one question make a customer likely to respond
differently when they arrive at another question later in the survey; in that
case, the questions probably should be grouped together and ordered to make best
- Look for dead ends and transitions and make sure they're smoothed over with
text (headlines, questions, and descriptions) that bridges these sections of
Test Your Survey
Once you have written your survey questions, the next step is to test your
survey with a small group of customers. The size of this group should be at
least 5-8 customers; in usability testing, often this number of people are
enough to identify the key issues. While you can test your survey online, it
probably will work best in person with a printout of the survey. Your goal is to
understand how your customers are likely to respond to each question, the inner
dialog that will be triggered by reading each of your questions. This dialog
is impossible to capture online. Instead, you need to be able to ask your test
audience how they responded to each question, what words, phrases, and sentences
confused them, what worked and what failed.
Testing also is critical because you should never change your survey once it
is put up and advertised. Changing questions in mid-survey will distort your
results, creating two surveys. Your goal is to ensure that the results you get
the first time are adequate for your needs.
Deploy Your Survey
Posting your questions to eSurveysPro and building your survey
is a fairly straightforward process. You set up a survey and define your
parameters (for example, where does the customer go when they complete the
survey), then build your questions one by one. You should take the survey at
least once to ensure it works as you need.
The next step is to announce your survey to the audience you want to have
participate and ensure maximum participation. The best way to ensure you get the
minimum responses you need is to advertise to more people than you need. If you
need 100 responses, try to advertise to 200 of your customers. This is not a
precise science so you will need to track what percent of customers notified
I would recommend that, whenever possible, you invite customers in person to
participate, as part of an ongoing conversation. Call up the customers you need,
explain the benefit to them and to you, ask if they need anything to help them
participate, then email them the URL for your online survey.
You also need to be aware of other conflicts that limit participation: date
and time of the week, the environment where the test is to be taken, holidays,
and similar constraints. For example, if your customers are businesses, sending
notice of your survey on Monday morning is probably less effective than Tuesday
afternoon or Wednesday late morning, times when your customers are less likely
to be dealing with weekend backlog and/or the day's first mail and emails.
If you put your survey up for an extended period of time, you also should
send follow up emails to remind customers who have not responded. This can be in
the form of email, mentions during sales and phone calls, publishing the survey
URL in fliers or other printed materials, and other means.
Measure Your Results
Because the survey process described in this article is not statistically
rigorous, you will want to look for certain details in your survey responses.
Instead of precise quantifiable results within a margin of error, you want to
find strong responses that indicate a solid set of your customers respond in a
particular way. You also want to find the unexpected responses, for example,
customer comments that raise issues you had not considered.
If you have designed, written, and tested your survey well, you will find
that your results are often very clear in their message. You may not have the
confidence gained if you surveyed thousands of customers who had been
scientifically selected to minimize bias. But you will know more than you might
Final Thoughts: To Pay or Not To Pay For Participation?
While online surveys can help your customers to provide their responses to
your business products and services, they also can be useful tools to build
customer confidence in your business. People appreciate being asked for their
ideas if participation is not too difficult. That's why I recommend that you
view an online survey as a structured dialog with your customers. Where
possible be sure to work with customers in person as you design and test your
survey. Be sure to thank them afterwards for their participation.
Which leads to a final thought about compensation. Many surveys pay people
who respond. Whether your pay or not should depend on your budget. But there are
other ways to pay, with discounts, for example. Even providing tangible
responses that show you listened to a customer also count as compensation:
you've made their life easier. Don't assume that money is the only compensation.
Payment should be handled in context of your overall conversations with