Writing great questions is an art that like all arts requires a
great amount of work, practice, and help from others. The following discussion
is one that identifies some of the common pitfalls in creating a great
questionnaire.
Avoid loaded or leading words or questions
Slight
wording changes can produce great differences in results. Could, Should, Might
all sound almost the same, but may produce a 20% difference in agreement to a
question (The supreme court could.. should.. might.. have forced the breakup of
Microsoft Corporation). Strong words that represent control or action, such as
prohibit produces similar results (Do you believe that congress should prohibit
insurance companies from raising rates?) Sometimes wording is just biased: You
wouldn't want to go to Rudolpho's Restaurant for the company's annual party
would you?
Misplaced questions
Questions placed out of order or out
of context should be avoided. In general, a funnel approach is advised. Broad
and general questions at the beginning of the questionnaire as a warm-up. Then
more specific questions, followed by more general easy to answer questions like
demographics.
Mutually non-exclusive response categories
Multiple
choice response categories should be mutually exclusive so that clear choices
can be made. Non-exclusive answers frustrate the respondent and make
interpretation difficult at best.
Nonspecific questions
Do you like orange juice? This is
very unclear...do I like what? Taste, texture, nutritional content, Vitamin C,
the current price, concentrate, fresh squeezed? Be specific in what you want to
know about. Do you watch TV regularly? (what is regularly?).
Confusing or unfamiliar words
Asking about caloric
content, bits, bytes, mbs, and other industry specific jargon and acronyms are
confusing. Make sure your audience understands your language level, terminology
and above all, what you are asking.
Non-directed questions give respondents excessive
latitude
What suggestions do you have for improving tomato juice? The
question is about taste, but the respondent may offer suggestions about texture,
the type of can or bottle, mixing juices, or something related to use as a mixer
or in recipes.
Forcing answers
Respondents may not want, or may not be
able to provide the information requested. Privacy is an important issue to most
people. Questions about income, occupation, finances, family life, personal
hygiene and beliefs (personal, political, religious) can be too intrusive and
rejected by the respondent.
Non-exhaustive listings
Do you have all of the options
covered? If you are unsure, conduct a pretest using the "Other (please specify)
__________" option. Then revise the question making sure that you cover at least
90% of the respondent answers.
Unbalanced listings
Unbalanced scales may be appropriate
for some situations and biased in others. When measuring alcohol consumption
patterns, One study used a quantity scale that made the heavy drinker appear in
the middle of the scale with the polar ends reflecting no consumption and an
impossible amount to consume. However, we expect all hospitals to offer good
care and may use a scale of excellent, very good, good, fair. We do not expect
poor care.
Double barreled questions
What is the fastest and most
convenient Internet service for you? The fastest is certainly not the most
economical. Two questions should be asked.
Dichotomous questions
Make sure answers are independent.
For example the question "Do you think basketball players as being independent
agents or as employees of their team?" Some believe that yes, they are both.
Long questions
Multiple choice questions are the longest
and most complex. Free text answers are the shortest and easiest to answer. When
you Increase the length of questions and surveys, you decrease the chance of
receiving a completed response.