Questionnaire Development

Survey questionnaires are typically used for feedback research to determine the current status or situation or to estimate the distribution of characteristics in a population. Writing a survey questionnaire is one of the most critical stages in the survey development process. Much of survey questionnaire construction is common sense, but there are intricacies with which survey authors should be familiar. It is common sense to require that the concepts be clearly defined and questions unambiguously phrased; otherwise, the resulting data are apt to be seriously misleading. To overcome this problem, eSurveysPro offers a range of sample survey questionnaires, questions and responses to help you in constructing your survey questionnaires.

Here are some ground rules to keep in mind before writing the first word:

• Each question should relate directly to your survey questionnaire objectives.
• Every respondent should be able to answer every question (unless instructed otherwise).
• Each question should be phrased so that all respondents interpret it the same way.
• Each question should provide answers to what you need to know, not what would be nice to know.

The following are the four main parts of a survey questionnaire. Though each of these parts is different from each other, it is important to understand that all of them are necessary for drafting a good survey questionnaire.

Four main parts of a survey questionnaire

Invitation

Invitation involves identifying how you are going to invite your respondents to answer the survey questionnaire. Several ways that are commonly used are emails, website links, or online advertising. By identifying at least one tangible or intangible benefit to respondents for answering the survey questionnaire will help you compose an invitation that encourages respondents to click through. A tangible benefit could be in the form of money or a gift; whereas an intangible benefit is a chance to voice opinions or contribute to research they view as valuable. There are five main parts of an invitation:

1. Introduction
2. Why the respondents have been selected to respond
3. How long will the survey questionnaire take
4. What benefit will they get for responding
5. How their responses will be used / confidentiality

Introduction

The beginning of your survey questionnaire should include an introduction that is enticing and clearly states the purpose of your research. Because web survey questionnaires are self-selecting (i.e., you have no control over who chooses to participate), it is important that your introduction grabs the attention of potential respondents and encourages their participation. It is easy for online survey questionnaire participants to abandon the process, so you must communicate up-front why they should help you with your questions. Failure to do so will decrease the number of participants. The introduction should also include any instructions about completing the survey questionnaire, and an estimate of how much time it will take.

Question types

Based on your feedback objective you have to decide what types of questions will give you the information you need. eSurveysPro helps you develop your survey questionnaires quickly -- and lets you save specialized questions to assure consistency from survey to survey. The different types of questions provided by eSurveysPro are as follows:

Select only one	Single select questions with responses shown vertically, horizontally, in columns, or in a pull-down menu.
Select all that apply	Multiple select questions where users may select several different responses to a specific question.
Free form text	Limited: 1 to 250 characters long. Unlimited: respondents may enter as much text as they want.
Numeric value	Requires respondents to enter a numeric value within a range you specify.
Date value	Requires respondents to enter a date in a format you specify, and within a range you specify.
Matrix	Group of questions that have the same response options or scales. They can be either single select or multiple select.
Data block	A group of questions with related responses. Response types can be specified as text, numeric or date values. You can specify "sum to" values for numeric data blocks.
Rank order	Place in order of importance items from a defined list. You can specify the number of rank options.

eSurveysPro allows you to set data validation parameters for many question types unlike most of the other online survey questionnaire applications. For instance, you can specify a range for numeric answers, or date format for date answers. Additionally, you can mark certain questions as "Response Required," which will enable you to have complete responses and reduce the number of incomplete survey questionnaires.

Our next Best Practice Article will discuss each of these question types in detail, including what results you can expect from each of them.

Close

You can include text, instructions, or additional information at the end of your survey questionnaire. Many surveyors find that the "Closing Text" option under Survey Details gives them an easy way to thank their respondents for their time and effort.

Five steps in writing a survey questionnaire

First: Determine the objective As discussed in last month's Best Practices Article "How to create an Effective Survey Plan," a clear, quantitative feedback objective helps you define the scope of your survey questionnaire and measure its success following completion.

Second: Decide the attributes to measure As with determining the objective, choose which attributes to measure based on your objectives, and with the data evaluation you plan to do. Several attributes you may choose to measure are:

1. Attitude
2. Knowledge
3. Skills
4. Behaviors and practices
5. Perceptions of knowledge, skills or behavior
6. Goals, intentions, aspirations
7. Demographics

Of course, it's possible you might measure more than one, but the questions will be clearly different based on the information you are trying to gather.

Third: Identify the audience As discussed in last month's best practices, identifying which audience you intend to survey will affect the scope of your research. It will also affect how you compose your questionnaire. To ensure that it is appropriate for your audience, "field test" your survey questionnaire with people similar to your respondents before administering the final version. You can further ensure that you measure the right audience by starting the survey questionnaire with appropriate qualifying questions that filter out respondents who aren't a part of your target audience.

Fourth: Choose measurement scales Use scales that provide the information needed and are appropriate for respondents. Some choices are:

1. Fixed Response (Quantitative)
   • Yes-No
   • Multiple Choice
   • Rating scale/Continuum (such as Likert-type scale)
   • Rank ordering

   (Each of these question types will be explained in detail in the next Best Practices Article.)

   Fixed response questions are quick to answer and eSurveysPro graphs them automatically, which facilitates analyzing the results.

   Occasionally, however, fixed response questions may draw misleading conclusions because the respondent cannot qualify responses, e.g. "Yes, but." or "It depends" where only Yes/No are given as options.

2. Narrative Response (Qualitative)
   
   Narrative responses allow respondents greater freedom of expression. There is no bias due to limited response ranges and the respondents can qualify their answers. On the other hand, these responses are time consuming to code and the researcher may misinterpret (and therefore misclassify) a response.

Fifth: Check reliability Reliability is a measure of how consistent the results of using a survey questionnaire will be. By consistent we mean that respondents understand the true meaning of the question as it is stated. Reliability is often first determined using a "pilot test" with the proposed survey questionnaire and might also be repeated with the final version.

An example of a reliability measure is test and then retest the survey, which allows you to determine the repeatability of the instrument. After testing / retesting, republish your eSurveysPro survey questionnaire, selecting "No, remove any responses that have been collected" to eliminate responses collected from your test audience.

Checking for errors

Sometimes survey questionnaires fail to achieve their objectives due to presence of errors. Random error is the most common cause for diminished survey questionnaire reliability, and occurs when questions are poorly worded, or presented leading to inaccurate or uninterpretable answers.

For example: A survey questionnaire of small business owners asked, "How has business activity changed during the past year?" The response options were:

• Increased a lot
• Increased somewhat
• Increased a little
• Decreased

A better way to word this would be:

During the past year, my company's sales revenue has

• Increased significantly
• Increased slightly
• Decreased slightly
• Decreased significantly

We can see in the second example that the question is very specific. The respondents are asked to evaluate their company's sales revenue rather than rate business activity, which could refer to sales revenue, profit, number of employees, etc. Secondly, the scale in the second question is evenly distributed versus the first one, which is biased towards the positive end of the scale.

However, a great deal can usually be done to address random error issues. For example, you can reduce "random" error by removing unusual or confusing questions or by changing their arrangement.

Tips for writing an effective survey questionnaire

Here are the Top 12 points to keep in mind as you write your survey questionnaire:

1. Be brief: You should focus on "need to know" questions and minimize "nice to know" information. Brief questionnaires have higher response rates.
   
   
2. Start with non-threatening questions: Make the first question relevant to the title/purpose, and make it easy to answer. Avoid asking for identifying information in the beginning of the survey questionnaire.
   
   
3. Use plain language: Be direct and use simple language so that respondents can answer quicker and more accurately.
   
   
4. Include simple instructions: When necessary, include instructions about how to complete each section and how to mark the answers to ensure that the survey questionnaire is completed correctly.
   
   
5. Make sure it looks professional: Always proofread your questionnaire and assure that the survey design is appropriate to the topic. A professional survey creates a favorable image in the mind of the respondents about you.
   
   
6. Ask only one question at a time: Avoid "double-barreled" questions that confuse the respondent. Consider the confusion created by these examples:
   
   
   • Do you like cats and dogs?
   • Do you like tennis or do you like golfing?
     
     
7. Use open-ended questions only when the responses add value to the survey research.
   
   
8. Provide space to tell more: At the end of the survey questionnaire, give respondents an opportunity to comment about the survey or general topic using an open-ended question.
   
   
9. Put important questions first: Respondents may get fatigued or hurried by later questions. Include questions about demographic information at the end so the earlier parts of survey questionnaire focus on gathering data necessary to meet your survey objectives.
   
   
10. Avoid agreement bias: By framing both positive and negative questions, your respondents will evaluate each question rather than uniformly agreeing or disagreeing to all of the responses.
    
    
11. Avoid the response option "other": Careless responders will overlook the option they should have designated and conveniently mark the option "other."
    
    
12. Keep your survey questionnaire short: Limit the number of questions based on your target audience. For example you can ask more questions to customers as compared to web-site visitors.
    
    

By evaluating how important each question is to gathering the information you need, and by carefully wording the response options, you will collect information which will yield more satisfactory and meaningful results.