Questionnaires | Definition, Design & Examples
Questionnaires are tools used to collect information from people about their beliefs, experiences, and other characteristics. Questionnaires are used for many purposes, including consumer research, election polling, clinical diagnosis, and academic research.
Survey vs questionnaire
Though the terms survey and questionnaire are often used interchangeably, they are not the same.
A survey is a research method used to collect primary data (the other two methods are direct measurement and observation; Slattery et al., 2011). Surveys involve collecting information from a sample with the aim of describing a population.
On the other hand, a questionnaire is a tool (or “instrument”) that can be used to collect information from a person. It generally contains a series of questions a person answers, either by filling out a digital/paper form or in an interview. A questionnaire that provides a quantitative score is referred to as a scale.
Much like a farmer harvesting a crop might use a tractor to gather fruits or vegetables, a researcher conducting a survey might use a questionnaire to gather data. A survey is a research method, and a questionnaire is a data collection tool that can be used for this method.
General questionnaire template
Despite their wide range of applications, questionnaires follow a general structure: an introduction, followed by the main body, demographics questions, and a conclusion.
| Section | Purpose | Considerations |
|---|---|---|
| Introduction | Introduce the participant to the questionnaire |
|
| Main body | Ask the respondent key questions |
|
| Demographics | Collect any essential background information |
|
| Conclusion | Thank the participant |
|
Question-order bias
When designing your questionnaire, you should carefully consider the order in which you ask questions. How someone answers one question may influence how they respond to subsequent questions.
General to specific:
- How would you rate your overall well-being?
- How frequently do you exercise?
Specific to general:
- How frequently do you exercise?
- How would you rate your overall well-being?
In the general to specific example, the respondent may reflect on multiple facets of health (physical, mental, social, etc.) when rating their overall health. If you instead use a specific to general order and ask about exercise first, the respondent may over-consider exercise habits when rating their overall health.
Because earlier questions can “prime” a respondent to answer in a certain way, you should carefully consider question order. It’s often most effective to start with general, easy-to-answer questions before moving on to more specific or probing ones.
Randomizing question order can minimize question-order bias; however, presenting questions in a random order can make a questionnaire confusing or more cognitively demanding to fill out.
Types of questions
Open-ended vs close-ended questions
A questionnaire might include open-ended questions, close-ended questions, or a combination of the two.
Open-ended questions allow the respondent to use their own words to respond to a question. Though this allows the respondent to provide more detail or to respond in ways the researcher may not have anticipated, these questions can also be tedious to respond to and difficult to parse and analyze.
Close-ended questions (also known as restricted-choice questions) have fixed response options. These may include multiple-choice questions, rating scale questions, and Likert scale questions.
Close-ended questions are easy to respond to and result in data that are straightforward to analyze. However, close-ended questions are generally only useful when collecting quantitative, nominal, or ordinal data. The researcher must also think carefully about which response options to include.
Open-ended: Please describe your recent experience with our customer service team.
Close-ended: How satisfied were you with your recent experience with our customer service team?
- Very satisfied
- Satisfied
- Neutral
- Dissatisfied
- Very dissatisfied
The first question allows the respondent to provide as much detail as they’d like. This might help the company identify specific areas to improve. The second question is much easier to answer, but it provides limited detail on someone’s experience.
Some common forms of close-ended questions are described in the following sections.
Multiple-choice questions
Multiple-choice questions can be used when collecting nominal data (i.e., data that can be sorted into categories but have no inherent order, such as religion, gender, place of birth, or “yes/no” questions).
Multiple-choice questions can also be used to collect ordinal data (categorical data that can be ranked, such as level of education) and quantitative data that have been broken down into categories (e.g., age broken down into decades).
When selecting response options for multiple-choice questions, it’s important to think carefully about the response options you are providing. Responses should be exhaustive (cover all possible options) and mutually exclusive (have no overlap).
Consider the following example of a poorly designed multiple-choice question and how it could be improved:
What type of pet do you have?
- Hound
- Dog
- Cat
- Blue tang
The response options for this question have several issues:
- The answers are not mutually exclusive. Someone with a basset hound could select either “dog” or “hound.”
- The response options are not exhaustive:
- There are no response options for someone who owns a type of pet not listed.
- There are no response options for someone who does not have a pet.
- “Blue tang” is much more specific than the other options. Very specific response options should generally only be included if you have a clear reason to do so.
Improved example:
What type of pet do you have?
- I do not have a pet
- A cat
- A dog
- A fish
- Other (please specify)
This revision addresses the flaws of the previous question.
Rating scale questions
Rating scale questions are a type of multiple-choice question that ask respondents to evaluate something using ordered response options. These scales can be used to measure frequency, satisfaction, level of agreement, or other subjective experiences. Rating scales allow a researcher to convert abstract ideas into numerical measures.
The number of response options on a rating scale is an important design choice. Too few options may limit the accuracy of your data, but too many options may overwhelm the respondent. Two of the most common options are 5- and 7-point scales (Price et al., 2015):
- 5-point scales are best for unipolar response options (i.e., options range from zero to positive), like frequency or effectiveness.
- 7-point scales are frequently used for dichotomous or bipolar response options that range from positive to negative, like satisfaction.
1. How much has pain impacted your daily activities in the past week?
- Never
- Rarely
- Sometimes
- Often
- Always
Question 1 uses a 5-point scale with unipolar response options.
2. How would you describe the pain you are currently experiencing?
- Much worse than normal
- Worse than normal
- Slightly worse than normal
- Normal
- Slightly better than normal
- Better than normal
- Much better than normal
Question 2 uses a 7-point scale with bipolar response options.
Likert scale questions
Likert (pronounced “lick-ert”) scales are a specific type of rating scale that measure someone’s level of agreement with a series of statements about a topic. Likert scales generally have five response options (strongly disagree, disagree, neutral, agree, and strongly agree). Responses can be summed across statements to quantify someone’s general attitude toward a topic. Likert scales are technically a special case of rating scales, but “Likert scale” is often used as a general term.
Question wording
Question wording must be carefully considered when designing a questionnaire. The BRUSO model (Peterson, 2000) suggests that questions should be brief, relevant, unambiguous, specific, and objective. The following example shows how the wording of items on a questionnaire about customer satisfaction with a recent kettle purchase could be improved.
| Question | Description of issue | Revision |
|---|---|---|
| Given the circumstances, how would you systematically evaluate and assign a rating to your overall satisfaction with the cost of the kettle you recently acquired from our company? | This item is not brief; it is verbose, uses complex language, and provides unnecessary detail. | How satisfied are you with the cost of your recently purchased kettle? |
| What is your religion? | This item is not relevant to the purpose of the survey and is therefore inappropriate to include. | This item should be removed. |
| Are you happy with your kettle? | This question is not unambiguous—the respondent may interpret happiness in different ways or evaluate different aspects of the kettle’s function. | Create separate questions to gauge the respondent’s satisfaction with different elements of the kettle. |
| Rate your agreement with the following statement: my kettle heats water quickly and to the desired temperature. | This item is not specific. In fact, it is double-barrelled: it asks the user two questions at once. If they are satisfied with how quickly their kettle heats water but not with the temperature, how would they respond? | Create two separate items. |
| How happy are you that the kettle has high water capacity? | This item is not objective. It leads the respondent to favorably report the water capacity of the kettle. | How satisfied are you with the water capacity of the kettle? |
Steps to design a questionnaire
Careful design and planning are key to creating a valid and reliable questionnaire. The following steps can help ensure that you create a questionnaire that meets your business or research objective. Note that these steps may be more iterative than sequential—you may need to revisit earlier steps as you refine your work.
1. Define your research question
Before you start to create your questionnaire, you should have a good sense of the research question you hope to answer. Consider the topic you are studying, the group of people you wish to collect data from, and the information you need. Is a questionnaire an appropriate instrument to collect your data?
2. Develop your questionnaire
A considerably time-consuming element of questionnaire development is the creation of questions, or items. Carefully consider how response options, question wording, and question order may influence your participants’ responses. Ensure that your questions cover all facets of your topic of interest.
3. Pilot your questionnaire
A pilot is a small study used to test and help develop an experiment or research instrument. Piloting your questionnaire can help reveal gaps in your questions or procedures. You may also identify questions that respondents don’t understand or response options that should be revised.
4. Assess the reliability and validity of your questionnaire
A successful questionnaire should be reliable and valid. A reliable test yields consistent results, and a valid test measures what it is supposed to.
When evaluating reliability, consider elements such as test-retest reliability (does the test yield the same results each time?), interrater reliability (if two different researchers administer the test, will they obtain the same results?), and internal consistency (do questions about related topics receive similar responses?).
There are many types of validity that should be assessed when creating a new questionnaire. These include construct validity, face validity, content validity, criterion validity, convergent validity, and discriminant validity.
Once you are confident that your questionnaire accurately and consistently measures the topic it’s been designed for, you are ready to begin administering it to your sample!
Frequently asked questions about questionnaires
- What’s the difference between a survey and a questionnaire?
-
The terms survey and questionnaire are often used interchangeably, but they are not the same.
A survey is a research method that involves collecting a large amount of information about a topic.
A questionnaire is a tool used to collect data. It comprises a series of questions that someone answers. A questionnaire may be used to collect data when a researcher is conducting a survey.
- What is question-order bias?
-
People are highly sensitive to question order when filling out questionnaires. Question-order bias refers to how earlier questions may influence how someone responds to later questions (this is sometimes called “priming”).
For example, when people are first asked to rate their general life satisfaction and then asked about relationship satisfaction, they respond more positively than when these questions are reversed.
- Is a Likert scale ordinal?
-
Yes—the Likert scale, which is often included in questionnaires, is an example of an ordinal measurement.
Ordinal data have a clear order (items can be logically sorted), but the distance between items is not uniform nor quantifiable.
A Likert scale asks someone to rate how much they agree with a series of statements. Response options typically include “Strongly disagree,” “Disagree,” “Neutral,” “Agree,” and “Strongly agree.”
Because these options have a clear rank (we can easily and objectively order them) but unclear spacing (i.e., the distance between “Disagree” and “Neutral” isn’t necessarily the same as the distance between “Neutral” and “Agree”), a Likert scale is considered an ordinal measurement.
Note: For analysis, Likert scale data are sometimes converted to numbers and treated as integer data. This process allows the research to use certain analysis techniques. It’s always important to carefully consider and provide justification for any analyses you decide to conduct.
- What is a rating scale?
-
A rating scale is a type of multiple-choice question often included in a questionnaire. Rating scales include ordered response options that range from low to high (unipolar) or negative to positive (dichotomous). Rating scales are often used to assess frequency, satisfaction, or other subjective experiences.
A Likert scale is a special type of rating scale that is used to assess a person’s level of agreement with a series of statements about a topic.
- What’s the difference between open-ended and close-ended questions?
-
Open-ended and close-ended questions can both be included in questionnaires, but they serve different purposes.
Open-ended questions have no set response: the respondent can respond in their own words. This freedom may encourage more detailed or unpredicted responses. However, responses to open-ended questions are often more time-consuming and difficult to analyze.
Close-ended questions have set response options. They can be answered quickly but limit the detail the respondent is able to provide. Types of close-ended questions include multiple-choice questions, rating scales, and Likert scales.
