ASK AN EXPERT (link)
Researchers reusing data often want information on how data were collected. This includes the actual questions that are used in the questionnaire. There is other information (listed below) that is also relevant and needs to be included when documenting the survey questionnaire. Basically, any information surrounding the actual questionnaire and why the different questions were included (or not) should be included in the documentation.
Why Document the Questionnaire?
It is important that any documentation (metadata) surrounding the questionnaire be recorded as the questionnaire is being developed. This is not only good research practice, it will also ensure that the information is preserved for the future. There are different methods for creating questionnaires, and documenting the information is necessary at all times. Some of the different circumstances that can occur are: a separate company is designing the actual questionnaire; a different company is administering the actual survey; there is a team of researchers working on different parts of the questionnaire; there is a research assistant working on the questionnaire who is on a contract and will not be there for the duration of the research project; and documenting which CAI (computer assisted interviewing) system was used, if one was used. The commonality among the previous situations is that there may not be anyone to answer any questions about the questionnaire once the research is complete if the information has not been recorded. Thus, it is important that all information be recorded.
DDI is useful for documenting questionnaires as it provides comprehensive information about the data collection phase of the research data lifecycle. Data producers can export DDI profiles from the CAI systems they use, thereby capturing full metadata and making it much easier to document and archive the data. For more information on this subject, check out “Survey metadata reusability and exchange: A call to action for questionnaire documentation”.
Content to Include
Note: This is not an exhaustive list
- Actual questions
- Question order
- Skip patterns for the survey
- Introductions used (for the questionnaire and for the different sections)
- Any information that is not directly asked of the respondent but is determined by the interviewer (e.g., gender)
- Format of the questionnaire (e.g., paper, telephone, internet)
- Questionnaire designer
- Universe for each question
- Showcard information
- Interviewer instructions
Use Cases and Examples
The figure below is from the “Canadian General Social Survey, Cycle 25, 2011”. It is an illustration of the DDI information that can be garnered about a variable when the relevant and available information concerning the actual question is documented.
(taken from odesi.ca)
The metadata in this figure includes:
- the variable name and label
- the prequestion text
- the literal question
- the postquestion text
- the value labels and frequencies
- the summary statistics
- the interviewer instructions
- the universe for this particular question
This variable has been well documented and would serve as an excellent example to someone doing similar research.
Getting Started
Once you have designed your research question, conducted your literature review, determined your population of interest and your method of inquiry, it is time to figure out how you are going to collect your data. This is where the design of the questionnaire happens.
The type of research you are conducting (e.g., qualitative versus quantitative) will also help determine the types of questions that will be used. For example, if you are conducting qualitative research, you will want to use more open-ended and probing questions to help gather data. If you are conducting quantitative research, your literature review will help determine the groups and types of questions you will be using.
When you are designing a questionnaire for the first time, it is helpful to examine other questionnaires that have been used in similar studies. Many studies will publish their questionnaires so you will be able to examine them to get tips for your data collection.
Tools
Note: This is not an exhaustive list, it is merely intended to get you started.
Data Collection, Colectica Designer
http://docs.colectica.com/designer/manage-content/data-collection/
Michigan Questionnaire Documentation System (MQDS)
(based on the Blaise data model)
http://inventions.umich.edu/technologies/4975_michigan-questionnaire-documentation-system
Onyx
(open source software for epidemiological data management and analysis)
http://www.obiba.org/
REDCap
(for building and managing online surveys and databases, and to support data capture)
http://www.project-redcap.org/
REDCap and DDI Exporting Data and Metadata with the API
http://kuscholarworks.ku.edu/handle/1808/11047
“The DASISH Questionnaire Design Documentation Tool: Keeping Track of the Questionnaire Design Process” Bakkmoen, H.V., Orten, H. (Norwegian Social Science Data Services), Prestage, Y. (City University London)
http://www.eddi-conferences.eu/ocs/index.php/eddi/eddi14/paper/viewFile/164/98