Questionnaire design, measurement quality and comparability

Description of the course: In this course we will introduce participants to the development and testing of survey questions and survey questionnaires. The first part of this course is dedicated to the many choices and options available, how they are interrelated and how they can affect the quality of the measurement instruments and thus the quality of the survey data. As all decisions can interact with each other, it is difficult to decide what the best formulation for a question is, i.e. the formulation which will give the highest quality data. One tool available to help us in this decision is the Survey Quality Predictor (SQP) software. We will show how this tool works and how it can be used during the questionnaire design stage. In the second part, the course focuses on aspects related to questionnaire design in multilingual and multicultural research, like for example translation, using examples from cross-cultural surveys, such as the European Social Survey. In the third part, participants will be introduced to methods to evaluate questions pre- and post-data collection. We show the different experimental methods to estimate measurement quality, the additional paradata that can be collected to assess measurement quality after data collection, and the different approaches to test for measurement equivalence across groups or countries.

Course prerequisites: None

Software used: We will introduce the software Survey Quality Predictor (SQP) and use packages from the R environment. We therefore recommend to assist the “Introduction to R” course previously.

Main references: Saris, W.E., and Gallhofer, I.N. (2014) Design, Evaluation and Analysis of Questionnaires for Survey Research. New York: Wiley. Second Edition.

Guidelines for Best Practice in Cross-Cultural Surveys. Ann Arbor, MI: Survey Research Center, Institute for Social Research, University of Michigan. http://www.ccsg.isr.umich.edu/.