Summer school
Principles and Practices of Multifactorial Survey Experiments - 3 ECTS
Introduction to setting up multifactorial survey experiments for estimating causal effects in social, economic, environmental and political contexts
Many social, economic, environmental and political problems can only be solved by involving citizens and changing social norms in societies. It is therefore crucial to learn more about individuals’ beliefs, attitudes, preferences, and subjective norms. But survey research on such issues has to deal with different forms of bias. Respondents might untruthfully answer survey questions in line with social norms, political rules, and in a way to please the researchers. Multifactorial survey experiments (MFSEs) can help to avoid such biases because they do not measure the concepts directly via single survey items but indirectly, based on the variation of different factors. They further single out the importance of different factors and combinations thereof for evaluating social, economic, environmental or political problems. Based on an experimental design they allow to estimate causal effects of various factors on the outcome at hand. Therefore, MFSE can contribute to solving social problems. They are applied to a wide range of issues in both Global North and Global South including policy preferences for immigration, acceptance of environmental policy, perceived fairness of gender pay gaps, the valuation of environmental amenities, preferences for health care, normative beliefs related to marriage and child education. This course will provide an introduction to different types of MFSEs: factorial survey/vignette experiments, conjoint experiments, and stated/discrete choice experiments. Participants will get familiar with the theoretical foundations of these experiments, and empirical applications, as well as learn how to design MFSEs, construct a questionnaire, collect data, build a dataset, and analyse MFSE data. As part of this course participants will carry out their own MFSE (based on self-selected topics, for example, on preferences, attitudes or subjective norms related to immigration, working conditions, environmental policy, health care, crime, and discrimination).
Schedule
| Day | Time | Activity | Topic |
|---|---|---|---|
| Day 1 | 9.00-12.00 | Lecture | Uncovering Social Mechanisms: The Merits of Multifactorial Survey Experiments |
| 13.00-16.00 | Group work | Discussing Empirical Applications | |
| 16.00-18.00 | Seminar | Joint Reflection on Empirical Applicants | |
| Day 2 | 9.00-12.00 | Lecture | Designing Good Experiments and Asking Good Questions: Basic Principles of Experimental Designs for MFSEs and Questionnaire Design |
| 13.00-16.00 | Group work | Discussing Experimental Designs and Developing Ideas for own Eperiments | |
| 16.00-18.00 | Seminar | Joint Reflection on Experimental Designs and own Ideas | |
| Day 3 | 9.00-12.00 | Empirical Research | Bringing Ideas to Life I: Collecting MFSE Data for Own Experiments |
| 13.00-17.00 | Group work | Bringing Ideas to Life II: Creating a Dataset for Own MFSE | |
| 17.00-18.00 | Seminar | Joint Reflection on own Empirical Research | |
| Day 4 | 9.00-12.00 | Lecture | Getting to Know Your Data: Analysing and Presenting MFSE Data |
| 13.00-17.00 | Group work | Analysing MFSE Data with focus on own Experiment | |
| 17.00-18.00 | Seminar | Joint Reflection on Data Analysis | |
| Day 5 | 9.00-12.00 | Lecture | Discovering Heterogeneity: Analysing and Presenting Heterogeneity in MFSE Data |
| 13.00-16.00 | Group work | Analysing Heterogeneity in MFSE Data with Focus on Own Experiment | |
| 16.00-18.00 | Group presentation | Presenting Own Experimental Research and Talking about What We Have (Not) Learned |
Learning outcomes
After successful completion of this course, participants are expected to be able to:
- understand the (theoretical) foundations of different types of multifactorial survey experiments,
- understand under what conditions each type of multifactorial survey experiments is applied,
- understand the foundations of experimental designs for multifactorial survey experiments,
- carry out a multifactorial survey experiment including designing the survey,
- build a dataset for multifactorial survey experiments,
- analyse multifactorial survey experiment data,
- reflect critically on the pitfalls of conducting multifactorial survey experiments,
- reflect critically on the advantages and disadvantages of multifactorial survey experiments.
Activities
Lectures, seminar/workshops, group work, empirical research, group presentations. Suggested preparatory reading:
Auspurg, Katrin, and Thomas Hinz. 2015. Multifactorial Experiments in Surveys. Conjoint Analysis, Choice Experiments, Factorial Surveys. Pp. 291-315 in Experimente in den Sozialwissenschaften. Sonderband der Sozialen Welt 22, edited by M. Keuschnigg and T. Wolbring. Baden-Baden: Nomos. [This paper provides an overview of different types of MFSEs.]
Liebe, Ulf, and Jürgen Meyerhoff. 2021.Mapping potentials and challenges of choice modelling for social science research.Journal of Choice Modelling38 (Special Issue on Choice Modelling in Social Science Research): 100270. [Sections 1 and 2 of this paper refer to several applications of MFSEs and related choice models in the social sciences.]
Target group
Social Science students. Min. 10 students, Max: 20 students.
Assumed prior knowledge
Some familiarity with quantitative methods and data analysis (descriptive and bivariate analysis, as well as multiple regression analysis). There are no requirements regarding statistical software. Data analysis can be conducted with R, Stata, and SPSS. Material will be prepared for different software options.
Assessment
Participants work in groups of 2 or 3 persons on the assignments (participants may also choose to submit them individually if they
wish). The final grade is based on the presentation (final day of the course, 25%) and a 4000-word paper (75%) on the own multifactorial survey experiment.
The deadline for paper submission is 28 February 2023. Participants will not receive a grade, but a qualitative assessment of whether they passed the course.
Course fees
| WGS PhDs with TSP | € 250 |
| a) All other PhD candidates b) Postdocs and staff of the above mentioned Graduate Schools | € 500 |
| All others | € 750 |
The course fee includes additional training material, coffee/tea, lunches.