Summer school

Principles and Practices of Multifactorial Survey Experiments - 3 ECTS

Introduction to setting up multifactorial survey experiments for estimating causal effects in social, economic, environmental and political contexts

Organised by Wageningen School of Social Sciences (WASS)

Mon 12 December 2022 until Fri 16 December 2022

Venue Leeuwenborch, building number 201

Many social, economic, environmental and political problems can only be solved by involving citizens and changing social norms in societies. It is therefore crucial to learn more about individuals’ beliefs, attitudes, preferences, and subjective norms. But survey research on such issues has to deal with different forms of bias. Respondents might untruthfully answer survey questions in line with social norms, political rules, and in a way to please the researchers. Multifactorial survey experiments (MFSEs) can help to avoid such biases because they do not measure the concepts directly via single survey items but indirectly, based on the variation of different factors. They further single out the importance of different factors and combinations thereof for evaluating social, economic, environmental or political problems. Based on an experimental design they allow to estimate causal effects of various factors on the outcome at hand. Therefore, MFSE can contribute to solving social problems. They are applied to a wide range of issues in both Global North and Global South including policy preferences for immigration, acceptance of environmental policy, perceived fairness of gender pay gaps, the valuation of environmental amenities, preferences for health care, normative beliefs related to marriage and child education. This course will provide an introduction to different types of MFSEs: factorial survey/vignette experiments, conjoint experiments, and stated/discrete choice experiments. Participants will get familiar with the theoretical foundations of these experiments, and empirical applications, as well as learn how to design MFSEs, construct a questionnaire, collect data, build a dataset, and analyse MFSE data. As part of this course participants will carry out their own MFSE (based on self-selected topics, for example, on preferences, attitudes or subjective norms related to immigration, working conditions, environmental policy, health care, crime, and discrimination).


Day Time Activity Topic
Day 1 9.00-12.00 Lecture Uncovering Social Mechanisms: The Merits of Multifactorial Survey Experiments
13.00-16.00 Group work Discussing Empirical Applications
16.00-18.00 Seminar Joint Reflection on Empirical Applicants
Day 2 9.00-12.00 Lecture Designing Good Experiments and Asking Good Questions: Basic Principles of Experimental Designs for MFSEs and Questionnaire Design
13.00-16.00 Group work Discussing Experimental Designs and Developing Ideas for own Eperiments
16.00-18.00 Seminar Joint Reflection on Experimental Designs and own Ideas
Day 3 9.00-12.00 Empirical Research Bringing Ideas to Life I: Collecting MFSE Data for Own Experiments
13.00-17.00 Group work Bringing Ideas to Life II: Creating a Dataset for Own MFSE
17.00-18.00 Seminar Joint Reflection on own Empirical Research
Day 4 9.00-12.00 Lecture Getting to Know Your Data: Analysing and Presenting MFSE Data
13.00-17.00 Group work Analysing MFSE Data with focus on own Experiment
17.00-18.00 Seminar Joint Reflection on Data Analysis
Day 5 9.00-12.00 Lecture Discovering Heterogeneity: Analysing and Presenting Heterogeneity in MFSE Data
13.00-16.00 Group work Analysing Heterogeneity in MFSE Data with Focus on Own Experiment
16.00-18.00 Group presentation Presenting Own Experimental Research and Talking about What We Have (Not) Learned

Learning outcomes

After successful completion of this course, participants are expected to be able to:

  • understand the (theoretical) foundations of different types of multifactorial survey experiments,
  • understand under what conditions each type of multifactorial survey experiments is applied,
  • understand the foundations of experimental designs for multifactorial survey experiments,
  • carry out a multifactorial survey experiment including designing the survey,
  • build a dataset for multifactorial survey experiments,
  • analyse multifactorial survey experiment data,
  • reflect critically on the pitfalls of conducting multifactorial survey experiments,
  • reflect critically on the advantages and disadvantages of multifactorial survey experiments.


    Lectures, seminar/workshops, group work, empirical research, group presentations. Suggested preparatory reading:

    Auspurg, Katrin, and Thomas Hinz. 2015. Multifactorial Experiments in Surveys. Conjoint Analysis, Choice Experiments, Factorial Surveys. Pp. 291-315 in Experimente in den Sozialwissenschaften. Sonderband der Sozialen Welt 22, edited by M. Keuschnigg and T. Wolbring. Baden-Baden: Nomos. [This paper provides an overview of different types of MFSEs.]

    Liebe, Ulf, and Jürgen Meyerhoff. 2021.Mapping potentials and challenges of choice modelling for social science research.Journal of Choice Modelling38 (Special Issue on Choice Modelling in Social Science Research): 100270. [Sections 1 and 2 of this paper refer to several applications of MFSEs and related choice models in the social sciences.]

    Target group

    Social Science students. Min. 10 students, Max: 20 students.

    Assumed prior knowledge

    Some familiarity with quantitative methods and data analysis (descriptive and bivariate analysis, as well as multiple regression analysis). There are no requirements regarding statistical software. Data analysis can be conducted with R, Stata, and SPSS. Material will be prepared for different software options.


    Participants work in groups of 2 or 3 persons on the assignments (participants may also choose to submit them individually if they
    wish). The final grade is based on the presentation (final day of the course, 25%) and a 4000-word paper (75%) on the own multifactorial survey experiment.
    The deadline for paper submission is 28 February 2023. Participants will not receive a grade, but a qualitative assessment of whether they passed the course.

      Course fees

      WGS PhDs with TSP € 250
      a) All other PhD candidates b) Postdocs and staff of the above mentioned Graduate Schools € 500
      All others € 750

      The course fee includes additional training material, coffee/tea, lunches.

      Cancellation conditions:

      Participants can cancel their registration free of charge 1 month before the course starts. A cancellation fee of 100% applies if a participant cancels his/her registration less than 1 month prior to the start of the course.

      The organisers have the right to cancel the course no later than one month before the planned course start date in the case that the number of registrations does not reach the minimum.

      The participants will be notified of any changes at their e-mail addresses.