GESIS Training Courses
user_jsdisabled
Search

Scientific Coordination

André Ernst
Tel: +49 221 4703736

Administrative Coordination

Claudia O'Donovan-Bellante
Tel: +49 621 1246-221

Questionnaires for Cross-Cultural and Cross-National Surveys

About
Location:
Mannheim, B6, 4-5
General Topics:
Course Level:
Format:
Software used:
Duration:
Language:
Fees:
Students: 200 €
Academics: 300 €
Commercial: 600 €
 
Keywords
Additional links
Lecturer(s): Dorothée Behr, Cornelia Neuert, Lydia Repke

About the lecturer - Dorothée Behr

About the lecturer - Cornelia Neuert

About the lecturer - Lydia Repke

Course description

Cross-national and cross-cultural surveys are increasingly being set up in social science research as they allow testing for the generality of hypotheses across cultures and allow the inclusion of all members of the target population. When dealing with cross-national or cross-cultural instead of national data, additional methodological challenges and sources of bias must be taken into account as respondents come from different contexts and cultures, and speak different languages. This adds an “additional layer of complexity” to the development and analysis of such surveys.
This workshop focuses on what it means to develop, evaluate, and translate questionnaires for use in cross-national or cross-cultural research. The workshop will introduce a) the notion of measurement in comparative research; b) crucial steps in the development of the source questionnaire, such as the involvement of cultural experts, the use of cross-cultural cognitive interviewing methods, of SQP (Survey Quality Predictor), and of translatability assessments. Furthermore, it will c) introduce best practice approaches to questionnaire translation and assessment, such as TRAPD (Translation, Review, Adjudication, Pretesting, and Documentation), while also presenting modified frameworks or aspects to consider when actually translating. The course will be a mixture of lectures, short practical exercises, and discussions. The course is suited for beginners.
Please note that this course does not include statistical procedures to test for measurement equivalence.


Target group

Participants will find the course useful if:
  • they develop their own questionnaires for cross-cultural/cross-national surveys;
  • they want to understand the implications of questionnaire design and translation for the comparability of cross-national/cross-cultural survey data;
  • they organize questionnaire translation and assessment or translate and review translated questionnaires themselves; or
  • they work in a survey organization and are involved in cross-national/cross-cultural survey research.


  • Learning objectives

    By the end of the course participants will:
  • be aware of the challenges and procedures for cross-cultural questionnaire development;
  • know how a comparable understanding of questions can be achieved;
  • be familiar with best practices in questionnaire translation and assessment;
  • be familiar with several questionnaire evaluation methods.
  •  
    Organizational structure of the course
  • Within each thematic block, participants will have the possibility to work individually as well as jointly on hands-on exercises, such as the translation of a questionnaire.
  • We will also factor in time for the discussion of individual projects.


  • Prerequisites

  • Basic knowledge in questionnaire design and survey methodology will be beneficial.
  • Some practical experience in conducting cross-national/cross-cultural surveys or a planned survey in this field will be beneficial.
  • Knowing another language besides the language of instruction (i.e., English) will be beneficial to get involved in exercises.
  • There are no statistical prerequisites.
  •  
    Software and hardware requirements
    It will be enough to bring along your laptop. No specific software is needed.
     
    Schedule
    Monday, 13th November
    10:00 - 11:30Intro and measurement in comparative surveys
    11:30 - 13:00Developing questionnaires for cross-cultural surveys (I) (incl. short break)
    13:00-13:45Fingerfood(at)GESIS
    13:45 - 15:00Developing questionnaires for cross-cultural surveys (II)
    15:00 - 17:15Best practice in questionnaire translation (I) (incl. Coffee Break)
    17:15 - 17:45Wrap-up and open questions
     
    Further readings:
  • Behr, D. (2018). Translating questionnaires for cross-national surveys - a description of a genre and its particularities based on the ISO 17100 categorization of translator competences." Translation & Interpreting 10 (2): 5-20.
  • Harkness, J. A., Villar, A. & Edwards, B. (2010). Translation, Adaptation, and Design. In J. A. Harkness, M. Braun, B. Edwards, T. P. Johnson, L. Lyberg, P. Ph. Mohler, B.-E. Pennell, & T. W. Smith (Eds.), Survey Methods in Multinational, Multiregional, and Multicultural Contexts, (117-140). Hoboken, NJ.
  • Smith, T. (2004). Developing and evaluating cross-national survey instruments. In Methods for Testing and Evaluating Survey Questionnaires, edited by S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer, 431-452. Hoboken, NJ: Wiley.
  •  
    Tuesday, 14th November
    09:00 - 10:30Best practice in questionnaire translation (II)
    10:30 - 10:45Coffee break
    10:45 - 12:15Questionnaire testing: Cognitive Interviewing
    12:15-13:15Lunchbreak
    13:15 - 14:45Questionnaire testing: Web Probing
    14:45 - 15:00Coffee Break
    15:00 - 16:00Quantitative testing
    16:00 - 17:00Case studies
     
    Further readings:
  • Behr, D., Meitinger, K., Braun, M., & Kaczmirek, L. (2017). Web probing - implementing probing techniques from cognitive interviewing in web surveys with the goal to assess the validity of survey questions. Mannheim, GESIS - Leibniz-Institute for the Social Sciences (GESIS - Survey Guidelines). DOI: 10.15465/gesis-sg_en_023
  • Harkness, J., with (alphabetically) Bilgen, I., Córdova Cazar, A. L., Hu, M., Huang, L., Lee, S., Liu, M., Miller, D., Stange, M., Villar, A., & Yan, T. (2016). Questionnaire Design. Guidelines for Best Practice in Cross-Cultural Surveys. Ann Arbor, MI: Survey Research Center, Institute for Social Research, University of Michigan. https://ccsg.isr.umich.edu/chapters
  • Willis, G. B., & Miller, K. (2011). Cross-Cultural Cognitive Interviewing: Seeking Comparability and Enhancing Understanding. Field Methods, 23(4), 331-341. https://doi.org/10.1177/1525822X11416092
  •  
    Additional recommended literature
  • Jowell, R. (1998). How Comparative is Comparative Research? American Behavioral Scientist, 42(2), 168-177.
  • Johnson, T. P., & Braun, M. (2016). Challenges of Comparative Survey Research. In C. Wolf, D. Joye, T. W. Smith, & Y. Fu (Eds.), The SAGE Handbook of Survey Methodology, (41-53) London: Sage.
  • These publications allow familiarizing oneself with statistical techniques for testing for equivalence, which will not be covered in the course.
  • Braun, M., & Johnson, T. P. (2010). An Illustrative Review of Techniques for Detecting Inequivalences. In J. A. Harkness, M. Braun, B. Edwards, T. P. Johnson, L. Lyberg, P. Ph. Mohler, B.-E. Pennell, & T. W. Smith (Eds.), Survey Methods in Multinational, Multiregional, and Multicultural Contexts. Hoboken, NJ, USA. John Wiley & Sons, Inc.
  • Cieciuch, J., Davidov, E., Schmidt, P., & Algesheimer, R. (2016). Assessment of cross-cultural comparability. In C. Wolf, D. Joye, T. W. Smith & Y. Fu (Eds.), The SAGE Handbook of Survey Methodology, (630-648). London: Sage.
  • Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., & Billiet, J. (2014). Measurement Equivalence in Cross-National Research. Annual Review of Sociology, 40, 55-75.


  •