Overview
In this module, we will consider two important forms of quantitative data: surveys and official statistics. We will begin by considering the many different types of surveys, and the importance of question wording in survey research. We will examine how researchers use secondary survey data – that is, survey data collected by other researchers – and then move to consider another form of secondary data: official statistics.
As always, to help you better understand and engage with the ideas that we are covering in the modules ahead, I strongly encourage you to discuss the course material in the class discussion boards.
When you have finished this module, you should be able to do the following:
- Outline the primary modes of survey research;
- Discuss how different modes of survey research are subject to coverage and non-response bias;
- Discuss how different modes of survey research are subject to the interviewer effect;
- Distinguish between open and closed questions;
- Identify common close-ended survey question types;
- Assess survey questions for potential problems;
- Outline the advantages and limitations of using secondary survey datasets;
- Critically assess research that uses data collected from surveys; and
- Outline the advantages and limitations of using official statistics.
- Read Chapter 10 of the 3rd edition of our textbook or Chapter 8 of the 4th edition of our textbook. Create self-study flashcards for the chapter.
- Watch mini-lectures “Survey Mode”, “Survey Questions”, “Secondary Survey Datasets”, “Critiquing Research Using Survey Data”, “Official Statistics”.
- Complete Learning Activity.
- Closed question
- Open question
- Forced choice question
- Non-singular
- Ambiguous questions
- Skew the opinion
- Exhaustive
- Mutually exclusive
- Secondary data
- Cross-sectional
- Longitudinal studies
- Panel studies
- Metadata
- Official statistics
- Aggregate data
- Ecological fallacy
- Microdata
- Chapter 10 in Berdahl, Loleen and Keith Archer. Explorations: Conducting Empirical Research in Canadian Political Science (Third Edition). Oxford University Press OR Chapter 8 of Berdahl, Loleen and Jason Roy. Explorations: Conducting Empirical Research in Canadian Political Science (Fourth Edition). Oxford University Press.
Learning Material
Learning Activity
Spot the Problems Exercise
- Watch the (spoof) The Onion video, Most Children Not in Favor of Child Healthcare (see https://www.youtube.com/watch?v=OETJFrpnwZc ). In 300-500 words, identify three or more problems with the survey and/or its interpretation by the commentator. Explain what changes you would want to see to treat this as a legitimate study. In your response, use at least two terms covered in the module (readings and/or videos), and be sure that all terminology used from the module is used correctly. Proofread carefully.
- Post your response in your Learning Activity Discussion Board.
- Provide a constructive response to at least one of your fellow group members’ posts. A constructive response is one that (a) uses supportive language to (b) identify for the author an area in which the work can be strengthened. For example, it may identify an issue where the wording is unclear or a point where terminology is used incorrectly, or suggest ideas for examples or ways to strengthen the argument, or let the author know of questions that the work raised for them. A constructive response goes beyond ‘I agree’ or ‘that is interesting’ to assist the author in improving the work. It should provide feedback that is intended to assist the author of the learning activity in improving their work.
Reminder: At the end of Module 8, you are required to select one learning activity for submission from Modules 5-8. You can use the feedback that you receive in the group forum to revise your selected learning activity prior to submission.
Glossary
aggregate data grouped data for a specified geographic area.
ambiguous questions survey questions that are unclear and/or are misunderstood by the respondents, making their responses unreliable (e.g. questions with multiple stimuli).
census an enumeration or a record of the full population.
closed (or close-ended) questions questions that include a full set of acceptable responses for participants to choose from.
cross-sectional data the data collected from individuals at one point in time.
cross-sectional studies a type of research design intended to gather empirical evidence on an entire population or a representative sample of the population at a particular point in time.
ecological fallacy: the assumption that group level patterns imply individual level patterns
exhaustive: inclusion of all possible response options in a measure
forced choice question a question that asks respondents to select between a limited number of statements.
interviewer effect a situation in which a respondent provides the answer that he or she thinks the interviewer wants.
longitudinal data data collected from the same individuals over time.
longitudinal (or panel) studies studies that interview the same respondents at different points in time in order to study change.
metadata the technical documentation that accompanies secondary data.
microdata non-aggregated data that allows the researcher to consider individual units of analysis (such as individuals or households).
mutually exclusive the idea that two elements cannot coexist
non-singular a model that is open to different possible interpretations.
official statistics statistics that governments and international organizations release to the public.
open (or open-ended) questions questions that are framed in such a way that respondents can state their position without any cueing from the researcher.
secondary data data that are not collected directly by the researcher (e.g. official statistics and data gathered by other research teams).
skew the opinion an uneven distribution of opinion on a topic, in which a disproportionate percentage of the respondents hold an opinion on one side of the issue.
Note: Unless otherwise stated, glossary source is Berdahl, Loleen and Keith Archer. 2015. Explorations: Conducting Empirical Research in Canadian Political Science (Third Edition). Oxford University Press.
References
Berdahl, Loleen and Keith Archer. 2015. Explorations: Conducting Empirical Research in Canadian Political Science (Third Edition). Oxford University Press.
Rubin, Allen. 2008. Practitioner’s Guide to Using Research for Evidence-Based Practice. New Jersey: John Wiley & Sons Inc.
Schnell, Rainer and Frauke Kreuter. 2005. “Separating Inteviewer and Sampling Point Effects.” Journal of Official Statistics.
Statistics Canada. 2009. Overview of the Census: Taking a Census of Population. www12.statcan.ca/census-recensement/ 2006/ref/dict/overview-apercu/pop2-eng.cfm, accessed 5 January 2010.
Züll, C. (2016). Open-Ended Questions. GESIS Survey Guidelines. Mannheim, Germany: GESIS – Leibniz Institute for the Social Sciences. doi: 10.15465/gesis-sg_en_002 https://www.gesis.org/fileadmin/upload/SDMwiki/Zuell_Open-Ended_Questions.pdf