Overview
In this module, we will look at two additional quantitative data collection approaches: experiments and content analysis. The experimental model is often associated with the discipline of psychology, but is increasingly used in political science research, particularly with the study of political behaviour. Content analysis shifts the focus of study from individual people to the study of texts (such as media reports and social media posts).
As always, to help you better understand and engage with the ideas that we are covering in the modules ahead, I strongly encourage you to discuss the course material in the class discussion boards.
When you have finished this module, you should be able to do the following:
- Identify the two key elements of an experimental design;
- Explain why experiments allow political scientists to examine causal relationships;
- Describe the three types of experiments commonly used in political science research, and discuss the advantages and limitations of each;
- Critically assess research that uses data collected from experiments;
- Explain what textual analysis is, and how is it used in political science research;
- Explain the content analysis process; and
- Critically assess research that uses content analysis.
- Read Chapter 11 of the 3rd edition of our textbook or Chapter 9 of the 4th edition of our textbook. Create self-study flashcards for the chapter.
- Watch mini-lectures “Experiments”, “Experiments and Causal Arguments”, “Types of Experiments”, and “Critically Assessing Experimental Research.”
- Read Chapter 12 of the 3rd edition of our textbook or Chapter 6 of the 4th edition of our textbook. Create self-study flashcards for the chapter.
- Watch mini-lectures “Textual Analysis”, “Content Analysis”, and “Critically Assessing Content Analysis.”
- Complete Learning Activity.
- Random assignment
- Experimental group
- Control group
- Planned intervention
- Stimulus
- Laboratory experiments
- Survey experiments
- Field experiments
- Feasibility
- Textual analysis
- Content
- Content analysis
- Structural features
- Substantive features
- Manifest content
- Latent content
- Coding
- Idea
- Frequency
- Direction
- Intensity
- Prominence
- Size
- Codebook
- Coding sheet
- Case identification
- Chapter 11 and Chapter 12 in Berdahl, Loleen and Keith Archer. Explorations: Conducting Empirical Research in Canadian Political Science (Third Edition). Oxford University Press OR Chapter 6 and Chapter 9 in Berdahl, Loleen and Jason Roy. Explorations: Conducting Empirical Research in Canadian Political Science (Fourth Edition). Oxford University Press.
Learning Material
Learning Activity
Considering Textual Analysis
- Read the Farris and Mohamed article, “Picturing immigration: how the media criminalizes immigrants.” This article is behind a paywall. I will email all students a PDF of this article at the beginning of Week 8. I have also placed a PDF of this article under the announcement tab in Canvas. In 300-500 words, explain how Farris and Mohamed combine quantitative and qualitative approaches to textual analysis. Summarize their findings, and indicate whether you find their analysis convincing. In your response, use at least two terms covered in the module (readings and/or videos), and be sure that all terminology used from the module is used correctly. Proofread carefully.
- Post your response in your Learning Activity Discussion Board.
- Provide a constructive response to at least one of your fellow group members’ posts. A constructive response is one that (a) uses supportive language to (b) identify for the author an area in which the work can be strengthened. For example, it may identify an issue where the wording is unclear or a point where terminology is used incorrectly, or suggest ideas for examples or ways to strengthen the argument, or let the author know of questions that the work raised for them. A constructive response goes beyond ‘I agree’ or ‘that is interesting’ to assist the author in improving the work. It should provide feedback that is intended to assist the author of the learning activity in improving their work.
Emily M. Farris & Heather Silber Mohamed (2018) Picturing immigration: how the media criminalizes immigrants, Politics, Groups, and Identities, 6:4, 814-824, DOI: 10.1080/21565503.2018.1484375
Reminder: At the end of Module 8, you are required to select one learning activity for submission from Modules 5-8. You can use the feedback that you receive in the group forum to revise your selected learning activity prior to submission.
Glossary
case identification a number or combination of numbers and letters used to identify the case.
causal effect the difference between the value of an outcome when a subject receives a treatment and when a subject does not receive the treatment.
census an enumeration or a record of the full population.
codebook developed to guide coding, an instrument that lists the elements of the text that will be observed (variables) and itemizes how they will be categorized (values).
coding the systematic categorization of different aspects of communication.
coding sheet a sheet on which coders can record the values for each case.
content any message that can be communicated, including words, meanings, symbols, and themes.
content analysis quantitative research technique used to analyze the message characteristics in any form of communication (Neuendorf 2002, 1; Neuman 2007, 227).
control group a group of subjects randomly assigned not to receive the treatment in an experiment; identical to the treatment group in all other respects.
direction placement of a message along a continuum or classification scheme in order to identify its type.
experimental (or treatment) group a group of subjects exposed to the intervention of interest in an experiment; identical to the control group is all respects except that the control does not receive the treatment.
external validity the extent to which the findings drawn from the cases under examination may be used to make generalizations about phenomena outside the original study.
feasibility the extent to which a study is capable of being completed, given the researcher’s skills and resources.
field experiment an experiment in which a researcher’s intervention is implemented in a subjects’ natural environment.
frequency a measurement of whether something occurs and, if it does, how often.
idea the issue or message being communicated.
intensity the strength of a message.
intercoder reliability the extent to which different coders reach the same conclusions (i.e. assign the same values when coding); a high level of agreement indicates that the coding instrument is reliable.
internal validity the extent to which the researcher has produced results reflective of reality, as measured within the confines of the study.
laboratory experiment an experiment in which subjects are recruited to a common location where the researcher exerts a relatively large degree of control over the experimental setting.
latent content the underlying or implied meaning of a message.
manifest content the literal, or surface, meaning of the message.
planned intervention a situation in which an experimenter purposefully manipulates one or several aspects of subjects’ conditions according to a predefined scheme.
prominence the placement and importance of a message in a text.
random assignment the process of assigning some members of a population to a treatment group and others to a control group; random assignment ensures that the two groups are identical in all respects aside from the receipt of treatment.
size the amount of space or time a message takes up within the text.
stimulus/stimuli the treatment(s)—often involving exposure to different forms of information—to which subjects in an experimental group are exposed.
structural features elements used to convey a message within a text; structural features of a newspaper story include the headline, lead paragraph, author, length, type of story, etc.
substantive features the substantive meanings communicated by a text.
survey experiment an experiment implemented in the context of a survey involving the random manipulation of a part (or parts) of the survey instrument.
textual analysis the systematic examination of the messages and meanings conveyed by texts.
unit of analysis the portion of text that will be coded (e.g. the headline of a news story).
Note: Unless otherwise stated, glossary source is Berdahl, Loleen and Keith Archer. 2015. Explorations: Conducting Empirical Research in Canadian Political Science (Third Edition). Oxford University Press.
References
Berdahl, Loleen and Keith Archer. 2015. Explorations: Conducting Empirical Research in Canadian Political Science (Third Edition). Oxford University Press.
Cetinkaya-Rundel, Mine. 2015. Data Analysis and Statistical Inference. Coursera.org
Morton, Rebecca B., and Kenneth C. Williams. 2008. “Experimentation in Political Science.” In Janet M. Box-Steffensmeier, Henry E. Brady, and David Collier, eds., The Oxford Handbook of Political Methodology. Oxford: Oxford University Press.
Rubenson, Daniel. 2011. “Experiments.” In Archer, K. and L. Berdahl. 2011. Explorations: Conducting Empirical Research in Canadian Political Science (Second Edition). Oxford University Press.