Chapter 11
Nonexperimental Quantitative Research
(Reminder: Don’t forget to utilize the concept maps and study questions as you study this and the other chapters.)
Nonexperimental research is needed because there are many independent variables that we cannot manipulate for one reason or the other (e.g., for ethical reasons, for practical reasons, and for literal reasons such as it is impossible to manipulate some variables). Here’s an example of an experiment where you could not manipulate the independent variable (smoking) for ethical and practical reasons: Randomly assign 500 newborns to experimental and control groups (250 in each group), where the experimental group newborns must smoke cigarettes and the controls do not smoke.
Nonexperimental research is research that lacks manipulation of the independent variable by the researcher; the researcher studies what naturally occurs or has already occurred; and the researcher studies how variables are related.
Steps in
Nonexperimental Research
The pretty much the same as they were in experimental research; however, there are some new considerations to think about if you want to be able to make any cause and effect claims at all (i.e., that an IV>DV).
When examining or conducting nonexperimental research, it is important to watch out for the post hoc fallacy (i.e., arguing, after the fact, that A must have caused B simply because you have observed in the past that A preceded B).
Independent
Variables in Nonexperimental Research
This includes variables that cannot be manipulated, should not be manipulated, or were not manipulated.
Simple Cases of
CausalComparative and Correlational Research
Although the terms causalcomparative research and correlational research are dated, it is still useful to think about the simple cases of these (i.e., studies with only two variables). There are four major points in this section:
· The researcher checks to see if the observed difference between the groups is statistically significant (i.e., not just due to chance) using a "ttest" or an "ANOVA" (these are statistical tests discussed in a later chapter; they tell you if the difference between the means is statistically significant; they are discussed in chapter 16).
·
The researcher
checks to see if the observed correlation is statistically significant (i.e.,
not due to chance) using the "ttest for correlation coefficients"
(it tells you if the relationship is statistically significant; it is discussed
in chapter 16).
·
Remember that
the commonly used correlation coefficient (i.e., the Pearson correlation) only
detects linear relationships.
3. It is essential that you remember this point: Both of the simple cases of nonexperimental research are seriously flawed if you are interested in concluding that an observed relationship is a causal relationship.
·
That's because
"observing a relationship between two variables is not sufficient
grounds for concluding that the relationship is a causal relationship."
(Remember this important point!)
4. You can improve on the simple cases by controlling for extraneous variables and designing longitudinal studies (discussed below).
· And once you move on to these improved nonexperimental designs, you should drop the “correlational” and “causalcomparative” terminology and, instead, talk about the design in terms of the research objective and the time dimension (which is discussed below, and summarized in Table 11.3)
The Three
Necessary Conditions for
CauseandEffect Relationships
It is essential that your remember that researchers must establish three conditions if they are to make a defensible conclusion that changes in variable A cause changes in variable B. Here are the conditions (which have been stated in previous chapters) in a summary table:
Applying the Three
Necessary Conditions
for Causation in Nonexperimental Research
Nonexperimental research is much weaker than strong and quasi experimental research for making justified judgments about cause and effect.
· It is, however, quite easy to establish condition 1 in nonexperimental research—just see if the variables are related For example, Are the variables correlated? or Is there a difference between the means?.
· It is much more difficult to establish conditions 2 and 3 (especially 3).
· When attempting to establish condition 2, researchers use logic and theory (e.g., we know that biological sex occurs before achievement on a math test) and design approaches that are covered later in this chapter (e.g., longitudinal research is a strong design for establishing proper time order).
· Condition 3 is a serious problem in nonexperimental research because it is always possible that an observed relationship is "spurious" (i.e., due to some confounding extraneous variable or "third variable").
· When attempting to establish condition 3, researchers use logic and theory (e.g., make a list of extraneous variables that you want to measure in your research study), control techniques (such as statistical control and matching), and design approaches (such as using a longitudinal design rather than a crosssectional design).
· The rest of the chapter will be explaining these points.
·
To get things started, you need to understand the idea
of controlling for a variable. Here is an example: first, Did you know
that there is a correlation between the number of fire trucks responding to a fire
and the amount of fire damage? Obviously this is not a causal relationship
(i.e., it is a spurious relationship). In Figure 11.2 below, you can see that
after we control for the size of fire, the original positive correlation
between the number of fire trucks responding and the amount of fire damage
becomes a zero correlation (i.e., no relationship).
Techniques of
Control in Nonexperimental Research
We discuss three ways to control for extraneous variables in nonexperimental research.
You could do this by finding people for
each of the cells of the following table:

Low 
Medium 
High 
Low GPA 
15 people 
15 people 
15 people 
Medium GPA 
15 people 
15 people 
15 people 
High GPA 
15 people 
15 people 
15 people 
Now I am going to talk about the two key dimensions that should be used in constructing a nonexperimental research design: the time dimension and the research objective dimension. (Note that these dimensions eliminate the need for the terms correlational and causalcomparative in nonexperimental research.)
The Time Dimension
in Research
Nonexperimental research can be classified according to the time dimension. In particular, Figure 11.3 shows and summarizes the three key ways that nonexperimental research data can vary along the time dimension; in crosssectional research the data are collected at a single point in time, in longitudinal or prospective research data are collected at two or more time points moving forward, and in retrospective research the researcher looks backward in time to obtain the desired data. .
Classifying
Nonexperimental Research
by Research Objective
The idea here is that nonexperimental can be conducted for many reasons. The three most common objectives are description, prediction, and explanation.
One type of explanatory research that I want to mention in
this lecture is called theoretical modeling or causal modeling or structural
equation modeling (those are all synonyms). Causal modeling (i.e.,
constructing theoretical models and then checking their fit with the data) is
commonly used in nonexperimental research.
Here is a way to depict a direct effect: X > Y
Classifying
Nonexperimental Research
by
Time and Research Objective
So we talked about two key
dimensions for classifying nonexperimental research: the time dimension and the
research objective dimension. Notice that these two dimensions can be crossed,
which forms a 3by3 table, which results in 9 types of nonexperimental
research. Here is the resulting
Classification Table:
If
the above table seems complicated, then note that all you really have to do is
to remember to answer these two questions:
1. How
are your data collected in relation to time (i.e., are the data retrospective,
crosssectional, or longitudinal)?
2. What
is the primary research objective (i.e., description, prediction, or
explanation)?
Your answers to these two questions
will lead you to one of the nine cells shown in the above table.