(Reminder: Don’t forget to utilize the concept maps and
study questions as you study this and the other chapters.)
In this chapter we talk about what experiments are, we talk
about how to control for extraneous variables, and we talk about two sets of
experimental designs (weak designs and strong designs).
(Note: In the next chapter we will talk about middle of the
road experimental designs; they are better than the weak designs discussed in
this chapter, and they are not as good as the strong designs discussed in this
chapter. The middle of the road, or medium quality designs are called
It is important for you to remember that whenever an
experimental research study is conducted the researcher's interest is always in
determining cause and effect.
causal variable is the independent variable (IV) and the effect or outcome
variable is the dependent variable (DV).
research allows us to identify causal relationships because we observe the
result of systematically changing one or more variables under controlled
conditions. This process is called manipulation.
Here is our definition of an experiment: The
experiment is a situation in which a researcher objectively observes phenomena
which are made to occur in a strictly controlled situation where one or more
variables are varied and the others are kept constant.
means that we observe a person's response to a set of conditions that the
observations are made in an environment in which all conditions other than
the ones the researcher presents are kept constant or controlled.
conditions which the researcher presents are systematically varied to see
if a person's responses change with the variation in these conditions.
The independent variable is the variable that is assumed to
be the cause of the effect. It is the variable that the researcher varies
or manipulates in a specific way in order to learn its impact on the outcome
Ways of Manipulating the Independent Variable
In Figure 9.1 (on page 266) you can see three different ways to manipulate
the independent variable. Here is that figure reproduced for your convenience:
the independent variable can be manipulated by presenting a condition or
treatment to one group of individuals and withholding the condition or
treatment from another group of individuals. This is the presence or
the independent variable can be manipulated by varying the amount of a
condition or variable such as varying the amount of a drug which is given
to children with a learning disorder. This is the amount technique.
third way of manipulating the independent variable is to vary the type of
the condition or treatment administered. One type of drug may be
administered to one group of learning disabled children and another type
of drug may be administered to another group of learning disabled
children. This is the type technique.
Potential confounding variables can be controlled for
by using of one or more of a variety of techniques that eliminate the
differential influence an extraneous variable may have for the comparison
groups in a research study.
influence occurs when the influence of an extraneous variable is
different for the various comparison groups.
example, if one group is mostly females and the other group is mostly
males, then the gender may have a differentially effect on the outcome. As
a result, you will not know whether the outcome is due to the treatment or
due to the effect of gender.
the comparison groups are the same on all extraneous variables at the
start of the experiment, then differential influence is unlikely to occur.
experiments, we want our groups to be the same (or “equivalent” on all
potentially confounding extraneous variables). The control techniques are
essentially attempts to make the groups similar or equivalent.
Remember this important point: You want all of your
comparison groups to be similar to each other (on all characteristics or
variables) at the start of an experiment. Then, after manipulating the
independent variable you will be better able to attribute the difference observed
at the posttest to the independent variable because one group got a treatment
and the other group did not.
want the only systematic difference between the groups in an experiment to
be the variation of the independent variable. You want the groups to be
the same on all other variables (i.e., the same on extraneous or
Now we will discuss these six techniques that are used to control for
confounding variables: random assignment, matching, holding the extraneous
variable constant, building the extraneous variable into the research design,
counterbalancing, and analysis of covariance.
Random assignment is the most important technique that can be used
to control confounding variables because it has the ability to control for both
known and unknown confounding extraneous variables. Because of this
characteristic, you should randomly assign whenever and wherever possible.
assignment makes the groups similar on all variables at the start of the
random assignment is successful, the groups will be mirror images of each
You must be careful not to confuse random assignment with
random selection! The two techniques differ in purpose. (Note: I strongly
recommend that you re-read the section titled Random Selection and Ransom
Assignment on pages 216-217; it is only three paragraphs long, but will help
you with this very important distinction!)
The purpose of random selection is to generate a
sample that represents a larger population. This topic was covered in our
earlier chapter on Sampling (Chapter 7).
The purpose of random assignment is to take a
sample (usually a convenience sample) and use the process of randomization to
divide it into two or more groups that represent each other. That is, you use
random assignment to create probabilistically “equivalent” groups.
Note that random selection (randomly selecting a
sample from a population) helps ensure external validity, and random assignment (randomly dividing
a set of people into multiple groups) helps ensure internal validity.
Because the primary goal is experimental research is to
establish firm evidence of cause and effect, random assignment is more
important than random selection in experimental research. It that is
counterintuitive to you, then please reread it as many times as is necessary.
Random assignment controls for the problem of differential
influence (that was discussed earlier). It does they by insuring that each
participant has an equal chance of being assigned to each comparison group.
other words, random assignment eliminates the problem of differential
influence by making the groups similar on all extraneous variables.
equal probability of assignment means that not only are participants
equally likely to be assigned to each comparison group but that the
characteristics they bring with them are also equally likely to be
assigned to each comparison group.
means that the research participants and their characteristics should be
distributed approximately equally in all comparison groups!
random assignment is the best way to create equivalent groups for use in
Here is one way to carry out random assignment that we
included in the first edition of our textbook:
Another way to conduct random assignment is to assign each
person in your sample a number and then use a random assignment computer
program. Here is one: http://www.graphpad.com/quickcalcs/randomize1.cfm
Matching controls for confounding extraneous variables by equating
the comparison groups on one or more variables that are correlated with the
you have to do is to decide what extraneous variables you want to match on
(i.e.., decide what specific variables you want to make your groups
similar on). These variables that you decide to use are called the matching
controls for the matching variables. That is, it eliminates any
differential influence of the matching variables.
can match your groups on one or more extraneous variables.
example, let’s say that you decide to equate your two groups (treatment
and control group) on IQ. That is, IQ is going to be your only matching
variable. What you would do is to rank order all of the participants on
IQ. Then select the first two (i.e., the two people with the two highest
IQs) and put one in the experimental treatment group and the other in the
control group (The best way to do this is to use random assignment to make
these assignments. If you do this then you have actually merged two
control techniques: matching and random assignment). Then take the next
two highest IQ participants and assign one to the experimental group and
one to the control group. Then just continue this process until you assign
one of the lowest IQ participants to one group and the other lowest IQ
participant to the other group. Once you have completed this, your two
groups will be matched on IQ! If you use matching without random
assignment, you run into the problem that although you know that your
groups are matched on IQ you have not matched them on other potentially
- A weakness of matching when it is used
alone (i.e., without also using random assignment) is that you will know
that the groups are equated on the matching variable(s) but you will not
know whether the groups are similar on other potentially confounding
Holding the Extraneous Variable Constant
This technique controls for confounding extraneous variables
by insuring that the participants in the different treatment groups have the
same amount or type on a variable.
example, you might use only people who have an IQ of 120-125 in your
research study if you are worried about IQ as being a confounding
you are worried about gender, this if you used this technique you would
either study females only or males only, but not both.
problem with this technique it that it can seriously limit your ability to
generalize your study results (because you have limited your participants
to only one type).
Building the Extraneous Variable into the Research Design
This technique takes a confounding extraneous variable and
makes it an additional independent variable in your research study.
For example, you might decide to include females and
males in your research study.
This technique is especially useful when you want to
study any effect that the potentially
confounding extraneous variable might have (i.e., you will be able to study the
effect of your original independent variable as well as the additional
variable(s) that you built into your design.
Counterbalancing is a technique used to control for
sequencing effects (the two sequencing effects are order effects and carry-over
that this technique is only relevant for a design in which the
participants receive more than one treatment condition (e.g., such as the repeated
measures design that is discussed later in the chapter)
effects are biasing effects that can occur when each participant must
participate in each experimental treatment condition..
effects are sequencing effects that arise from the order in which the
treatments are administered. For example, as people complete their
participation in their first treatment condition they will become more
familiar with the setting and testing process. When these people
participate, later, in their second treatment condition, they may perform
better simply because are now familiar with the setting and testing that
they acquired earlier. This is how the order can have an effect on the
outcome. Order effects that need to be controlled.
effects are sequencing effects that occur when the effect of one
treatment condition carries over to a second treatment
condition. That is, participants’ performance in a later treatment is
different because of the treatment that occurred prior to it. When this
occurs the responses in subsequent treatment conditions are a function of
the present treatment condition as well as any lingering effect of the
prior treatment condition. Learning from the earlier treatment might
carry-over to later treatments. Physical conditions caused by the earlier
treatment might also carry-over if the time elapsing between the
treatments is not long enough for the earlier effect to dissipate.
is the good news! Counterbalancing is a control technique that can be used to control for order
effects and carry-over effects.
counterbalance by administering each experimental treatment condition to
all groups of participants, but you do it in different orders for
different groups of people.
example if you just had two groups making up your independent variable you
could counterbalance by dividing you sample into two groups and giving
this order to the first group (treatment one followed by treatment two)
and giving this order to the second group (treatment two followed by
Analysis of Covariance
Analysis of covariance (ANCOVA) is a statistical control technique
that is used to statistically equate groups that differ on a pretest or some
example, in multigroup designs that have a pretest, ANCOVA is used to
equate the groups on the pretest.
another example, in a learning research study you might want to control
for intelligence because if there are more brighter students in one of two
comparison groups (and these students are expected to learn faster) then
the difference between the groups might be because the groups differ on IQ
rather than the treatment variable; therefore, you would want to control
of covariance statistically adjusts the dependent variable scores for the
differences that exist on an extraneous variable (your control variable).
selecting variables to control for, note that the only relevant extraneous
variables are those that also affect participants' responses to the
A research design is the outline, plan, or strategy
that you are going to use to obtain an answer to your research question.
Research designs can be weak or strong (or quasi which are moderately strong;
that is, in between the weak and the strong designs) depending on the extent to
which they control for the influence of confounding variables.
Weak Experimental Research Designs
Some research designs are considered weak because they do not control for the
influence of many confounding variables.
The one-group posttest-only design is a very weak research
design where one group of research participants receives an experimental
treatment and is then post tested on the dependent variable.
serious problem with this design is that you do not know whether the
treatment condition had any effect on the participants because you have no
idea as to what their response would be if they were not exposed to the
treatment condition. That is, you don’t have a pretest or a control group
to make your comparison with.
problem with this design is that you do not know if some confounding
extraneous variable affected the participants' responses to the dependent
Because of the problems with this design it generally gives
little evidence as to the effect of the treatment condition.
The next design is the one-group pretest-posttest design.
Here is a depiction of it:
one-group pretest-posttest design is a research design where one
group of participants is pretested on the dependent variable and then
posttested after the treatment condition has been administered.
is a better design than the one-group posttest-only design because it at
least includes a pretest, that indicates how the participants did prior to
administration of the treatment condition.
this design, the effect is taken to be the difference between the pretest
and posttest scores.
does not control for potentially confounding extraneous variables such as
history, maturation, testing, instrumentation, and regression artifacts,
so it is still difficult to identify the effect of the treatment
The next of the weak experimental research designs is the posttest-only
design with nonequivalent groups.
posttest-only design with nonequivalent group includes an experimental
group that receives the treatment condition and a control group that does
not receive the treatment condition or receives some standard condition
and both groups are posttested on the dependent variable.
this design includes a control group (which gives something to compare the
treatment group with), the participants are not randomly assigned to the
groups so there is little assurance that the two groups are equated on any
potentially confounding variables prior to the administration of the
the participants were not randomly assigned to the comparison groups, this
design does not control for differential selection, differential
attrition, and the various additive and interaction effects
For a summary of the threats to validity for the weak
experimental designs, you should study Table 9.1 on page 277.
Strong Experimental Research Designs
A research design is considered to be a "strong research design" if
it controls for the influence of confounding extraneous variables. This
is typically accomplished by including one or more control techniques into the
most important of these control techniques is random assignment.
addition to including control techniques, strong research designs include
a control group which is the comparison group that either does not receive
the experimental treatment condition or receives some standard treatment
will briefly discuss these strong designs: the pretest-posttest
control-group design, the posttest-only control-group design, the
factorial design, the repeated measures design, and the factorial design
based on a mixed model. (For a summary of all of these, look at and study
Table 9.2 on page 281.)
The first strong experimental design is the pretest-posttest
control-group design. Here is a picture of it in its basic form:
pretest-posttest control-group design is a strong research design
in which a group of research participants is randomly assigned to an
experimental and control group. Both groups of participants are pre
tested on the dependent variable and then post tested after the
experimental treatment condition has been administered to the experimental
is an excellent research design because it includes a control or comparison
group and has random assignment.
design controls for all of the standard threats to internal validity.
Differential attrition may or may not be a problem depending on what
happens during the conduct of the experiment.
that while this design is often presented as a two group design, it can be
expanded to include a control group and as many experimental groups as are
needed to test your research question.
The next strong experimental research design is the posttest-only
control group design. Here is a picture of it:
The posttest-only control group design is a research
design in which the research participants are randomly assigned to an
experimental and control group and then post tested on the dependent variable
after the experimental group has received the experimental treatment condition.
is an excellent research design because it includes a control or
comparison group and has random assignment.
like the previous design, it controls for all of the standard threats to
internal validity. Differential attrition may or may not be a problem
depending on what happens during the conduct of the experiment.
design does not include a pretest of the dependent variable, but this does
not detract from its internal validity because it includes the control
group and random assignment which means that the experimental and control
groups are equated at the outset of the experiment.
The next strong experimental research design is the factorial
design. For a depiction of this design, please go to page 281 and look at
it in Table 9.2.
The layout for a factorial design with two independent
variables (Type of instruction and level of anxiety) is shown in Figure 9.14
(p.287) and here for your convenience.
- A factorial
design is a design in which two or more independent variables are
simultaneously investigated to determine the independent and interactive
influence which they have on the dependent variable. It also has random
assignment to the groups.
combination of independent variables is called a "cell."
participants are randomly assigned to as many groups are there are cells
of the factorial design if both of the independent variables can be
research participants are administered the combination of independent
variables that corresponds to the cell to which they have been assigned
and then they respond to the dependent variable.
data collected from this research give information on the effect of each
independent variable separately and the interaction between the
effect of each independent variable on the dependent variable is called a main
effect. There are as many main effects in a factorial design as
there are independent variables. If a research design included the
independent variables of gender and type of instruction, then there would
potentially be two main effects, one for gender and one for type of
- An interaction
effect between two or more independent variables occurs when the
effect which one independent variable has on the dependent variable
depends on the level of the other independent variable. For example,
if gender is one independent variable and method of teaching mathematics
is another independent variable, an interaction would exist if the lecture
method was more effective for teaching males mathematics and
individualized instruction was more effective in teaching females
The next strong experimental research design is the repeated-measures
design. Here is a picture of it in its basic form with counterbalancing:
- A repeated-measures
design is a design in which all research participants receive all
experimental treatment conditions.
example, if you were investigating the effect of type of instruction on
learning mathematics and you used two types of instruction (lecture method
and individualized instruction) the participants would experience both
types of instruction, first one and then the other.
design has the advantage of requiring fewer participants than other designs
because the same participants participate in all experimental conditions.
design also has the advantage of the participants in the various
experimental groups being equated because they are the same participants
in all of the treatment conditions.
you use counterbalancing with this design, then all of the standard
threats to internal validity are controlled for. Differential attrition may or may not be a problem depending
on what happens during the conduct of the experiment.
The last strong experimental research design discussed in
this chapter is the factorial design based on a mixed model. Here is a
picture of this design when it has two independent variables:
factorial design based on a mixed model is a factorial design in which different
participants are randomly assigned to the different levels of one
independent variable but all participants take all levels of another
the depiction above, participants are randomly assigned to variable B, and
all participants receive all levels of variable A.
of the standard threats to internal validity are controlled for with this
design if conuterbalancing is used for the repeated measures independent
variable. Differential attrition may or may not be a problem depending on
what happens during the conduct of the experiment.
As you study the designs in this chapter, two tables will be
of maximum help.
9.1 on page 277 shows the depictions of all of the weak experimental
research designs and the threats to internal validity for each of these
9.2 on page 281 shows the depictions of all of the strong experimental
research designs and the threats to internal validity for each of these
Here are copies of these two tables for your convenience.