Factorial Designs Intro

Outline:

-- why we do them
-- language
-- Main Effects and Interactions
    -- Definitions
    -- Graphs
    -- Math (ANOVA) approach
    -- When the Math and Graph do not agree


Factorial Designs are those that involve more than one factor (IV). In this course we will only deal with 2 factors at a time -- what are called 2-way designs.

-- why we do them

-- t-test let us make comparisons between two groups -- 2 different levels of one IV
-- one-way ANOVA let us compare multiple levels of one IV
The problem is that we are stuck with just using one IV so far. This is a problem because we seldom think that differences in a DV are just related to only one thing. We usually think that things are a little more complex, that there are several things related to any human behavior.


There are three basic reasons for doing 2-way designs.

-- 2 IVs of interest
-- For control
-- Critical Experiments


2 IVs (factors) of interest

This is the case if we think that there are two or more variables related to the phenomenon and we want to look at both at the same time. This way we can see how things work together to cause changes. Take for example my interest in how shared knowledge affects remembering. Shared knowledge consists of 1) knowledge of the to be remembered material and 2) background knowledge (familiarity with partner). I've started on one issue and suggested for our one-way design that we look at familiarity. How about if we want to study both IVs at the same time?

Draw as a box
 
IVA: Shared Knowledge of the Material
IVB: Familiarity of Conversation Partners
Knowledge of the Same Material 
Not Shared Knowledge
Stranger
Strangers remembering shared material
Strangers remembering different material
Roommates
Roommates remembering shared material
Roommates remembering different material

For control

Sometimes we are really interested in one IV but know that another IV (based on theories or previous research) is also related to the DV. Sometimes we include this other IV for control purposes -- 1) it will generally decrease our MSE and make it easier to find effects of the IV of interest, and 2) we can be sure that the IV of interest works the same way in all situations of interest.

1. For example, based on Deborah Tannen's work and previous work I have done with some students here I have reason to believe that men and women may talk about the past differently. Just doing the experiment without paying attention to gender may increase my within-group variability. If some men and some women talk to strangers and men and women differ, then I will have high variability in that group. If some men and some women talk to their roommates and men and women differ then I will have high variability in that group. If I group by gender as well, however, I will have lower within-group variability. Men talking to stranger will have low variability within groups, etc. This will make it easier to find an effect of familiarity.

2. In addition, I can then be sure that men and women will behave similarly in response to the variable of interest.

Draw as a box
 
IVA Gender
IVB: Familiarity of Conversation Partners
Males
Females
Stranger
Male Strangers remembering 
Female Strangers remembering 
Roommates
Male Roommates remembering 
Female Roommates remembering 

Critical Experiments

Sometimes we are lucky and bright enough to be able to compare to theories in one experiment by using each theory to suggest one IV. Example is Chi's work on memory development. As kids get older they are able to remember more. The maturation theory says it has to do with the development of the brain and the ability to process information. This is directly tied to age. The expanding knowledge base theory suggests that as you know more you can learn more. This is usually tied to age. But since it isn't directly tied to age it suggests that you can get some young kids who are experts in a given domain, and some adults who aren't experts in that domain. This is what Chi did in the domain of chess.

Draw as a box
 
IVA Age
IVB: Chess Expertise
Young
College Students
Novices
Young Novices
College Age Novices
Experts
Young Experts
College Age Experts

For DVs, Chi measured the ability to remember the location of chess pieces on a chess board and the ability to do simple memory tasks.

This nicely pits theories against one another and one very likely will be rejected.

-- language

IV (Independent Variable) = Factor = Treatment (there can be two or more in factorial design)

Levels (each IV has two or more levels)

Cells (the specific confluence of the levels of all IVs)

The simplest case is what is called a 2 x 2 design.

Draw as a box
 
IV  A
IV B
A = 1
A = 2
    B = 1
A1, B1
A2, B1
    B = 2
A1, B2
A2, B2

This is the simplest case of a two way design,
each IVhas two levels.
IV A has 1 and 2.
IVB has 1 and 2.
There are 4 cells: A1B1, A1B2, A2B1, A2B2
 

This is a 2 x 2 design. 2x2 tells you a lot about the design:
there are two numbers so there 2 IVs
the first number is a 2 so the first IV has 2 levels
the second number is a 2 so the second IV has 2 levels
2 x 2 = 4 and that is the number of cells
A 2x3 design
there are two numbers so there 2 IVs
the first number is a 2 so the first IV has 2 levels
the second number is a 3 so the second IV has 3 levels
2 x3 = 6 and that is the number of cells
A 2x2x3 design
there are three numbers so there 3 IVs
the first number is a 2 so the first IV has 2 levels
the second number is a 2 so the second IV has 2 levels
the third number is a 3 so the third IV has 3 levels
2 x 2 x 3 = 12 and that is the number of cells


-- Main Effects and Interactions

When doing factorial design there are two classes of effects that we are interested in: Main Effects and Interactions

-- There is the possibility of a main effect associated with each factor.

-- There is the possibility of an interaction associated with each relationship among factors. (With a two-way design there is only one relationship, A x B)

In a 2-way design

-- Main Effect of Factor A (1st IV): Overall difference among the levels of A that is consistent across the levels of B. (Difference here mostly refers to direction, not to the size of the difference).

-- Main Effect of Factor B (2nd IV): Overall difference among the levels of B that is consistent across the levels of A. (Difference here mostly refers to direction).

-- Interaction of AxB: Differences among the levels of one Factor depend on the levels on the other Factor. (Difference here refers to direction and size of the effect). This means, for example, that some difference between the levels of factor A may hold true at one level of B but not at another level of B; or that the difference between two levels of A may be much stronger at one level of B than at another level of B, even though it is in the same direction.

An easy way to look for Main Effects and Interactions is by graphing the Cell Means.

In each cell I have given you the cell Mean = MA,B
 
IV  A
IV B
A = 1
A = 2
    B = 1
M1,1 = 10
M2,1 = 15
    B = 2
M1,2 = 15
M2,2 = 20

 


-- Definitions

-- Main Effect of Factor A (1st IV): Overall difference among the levels of A that is consistent across the levels of B. (Difference here mostly refers to direction).

-- Main Effect of Factor B (2nd IV): Overall difference among the levels of B that is consistent across the levels of A. (Difference here mostly refers to direction).

-- Interaction of AxB: Differences among the levels of one Factor depend on the levels on the other Factor. (Difference here refers to direction and size of the effect).

-- Graphs Look at the examples done in class

ME of A: Difference between A1 and A2 is in the same direction for both levels of B.

ME of B: Difference between B1 and B2 is in the same direction for both levels of A.

Interaction: The slopes of the lines are not parallel.

-- Math (ANOVA) approach Definitions

-- Main Effect of Factor A (1st IV): Overall difference among the levels of A

-- Main Effect of Factor B (2nd IV): Overall difference among the levels of B

-- Interaction of AxB: Differences among the levels of one Factor depend on the levels on the other Factor.

** Note what is missing: There is no concern that the MEs are Consistent.

** Note what doesn't change: The definition of the interaction is constant.

New Terms

To make the judgements required to define MEs and interactions by the math, I have to introduce some more language.

The first information is Marginal Means: Marginal Means are the means for one level of an independent variable averaged across all level of the other IV. Thus you have a Marginal Mean with A=1, which is the mean for everyone who experienced A at level one, regardless of whether they experienced B at 1 or 2. In addition to the Marginal Means for each level of both IVs, you also have a Total Mean, which is the average across the entire experiment.
 
IV  A .
IV B
A = 1
A = 2
marginal Means
    B = 1
M1,1 = 10
M2,1 = 15
MB=1= 12.5
    B = 2
M1,2 = 15
M2,2 = 20
MB=2= 17.5
marginal Means MA=1= 12.5 MA=2 = 17.5 MT = 15.0

Now considering the three things we look for:

ME of A: Is there a large difference (compared to w/i group variability) among the A marginal means?

ME of B: Is there a large difference (compared to w/i group variability) among the B marginal means?

Interaction: Deciding on the interaction you have to know if the two factors are additive. If additive, no interaction; if non-additive, interaction.

Additive means that you can predict the cell means based on the marginal means. Here's an easy way to do that: Look at 3 of the cell means. Cover the fourth cell mean. in the example above, cover M2,2 = 20. Now try to predict that based on the other 3 cell means. At Level B=1, going from A=1 to A=2 adds 5 to the cell mean. Thus we should add 5 to the cell mean for B=2, A=1 to get the cell mean for B=2, A=2. Turns out to work this time. You can predict and thus things are additive and there is no interaction.

-- When the Math and Graph do not agree

Generally, believe whichever says no.

If the graph makes it look like something is happening, but the math (ANOVA) says no, then believe the math. The ANOVA is testing not only to see if there is a difference, but that the difference is large compared to w/i group variability.

If the math says there is a main effect, but looking at the graph indicates that there is not a consistent main effect, then your main effect is an artifact of the interaction. (Note, in order for this to happen, there must and will be an interaction.) Artifact: something created. In this case, created by the interaction. That means it is created because the effects of one Factor go in different directions at different levels of the other Factor -- but that one of these is larger than the other and pulls the average (marginal means) apart in one direction. In this case, when you look at the marginal means, there is an overall difference, but if you look at the cells it is not consistent. The true definition of a main effect is a consistent overall difference, but the ANOVA only looks at the overall part. You, the researcher, have to be concerned that the main effect is consistent. You only get artifacts when you have an interaction.