- Two nominal measures or one ordinal: use Cramer’s V
- Two ordinal measures: Kendall’s Tau B if table is square and Kendall’s Tau C if rectangular
- Two interval or ratio: use correlation (r)
- Less than + or -
0.10: very weak
- + or -0.10 to
0.19: weak
- + or - 0.20 to
0.29: moderate
- + or - 0.30 or
above: strong
- Less than + or -
0.25: extremely weak
- + or -0.25 to
0.34: weak
- + or - 0.35 to
0.39: moderate
- + or - 0.40 or
larger: strong

**Chapter 12. Significance and Measures of Association**

I. Introduction

We already introduced these ideas in the chapter on statistics. So this is mostly a review. But now we will go into more detail, especially in computing and interpreting the most commonly used significance test for crosstabulations, the chi square statistic.

Suppose you have a relationship that you have tested. It is tested in some kind of crosstab, an analysis of varience, or a regression. The purpose of this unit is to learn how to answer two questions.

1) How strong is that relationship?

2) What is the probability that this relationship is not real, that is the result of drawing a bad sample from a population in which no relationship exists?

II. Statistics that measure the strength of relationships: measures of association

Evaluating We answer the first question by using statistics that are measures of
association. Micro Case has several measures of association available that are
fairly standard in statistics. As
in the past, what you use __depends on the level of measurement__ of
the variable in the hypothesis and the type of test used to view a relationship.
For __crosstabs with nominal measures__ we can use the PRE test (proportional
reduction in error test). We can also use the Lambda test, or the generally
preferred Cramer's
V test. All of these have in common that they range from 0 to 1, and the
closer to 1, the stronger the relationship.

When dealing with __two ordinal measures that are related in a crosstabs
(or at least an ordinal independent variable and a dependent variable that can
be interpreted as dichotomous)__,
the most appropriate measures of association are the gamma, Kendall tau
(tau-b for square tables and tau-c for non-square tables), or the less preferred Somer's
D. These tests range form -1 to +1, with the sign telling the direction
of the relationship. Minus means that as one increases the other decreases.
Plus means that as one goes up so does the other. The closer to +1 or -1
the stronger the relationship.

When dealing with __interval/ratio measures__, the most frequently
used measure of association is the Pearson correlation, designated as r.
This also ranges from +1 to -1. It tells you the extent to which the points
of the two variables form a straight line on a scatterplot. The sign gives
you the direction of the line.

Here are some quick guidelines on which measures of association to use:

The adjectives we use to describe the strength of a relationship are rather imprecise. What does the value of a measure of association have to be for us to consider it strong? Here are some guidelines. Be aware that not all would agree with these guidelines!

**For
Cramer’s V and Tau B and C: **

**
For Correlations:**

II. Significance Tests--the chi square test

Signficance tests are produced for each kind of association. What they
tell you is __whether the relationship you found in the sample is likely
to really exist in the general population or how likely it is to be an accident from sampling
error__. In social science we want the chances of accident to be low,
so we __conventionally insist that the significance be .05 or lower__.
That means that at worst no more than a 5% chance exists that we could
get a relationship like this in the sample if no relationship existed in
the general population from which the sample was drawn.

Just like in computing sample error, the sample must be such that every member of the population has an equal chance of being chosen. Like sample error, significance tests are very sensitive to sample size. Larger samples will are more likely to produce significant associations. Just like you need a large sample to correctly detect a winner in a close election, you need a large sample to detect a weak relationship. Using small samples, only the strongest associations will come out as significant. You might think of this as like needing a larger more powerful lens in a microscope to detect more tiny things.

The most common test for crosstabs is the **chi square test **
(pronounced "ki" with a long "i.").
While we will learn how to compute it by hand because sometimes you may have a
table you need to evaluate but do not have a statistical program to produce the
chi square. But mostly in the real world we
just let the computer calculate it. Other tests are too hard to compute
by hand, at least for this course.

Here is the formula for the chi square test, which is the sum over all cells of (Fe-Fo) squared divided by Fe.

where Fo are observed frequencies in the actual crosstab

Fe are the frequencies expected by chance (meaning that this is what the frequencies would be if there were no relationship between the two variables), which can be computed as follows:

After computing the chi square
statistic, you look up the value in the table
below and note the probability of the column in which it falls. The bigger
the chi square, the more significant the relationship in the sample. If
the chi square is at least as big as the value in the .05 (or 5%) column,
then one concludes that __less than a 5% chance exists that we could
find such a relationship in the sample when no relationship exists in
the general population__. We would then reject the null hypothesis. (Remember
that the null means no relationship exists.)

The row you use in the chi square table
is determined by the ** degrees of freedom** (df) with

df = (# rows-1)(# columns-1).

Here is the chi square table and instructions on how to use it.

Degrees of freedom |
Min chi sq for a sig level of .10 or 10% |
Min chi sq for a sig level of .05 or 5% |
Min chi sq for a sig level of .01 or 1% |

1 |
1.64 |
2.71 |
5.41 |

2 |
3.22 |
4.60 |
7.82 |

3 |
4.64 |
6.25 |
9.84 |

4 |
5.99 |
7.78 |
11.67 |

5 |
7.29 |
9.24 |
13.39 |

6 |
8.56 |
10.64 |
15.03 |

7 |
9.80 |
12.02 |
16.62 |

8 |
11.03 |
13.36 |
18.17 |

** **

**
Examples:**
Suppose you have a crosstabulation with 2 rows
and 3 columns. Suppose you use the formulas and compute the chi square and find it
to be 4.25. The degrees of freedom are (2-1)(3-1)
= (1)(2) = 2. Therefore you will use the second row of the table above.
A value of 4.25 lies between the .10 column and the .05
column. So the significance level is somewhere between .10 and .05,
or between 10 percent and 5 percent. Because in social science we __insist
that the significance level be at 5% or lower__, we cannot conclude that
what we saw in this particular crosstabulation
is statistically significant. The proper
conclusion to draw could be worded as follows. *This is not a significant
relationship because more than a 5 percent chance exists that this
relationship could be found in a sample when no relationship exists in the
population.*

Had the value of the chi square been
greater than 4.60, say 5.33, we would have concluded: *This is a significant
relationship because less than a 5 percent chance exists that this
relationship could be found in a sample when no relationship exists in the
population.*

To take the example to another possibility,
suppose that the chi square turns out to be really large, say 64.32. That is
entirely possible when the sample is relatively large and the percentage
shifts in the crosstabulation rows are also
large. Sometimes you get chi squares in the hundreds! We would then conclude: *This is a significant relationship because
less than a 1 percent chance exists that this relationship could be found in
a sample when no relationship exists in the population.*

I know all this sounds complicated, but in practice it is really quite
simple. An example or two should sort it out for you. We will do several of
these in
class as lab exercises.

*last updated on 11/8/2011*