The Data Analysis Plan
I. Check the data
II. Summarize the data
A. Descriptive Statistics
1) Purpose
2) Measures of Central Tendency (Mean, Median, Mode)
3) Measures of Variability (Range, variance, standard deviation)
B. Graphical Summaries
III. Confirm what the data reveal: Inferential statistics
A. Purpose
B. Five Critical Terms
1) null hypothesis
2) alternative hypothesis
3) Type I error (alpha)
4) Type II error (beta)
5) Power
C. Three steps to a statistical decision
1) Assume the null hypothesis
2) ‘Calculate the probability of results as or more extreme than those obtained under the null hypothesis
3) decide whether you are willing to accept this risk of error. Decide to reject or fail to reject (retain) the null hypothesis
I. Checking the data
A. Checking for outliers and errors Do the numbers make sense? For example, an imported data file from your survey should have a column for each variable (one for responses to each Likert Scale item and then one for each demographic variable). Each row represents the data from one participant:
Item1 
Item2 
Item3 
… 
Item n 
Sex 
Religious fervor 
Political affiliation 
3 
2 
1 

3 
0 
1 
1 
1 
4 
1 

5 
0 
3 
4 
5 
4 
1 

2 
0 
2 
3 
3 
1 
1 

4 
1 
2 
3 
3 
5 
1 

9 
1 
3 
1 
5 
2 
1 

3 
0 
2 
7 
1 
4 
1 

2 
5 
1 
1 








Looking at these data, I would notice several peculiarities. The
columns with likert responses should never contain
numbers outside the range of the scale. For
example, I would not expect to see the “9” under item n. I would also be aware of the uniformity of
the responses to item3. This is not
necessarily an error, but may indicate a problem in the particular item. Within the demographic section, a “5” under
sex leads me to suspect a coding error, as would a “7” under political
affiliation. These are all most likely
errors on the part of the participant or in coding the data.
Another type of
anomaly is called an “outlier”. These
are extreme values on the DV, but are generally not coding errors but rather
true tendencies of the participant to give extreme scores on occasion. The participant may not have followed
instructions, may be distracted, or may truly be an “unusual” participant. For example, suppose your DV were a reaction
time measure. You could have a
participant who, overall, gives extremely fast (or slow) RT. In the Hadden
study, for example, there was one family choosen for
the study that was extremely hard to contact.
Rather than a few days between observations, this family often had weeks
or months between observations. The data
from this family were unusual and so she dropped that family from her analysis.
You can use a
histogram or a stem and leaf plot to determine the occurrence of outliers. This will also allow you to look for unusual
response distributions, such as in item3 above.
You will see a “skewed” response distribution.
Whatever you do
about your outliers or mistakes you must formalize your decision rule and
report this in the research report.
Indicate how much data was omitted and why. You should not drop data with
out informing the reader.