Generic data analysis procedures

Following these guidelines should get you started on most basic analyses:

• Before the experiment:
  • Use G*Power to determine required sample size. Power analysis parameters are generally set at the standards of .05 for the significance criterion (the probability of making a Type 1 / false positive error) and .8 for the power (the probability of avoiding a Type 2 / false negative error, or wrongly failing to reject the null hypothesis).  As statistical power increases, the probability of making a type 2 error decreases.
  • Set up your data entry spreadsheet (e.g., with Excel) to make it clear exactly what data will be collected. Create columns for demographic information (age, species, etc.), IVs (groups), and DVs (e.g., seizure duration, distance moved, # trials correct, etc.). Populate the pre-experiment sheet with “anticipated results” data.
  • Set up your graph file (e.g., with Prism) to make it clear exactly what graphs will be made… Using your pre-experiment anticipated results (“fake”) data, create bar graphs for each DV (with scatterplot overlaid) for each main and interaction effect with ALL data points (e.g., diet x drug, etc.) . Don’t worry about 3-way interactions (initially).

• After the data are collected:
  • Insert collected data into the pre-experiment graphs and identify outliers (generally defined as 2 SDs away from the group mean).
    • • Create same graphs with outliers excluded.
  • Test for normality of the data distribution (probably with Kolmogorov–Smirnov test) for all effects (overall AND within each group).
  • Check for other data violations:
    • • Homoscedasticity (homogeneity of variance) can be verified for each ANOVA test using Bartlett’s-Cochran test
      • Apply any appropriate corrections (e.g., sphericity violations indicated by Mauchly’s test can be corrected by the Greenhouse-Geisser correction).
      • Do any data distribution transformations required for the stats tests.
  • Run appropriate tests (ANOVAs/t-tests/non-parametrics) for each effect with post-hoc tests (probably Turkey or Scheffe) for significant effects.
  • Determine effect sizes (e.g., partial omega squared [ω̂p2] / Glass’s delta [Δ] values).
  • Generate a correlation matrix between all DVs and graph scatterplots of appropriate pairings.
  • Write up and attach Results section.