In short
The right test depends on four things: the type of outcome (continuous, categorical or time to event), how many groups you compare, whether measurements are paired or independent, and whether continuous data are roughly normally distributed. Decide the test when you plan the study, not after you see the results.
Quick guide
| Your question | Normally distributed data | Not normal, or ranks |
|---|---|---|
| Compare 2 independent groups | Unpaired t-test (Welch if variances differ) | Mann–Whitney U |
| Compare 2 paired measurements | Paired t-test | Wilcoxon signed-rank |
| Compare 3 or more independent groups | One-way ANOVA with post-hoc test | Kruskal–Wallis with Dunn's test |
| Compare 3 or more repeated measurements | Repeated-measures ANOVA | Friedman test |
| Two factors, such as treatment and time | Two-way ANOVA (repeated measures if needed) | Consider transformation or a mixed model |
| Relationship between two variables | Pearson correlation or linear regression | Spearman correlation |
| Dose–response | Nonlinear regression to estimate EC50 or IC50 | |
Categorical and time-to-event outcomes
- Proportions in 2 or more groups: chi-square test, or Fisher's exact test when expected counts are small.
- Paired yes/no outcomes: McNemar's test.
- Time to an event: Kaplan–Meier curves with the log-rank test, and Cox regression for adjusted comparisons.
Choosing the post-hoc test after ANOVA
- All groups against one control group: Dunnett's test.
- Every group against every other group: Tukey's test.
- A few planned comparisons only: Bonferroni or Šídák correction.
Common mistakes
- Running several t-tests against a control instead of ANOVA with Dunnett's test.
- Treating repeated measurements on the same animal as independent.
- Relying only on a normality test with very small groups; consider the design and use a transformation where justified.
- Reporting mean ± SEM where the spread of the data (SD) is what readers need.
- Reporting only p-values; add the effect size and its 95% confidence interval.
- Changing the test after seeing the result to reach significance.
Frequently asked questions
When should I use a non-parametric test?
When a continuous outcome is clearly not normal and cannot be sensibly transformed, or when the data are ranks or scores.
Should I report SD or SEM?
Report SD to describe the variability of your data. SEM describes the precision of the mean and is better replaced by a 95% confidence interval.
Which post-hoc test compares all groups with a control?
Dunnett's test.
Related
Need help with your project?
Book a free 20-minute consultation. We reply within 1 working day.