12. Non-Parametric Statistical Tests

Making sense of it all

Author

Affiliation

Published

January 1, 2021

In Chapters 7, 8, 9, and 10 we have seen t-tests, ANOVAs, simple linear regressions, and correlations. These tests may be substituted with non-parametric tests if our assumptions about our data fail us. The most commonly encountered non-parametric methods include the following:

Wilcoxon rank-sum test The test is used when the two samples being compared are related, meaning that each observation in one sample is paired with a corresponding observation in the other sample. The test is designed to detect whether there is a difference between the paired observations. Specifically, the Wilcoxon signed-rank test ranks the absolute differences between the pairs of observations, and then compares the sum of the ranks for positive differences to the sum of the ranks for negative differences. The test produces a p-value indicating the probability of observing such a difference by chance, assuming the null hypothesis that there is no difference between the paired observations. Use the Wilcoxon test as a non-parametric substitute for a paired sample t-test. See wilcox.test().
Mann-Whitney $U$ test This test is used when the two samples being compared are independent, meaning that there is no pairing between observations in the two samples. The test is designed to detect whether there is a difference between the two groups based on the ranks of the observations. Specifically, the Mann-Whitney $U$ test ranks all observations from both samples, combines the ranks across the two samples, and calculates a test statistic ($U$) that indicates whether one sample tends to have higher ranks than the other sample. The test produces a p-value indicating the probability of observing such a difference by chance, assuming the $H_0$ that there is no difference between the two groups. Use this test in stead of a one- or two-sample t-test when assumptions of normality or homoscedasticity are not met. See wilcox.test().
Kruskal-Wallis test The Kruskal-Wallis test is a non-parametric statistical test used to compare three or more independent groups on a continuous outcome variable. The test is designed to detect whether there is a difference in the medians of the groups. The Kruskal-Wallis test works by ranking all the observations from all the groups, then calculating a test statistic ($H$) that measures the degree of difference in the ranked values between the groups. The test produces a p-value indicating the probability of observing such a difference by chance, assuming the $H_0$ that there is no difference in the medians of the groups. The Kruskal-Wallis test is often used as a non-parametric alternative to the one-way ANOVA. See kruskal.test().
Friedman test This test is a non-parametric statistical test used to compare three or more related (i.e. not-independent) groups on a continuous outcome variable. The test is designed to detect whether there is a difference in the medians of the groups. The Friedman test works by ranking all the observations within each group, then calculating a test statistic ($\chi^2$) that measures the degree of difference in the ranked values between the groups. The test produces a $p$-value indicating the probability of observing such a difference by chance, assuming the $H_0$ that there is no difference in the medians of the groups. The Friedman test is often used as a non-parametric alternative to the repeated measures ANOVA. You can use the friedman_._test() in the rstatix package or the friedman.test() in Base R.

Tables 1 and 2 summarise common parametric and non-parametric statistical tests, along with a brief explanation of each test and the most common R function used to perform the test. Non-parametric tests are robust alternatives to parametric tests when the assumptions of the parametric test are not met. Also provided is additional information on the nature of the independent (IV) and dependent variables (DV) for each test.

Table 1: When our data are normal with equal variances across groups, choose the suitable parametric test

Statistical Test	Explanation	Variables	R Function	Non-Parametric Substitute
Parametric Tests
Paired-sample t-test	Tests if the difference in means between paired samples is significantly different from zero. Assumes normality and equal variances.	Continuous (DV)	`t.test(..., var.equal = TRUE)`	Wilcoxon signed-rank test
Student’s t-test	Tests if the means of two independent groups are significantly different. Assumes normality and equal variances.	Continuous (DV) and categorical (IV)	`t.test(..., var.equal = TRUE)`	Mann-Whitney U test
Welch’s t-test (unequal variances)	Use this test when data are normal but variances differ between the two groups. It can be used for paired- and two-sample t-tests.	Continuous (DV) and categorical (IV)	`t.test()`	Mann-Whitney U test or Wilcoxon signed-rank test
ANOVA (one-way ANOVA only; ANOVAs with interactions do not have non-parametric tests)	Tests if the means of three or more independent groups are significantly different. Assumes normality, equal variances, and independence.	Continuous (DV) and categorical (IV)	`aov()`	Kruskal-Wallis test
ANOVA with Welch’s approximation of variances	Tests if the means of three or more independent groups are significantly different. Assumes normality but variances may differ.	Continuous (DV) and categorical (IV)	`oneway.test()`	Kruskal-Wallis test
Regression Analysis	Models the relationship between two continuous variables. Assumes linearity, normality, and equal variances of errors.	Continuous (DV) and continuous (IV)	`lm()`	N/A
Pearson Correlation	Measures the strength and direction of the linear relationship between two continuous variables. Assumes normality and linearity.	Continuous (DV) and continuous (IV)	`cor.test()`	Spearman’s $\rho$ or Kendall’s $\tau$ rank correlation

Table 2: Should the data not be normal and/or are heteroscedastic, substitute the parametric test with a non-parametric option.

Statistical Test	Explanation	Variables	R Function	Parametric Equivalent
Non-Parametric Tests
Wilcoxon signed-rank test	Tests if the medians of two related samples are significantly different. Does not assume normality.	Continuous (DV)	`wilcox.test()`	Paired-sample t-test
Mann-Whitney U test	Tests if the medians of two independent groups are significantly different. Does not assume normality or equal variances.	Continuous (DV) and categorical (IV)	`wilcox.test()`	Student’s t-test
Kruskal-Wallis test	Tests if the medians of three or more independent groups are significantly different. Does not assume normality or equal variances.	Continuous (DV) and categorical (IV)	`kruskal.test()`	ANOVA, or ANOVA with Welch’s approximation of variances
Friedman test	Tests if the medians of three or more related samples are significantly different. Use when assumption of independence of data cannot be accepted and data might therefore be non-normal (such as repeated measures or unreplicated full-block design).	Continuous (DV) and categorical (IV)	`friedman.test()`	Repeated measures ANOVA
Spearman’s rank correlation	Measures the strength and direction of the monotonic relationship between two continuous variables. Does not assume normality or linearity.	Continuous (DV) and continuous (IV)	`cor.test(method = "spearman")`	Pearson correlation

Reuse

CC BY-NC-SA 4.0

Citation

BibTeX citation:

@online{j._smit2021,
  author = {J. Smit, Albertus},
  title = {12. {Non-Parametric} {Statistical} {Tests}},
  date = {2021-01-01},
  url = {http://tangledbank.netlify.app/BCB744/basic_stats/12-glance.html},
  langid = {en}
}

For attribution, please cite this work as:

J. Smit A (2021) 12. Non-Parametric Statistical Tests. http://tangledbank.netlify.app/BCB744/basic_stats/12-glance.html.

--- date: "2021-01-01" title: "12. Non-Parametric Statistical Tests" subtitle: "Making sense of it all" --- In Chapters 7, 8, 9, and 10 we have seen [*t*-tests](07-t_tests.qmd), [ANOVAs](08-anova.qmd), [simple linear regressions](09-regressions.qmd), and [correlations](10-correlations.qmd). These tests may be substituted with non-parametric tests if our assumptions about our data fail us. The most commonly encountered non-parametric methods include the following: - **Wilcoxon rank-sum test** The test is used when the two samples being compared are related, meaning that each observation in one sample is paired with a corresponding observation in the other sample. The test is designed to detect whether there is a difference between the paired observations. Specifically, the Wilcoxon signed-rank test ranks the absolute differences between the pairs of observations, and then compares the sum of the ranks for positive differences to the sum of the ranks for negative differences. The test produces a *p*-value indicating the probability of observing such a difference by chance, assuming the null hypothesis that there is no difference between the paired observations. *Use the Wilcoxon test as a non-parametric substitute for a paired sample *t*-test*. See `wilcox.test()`. - **Mann-Whitney $U$ test** This test is used when the two samples being compared are independent, meaning that there is no pairing between observations in the two samples. The test is designed to detect whether there is a difference between the two groups based on the ranks of the observations. Specifically, the Mann-Whitney $U$ test ranks all observations from both samples, combines the ranks across the two samples, and calculates a test statistic ($U$) that indicates whether one sample tends to have higher ranks than the other sample. The test produces a *p*-value indicating the probability of observing such a difference by chance, assuming the $H_0$ that there is no difference between the two groups. *Use this test in stead of a one- or two-sample *t*-test when assumptions of normality or homoscedasticity are not met*. See `wilcox.test()`. - **Kruskal-Wallis test** The Kruskal-Wallis test is a non-parametric statistical test used to compare three or more independent groups on a continuous outcome variable. The test is designed to detect whether there is a difference in the medians of the groups. The Kruskal-Wallis test works by ranking all the observations from all the groups, then calculating a test statistic ($H$) that measures the degree of difference in the ranked values between the groups. The test produces a *p*-value indicating the probability of observing such a difference by chance, assuming the $H_0$ that there is no difference in the medians of the groups. *The Kruskal-Wallis test is often used as a non-parametric alternative to the one-way ANOVA*. See `kruskal.test()`. - **Friedman test** This test is a non-parametric statistical test used to compare three or more related (i.e. not-independent) groups on a continuous outcome variable. The test is designed to detect whether there is a difference in the medians of the groups. The Friedman test works by ranking all the observations within each group, then calculating a test statistic ($\chi^2$) that measures the degree of difference in the ranked values between the groups. The test produces a $p$-value indicating the probability of observing such a difference by chance, assuming the $H_0$ that there is no difference in the medians of the groups. *The Friedman test is often used as a non-parametric alternative to the repeated measures ANOVA*. You can use the `friedman_._test()` in the **rstatix** package or the `friedman.test()` in Base R. Tables 1 and 2 summarise common parametric and non-parametric statistical tests, along with a brief explanation of each test and the most common R function used to perform the test. Non-parametric tests are robust alternatives to parametric tests when the assumptions of the parametric test are not met. Also provided is additional information on the nature of the independent (IV) and dependent variables (DV) for each test. :::{.column-page-inset-right} **Table 1:** When our data are normal with equal variances across groups, choose the suitable parametric test | Statistical Test | Explanation | Variables | R Function | Non-Parametric Substitute | | --- | --- | --- | --- | --- | | **Parametric Tests** | | | | | | Paired-sample *t*-test | Tests if the difference in means between paired samples is significantly different from zero. Assumes normality and equal variances. | Continuous (DV) | `t.test(..., var.equal = TRUE)` | Wilcoxon signed-rank test | | Student's *t*-test | Tests if the means of two independent groups are significantly different. Assumes normality and equal variances. | Continuous (DV) and categorical (IV) | `t.test(..., var.equal = TRUE)` | Mann-Whitney U test | | Welch's *t*-test (unequal variances) | Use this test when data are normal but variances differ between the two groups. It can be used for paired- and two-sample *t*-tests. | Continuous (DV) and categorical (IV) | `t.test()` | Mann-Whitney U test or Wilcoxon signed-rank test | | ANOVA (one-way ANOVA only; ANOVAs with interactions do not have non-parametric tests) | Tests if the means of three or more independent groups are significantly different. Assumes normality, equal variances, and independence. | Continuous (DV) and categorical (IV) | `aov()` | Kruskal-Wallis test | | ANOVA with Welch's approximation of variances | Tests if the means of three or more independent groups are significantly different. Assumes normality but variances may differ. | Continuous (DV) and categorical (IV) | `oneway.test()` | Kruskal-Wallis test | | Regression Analysis | Models the relationship between two continuous variables. Assumes linearity, normality, and equal variances of errors. | Continuous (DV) and continuous (IV) | `lm()` | N/A | | Pearson Correlation | Measures the strength and direction of the linear relationship between two continuous variables. Assumes normality and linearity. | Continuous (DV) and continuous (IV) | `cor.test()` | Spearman's $\rho$ or Kendall's $\tau$ rank correlation | ::: **Table 2:** Should the data not be normal and/or are heteroscedastic, substitute the parametric test with a non-parametric option. :::{.column-page-inset-right} | Statistical Test | Explanation | Variables | R Function | Parametric Equivalent | | --- | --- | --- | --- | --- | | **Non-Parametric Tests** | | | | | | Wilcoxon signed-rank test | Tests if the medians of two related samples are significantly different. Does not assume normality. | Continuous (DV) | `wilcox.test()` | Paired-sample *t*-test | | Mann-Whitney U test | Tests if the medians of two independent groups are significantly different. Does not assume normality or equal variances. | Continuous (DV) and categorical (IV) | `wilcox.test()` | Student's *t*-test | | Kruskal-Wallis test | Tests if the medians of three or more independent groups are significantly different. Does not assume normality or equal variances. | Continuous (DV) and categorical (IV) | `kruskal.test()` | ANOVA, or ANOVA with Welch's approximation of variances | | Friedman test | Tests if the medians of three or more related samples are significantly different. Use when assumption of independence of data cannot be accepted and data might therefore be non-normal (such as repeated measures or unreplicated full-block design). | Continuous (DV) and categorical (IV) | `friedman.test()` | Repeated measures ANOVA | | Spearman's rank correlation | Measures the strength and direction of the monotonic relationship between two continuous variables. Does not assume normality or linearity. | Continuous (DV) and continuous (IV) | `cor.test(method = "spearman")` | Pearson correlation | :::