Non-Parametric Tests

A non-parametric test is a hypothesis test that does not make any assumptions about the distribution of the samples.

The Mann-Whitney test, also known as the Wilcoxon rank sum test or the Wilcoxon-Mann-Whitney test, tests the hypothesis that two samples were drawn from the same distribution. The test relies only on the relative ranks of the observations in the combined sample. It does not rely on any properties of the distributions.

Definition

The null hypothesis ($H_0$) is that the samples were drawn from the same distribution. The test statistic is the U-statistic, calculated as:

$$U=min(U_1,U_2)$$

where $U_1$ and $U_2$ are the sums of the ranks for the two samples. The U-statistic follows a normal distribution for large samples.

Assumptions

The Mann-Whitney test assumes that the samples are randomly selected and independent. It does not assume any specific distribution for the data.

Applications

The Mann-Whitney test is commonly used in medical research, social sciences, and any field where comparing the central tendency of two independent samples is important.

The MannWhitneyTest class

The Mann-Whitney test is implemented by the MannWhitneyTest<T> class. It has three constructors. The first constructor takes no arguments. All test parameters must be provided by setting the properties of the MannWhitneyTest<T> object. The samples the test is to be applied to must be specified by setting the Sample1 and Sample2 properties. The second constructor takes two vector arguments that represent the samples the test is to be applied to.

The third constructor also takes two arguments. The first is once again a Vector<T> that contains the observations from the two samples combined. The second argument must be a CategoricalVector<T> with two levels that specifies which sample the observations in the first variable belong to.

The distribution of the Mann-Whitney statistic U can be computed exactly. For larger samples, an approximation in terms of the normal distribution can be used. If the observations don't have distinct ranks (i.e. there are ties), then the exact calculations are not available. The exact test is used by default if there are no ties, and if the product of the two sample sizes is less than or equal to 1600. You can override the default behavior by setting the Exactness property. Note that the 'exact' calculation gives incorrect results if ties are present.

Example

In this example, we examine whether the onset of type II diabetes is different for males and females. The onset in our sample is as follows:

• Males: 19 22 16 29 24

• Females: 20 11 17 12

The code below performs the exact Mann-Whitney test on this sample:

var males = Vector.Create(19, 22, 16, 29, 24);
var females = Vector.Create(20, 11, 17, 12);
var mwTest = new MannWhitneyTest<int>(males, females);
Console.WriteLine("Test statistic: {0:F4}", mwTest.Statistic);
Console.WriteLine("P-value:        {0:F4}", mwTest.PValue);
Console.WriteLine("Reject null hypothesis? {0}",
    mwTest.Reject() ? "yes" : "no");

Dim males = Vector.Create(19, 22, 16, 29, 24)
Dim females = Vector.Create(20, 11, 17, 12)
Dim mwTest = New MannWhitneyTest(Of Integer)(males, females)
Console.WriteLine("Test statistic: {0:F4}", mwTest.Statistic)
Console.WriteLine("P-value:        {0:F4}", mwTest.PValue)
Console.WriteLine("Reject null hypothesis? {0}",
    IIf(mwTest.Reject(), "yes", "no"))

No code example is currently available or this language may not be supported.

let males = Vector.Create(19, 22, 16, 29, 24)
let females = Vector.Create(20, 11, 17, 12)
let mwTest = MannWhitneyTest<int>(males, females)
printfn "Test statistic: %.4f" mwTest.Statistic
printfn "P-value:        %.4f" mwTest.PValue
printfn "Reject null hypothesis? %s"
    (if mwTest.Reject() then "yes" else "no")

The value of the U-statistic is 3, giving a two-tailed p-value of 0.1111. As a result, the hypothesis that on average, type II diabetes starts around the same age in males and females is not rejected at the 0.05 significance level.

The Kruskal-Wallis test, also known as the Kruskal-Wallis H test, is a generalization of the Mann-Whitney test to more than two samples. It tests the hypothesis that the supplied samples were all drawn from the same distribution.

Definition

The null hypothesis ($H_0$) is that the samples were drawn from the same distribution. The test statistic is the H-statistic, calculated as:

$$H=\frac{12}{N(N+1)}\sum _{i=1}^k\frac{R_i^2}{n_i}-3(N+1)$$

where $N$ is the total number of observations, $k$ is the number of samples, $R_i$ is the sum of ranks for the $i$-th sample, and $n_i$ is the size of the $i$-th sample. The H-statistic follows a chi-square distribution with $k-1$ degrees of freedom.

Assumptions

The Kruskal-Wallis test assumes that the samples are randomly selected and independent. It does not assume any specific distribution for the data.

Applications

The Kruskal-Wallis test is commonly used in medical research, social sciences, and any field where comparing the central tendency of more than two independent samples is important.

The KruskalWallisTest class

The Kruskal-Wallis test is implemented by the KruskalWallisTest class. It has three constructors. The first constructor takes no arguments. All test parameters must be provided by setting the properties of the KruskalWallisTest object. The samples the test is to be applied to must be specified through the Samples property.

The second constructor takes an array of Vector<T> objects that represent the samples the test is to be applied to. The third constructor also takes two arguments. The first is once again a Vector<T> that contains the observations from all the samples combined. The second argument must be a CategoricalVector<T> that specifies which sample the observations in the first variable belong to.

Although the distribution of the Kruskal-Wallis statistic can be computed exactly, it is impractical except for very small values. We use an approximation in terms of the beta distribution which is more accurate than the more commonly used approximation in terms of a chi-square distribution.

Example

In this example, we examine whether different teaching methods have an effect on a standardized exam score. The exam scores for students that were taught using each of the three methods were as follows:

• Method 1: 94, 87, 90, 74, 86, 97

• Method 2: 82, 85, 79, 84, 61, 72, 80

• Method 3: 89, 68, 72, 76, 69, 65

The code below performs the Kruskal-Wallis test on these samples:

var method1 = Vector.Create<double>(94, 87, 90, 74, 86, 97);
var method2 = Vector.Create<double>(82, 85, 79, 84, 61, 72, 80);
var method3 = Vector.Create<double>(89, 68, 72, 76, 69, 65);
var kwTest = new KruskalWallisTest(method1, method2, method3);
Console.WriteLine("Test statistic: {0:F4}", kwTest.Statistic);
Console.WriteLine("P-value:        {0:F4}", kwTest.PValue);
Console.WriteLine("Reject null hypothesis? {0}",
    kwTest.Reject() ? "yes" : "no");

Dim method1 = Vector.Create(Of Double)(94, 87, 90, 74, 86, 97)
Dim method2 = Vector.Create(Of Double)(82, 85, 79, 84, 61, 72, 80)
Dim method3 = Vector.Create(Of Double)(89, 68, 72, 76, 69, 65)
Dim kwTest = New KruskalWallisTest(method1, method2, method3)
Console.WriteLine("Test statistic: {0:F4}", kwTest.Statistic)
Console.WriteLine("P-value:        {0:F4}", kwTest.PValue)
Console.WriteLine("Reject null hypothesis? {0}",
    IIf(kwTest.Reject(), "yes", "no"))

No code example is currently available or this language may not be supported.

let method1 = Vector.Create(94., 87., 90., 74., 86., 97.)
let method2 = Vector.Create(82., 85., 79., 84., 61., 72., 80.)
let method3 = Vector.Create(89., 68., 72., 76., 69., 65.)
let kwTest = KruskalWallisTest(method1, method2, method3)
printfn "Test statistic: %.4f" kwTest.Statistic
printfn "P-value:        %.4f" kwTest.PValue
printfn "Reject null hypothesis? %s"
    (if kwTest.Reject() then "yes" else "no")

The value of the Kruskal-Wallis statistic is 7.5023, giving a two-tailed p-value of 0.0235. As a result, the hypothesis that on average, the teaching method has no effect on exam scores is rejected at the 0.05 significance level.

The runs test, also called the Wald-Wolfowitz test, is a test of randomness. It compares the lengths of runs of the same value in a sample to what would be expected in a random sample.

Definition

The null hypothesis ($H_0$) is that the samples were drawn randomly. The test statistic is the Z-statistic, calculated as:

$$Z=\frac{R-{\mu }_R}{{\sigma }_R}$$

where $R$ is the number of runs, ${\mu }_R$ is the expected number of runs, and ${\sigma }_R$ is the standard deviation of the number of runs. The Z-statistic follows a standard normal distribution.

Assumptions

The runs test assumes that the sample is randomly selected. It does not assume any specific distribution for the data.

Applications

The runs test is commonly used in quality control, environmental studies, and any field where testing for randomness in a sequence of observations is important.

The RunsTest class

The runs test is implemented by the RunsTest<T> class. It has three constructors. The first constructor takes no arguments. All test parameters must be provided by setting the properties of the RunsTest<T> object. The samples the test is to be applied to must be specified by setting the Sample1 and Sample2 properties. The second constructor takes two vector arguments that represent the samples the test is to be applied to.

The third constructor also takes two arguments. The first is once again a Vector<T> that contains the observations from the two samples combined. The second argument must be a CategoricalVector<T> with two levels that specifies which sample the observations in the first variable belong to.

Example

In this example, we investigate a series of 23 estimates for the density of the Earth by Henry Cavendish. We want to know if there is a correlation between successive observations, which would mean not all measurements were made independently.

Console.WriteLine("\nRuns Test\n");
var densities = Vector.Create(5.36, 5.29, 5.58, 5.65, 5.57,
    5.53, 5.62, 5.29, 5.44, 5.34, 5.79, 5.10, 5.27, 5.39,
    5.42, 5.47, 5.63, 5.34, 5.46, 5.30, 5.75, 5.68, 5.85);
var rt = new RunsTest<double>(densities, densities.Median());
Console.WriteLine("Test statistic: {0:F4}", rt.Statistic);
Console.WriteLine("P-value:        {0:F4}", rt.PValue);
Console.WriteLine("Reject null hypothesis? {0}", 
    rt.Reject() ? "yes" : "no");

Console.WriteLine("\nRuns Test\n")
Dim densities = Vector.Create(5.36, 5.29, 5.58, 5.65, 5.57,
    5.53, 5.62, 5.29, 5.44, 5.34, 5.79, 5.1, 5.27, 5.39,
    5.42, 5.47, 5.63, 5.34, 5.46, 5.3, 5.75, 5.68, 5.85)
Dim rt = New RunsTest(Of Double)(densities, densities.Median())
Console.WriteLine("Test statistic: {0:F4}", rt.Statistic)
Console.WriteLine("P-value:        {0:F4}", rt.PValue)
Console.WriteLine("Reject null hypothesis? {0}",
    IIf(rt.Reject(), "yes", "no"))

No code example is currently available or this language may not be supported.

let densities = 
    Vector.Create(
        5.36, 5.29, 5.58, 5.65, 5.57, 5.53, 5.62, 5.29, 
        5.44, 5.34, 5.79, 5.10, 5.27, 5.39, 5.42, 5.47, 
        5.63, 5.34, 5.46, 5.30, 5.75, 5.68, 5.85)
let rt = RunsTest<double>(densities, densities.Median())
printfn "Test statistic: %.4f" rt.Statistic
printfn "P-value:        %.4f" rt.PValue
printfn "Reject null hypothesis? %s" 
    (if rt.Reject() then "yes" else "no")

The value of the test statistic is -1.7477, corresponding to a p-value of 0.08051. As a result, the hypothesis that the measurements are uncorrelated is not rejected at the 0.05 significance level.

• "Nonparametric Statistical Methods" by Myles Hollander, Douglas A. Wolfe, and Eric Chicken.

• "Practical Nonparametric Statistics" by W.J. Conover.

• "Nonparametric Statistics for the Behavioral Sciences" by Sidney Siegel and N. John Castellan Jr.

• "The Mann-Whitney U Test" in "Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach" by Gregory W. Corder and Dale I. Foreman.

• "The Kruskal-Wallis Test" in "Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach" by Gregory W. Corder and Dale I. Foreman.

• "The Runs Test" in "Nonparametric Statistical Inference" by Jean Dickinson Gibbons and Subhabrata Chakraborti.

Other Resources