Non-Parametric Tests
A non-parametric test is a hypothesis test that does not make any assumptions about the distribution of the samples.
The Mann-Whitney Test
The Mann-Whitney test, also known as the Wilcoxon rank sum test or the Wilcoxon-Mann-Whitney test, tests the hypothesis that two samples were drawn from the same distribution. The test relies only on the relative ranks of the observations in the combined sample. It does not rely on any properties of the distributions.
The null hypothesis is that the samples were drawn from the same distribution. Location is the dominant factor in the comparison, but if the two distributions have different shapes, this may also affect the result. The alternative hypothesis for the two-tailed test (the default) is that the two samples were drawn from different distributions.
For the one-tailed test, the alternative hypothesis is roughly that the population from which the first sample was drawn has a median that is either less than (lower tailed) or greater than (upper tailed) the median of the population from which the second sample was drawn.
The Mann-Whitney test is implemented by the MannWhitneyTest<T> class. It has three constructors. The first constructor takes no arguments. All test parameters must be provided by setting the properties of the MannWhitneyTest<T> object. The samples the test is to be applied to must be specified by setting the Sample1 and Sample2 properties. The second constructor takes two vector arguments objects that represent the samples the test is to be applied to.
The third constructor also takes two arguments. The first is once again a Vector<T> that contains the observations from the two samples combined. The second argument must be a CategoricalVector<T> with two levels that specifies which sample the observations in the first variable belong to.
The distribution of the Mann-Whitney statistic U can be computed exactly. For larger samples, an approximation in terms of the normal distribution can be used. If the observations don't have distinct ranks (i.e. there are ties), then the exact calculations are not available. The exact test is used by default if there are no ties, and if the product of the two sample sizes is less than or equal to 1600. You can override the default behaviour by setting the Exactness property. Note that the 'exact' calculation gives incorrect results if ties are present.
Example
In this example, we examine whether the onset of type II diabetes is different for males and females. The onset in our sample is as follows:
Males: 19 22 16 29 24
Females: 20 11 17 12
The code below performs the exact Mann-Whitney test on this sample:
var males = Vector.Create(19, 22, 16, 29, 24);
var females = Vector.Create(20, 11, 17, 12);
var mwTest = new MannWhitneyTest<int>(males, females);
Console.WriteLine("Test statistic: {0:F4}", mwTest.Statistic);
Console.WriteLine("P-value: {0:F4}", mwTest.PValue);
Console.WriteLine("Reject null hypothesis? {0}",
mwTest.Reject() ? "yes" : "no");The value of the U-statistic is 3, giving a two-tailed p-value of 0.1111. As a result, the hypothesis that on average, type II diabetes starts around the same age in males and females is not rejected at the 0.05 significance level.
The Kruskal-Wallis Test
The Kruskal-Wallis test is a generalization of the Mann-Whitney test to more than two samples. It tests the hypothesis that the supplied samples were all drawn from the same distribution. The test relies only on the relative ranks of the observations in the combined sample. It does not rely on any properties of the distributions.
The null hypothesis is that the samples were drawn from the same distribution. Location is the dominant factor in the comparison, but if the distributions have different shapes, this may also affect the result. The alternative hypothesis for the two-tailed test is that the samples were drawn from different distributions.
The Kruskal-Wallis test is implemented by the KruskalWallisTest class. It has three constructors. The first constructor takes no arguments. All test parameters must be provided by setting the properties of the KruskalWallisTest object. The samples the test is to be applied to must be specified through the Samples property.
The second constructor takes an array of Vector<T> objects that represent the samples the test is to be applied to. The third constructor also takes two arguments. The first is once again a Vector<T> that contains the observations from all the samples combined. The second argument must be a CategoricalVector<T> that specifies which sample the observations in the first variable belong to.
Although the distribution of the Kruskal-Wallis statistic can be computed exactly, it is impractical except for very small values. We use an approximation in terms of the beta distribution which is more accurate than the more commonly used approximation in terms of a Chi-square distribution.
Example
In this example, we examine whether different teaching methods have an effect on a standardized exam score. The exam scores for students that were taught using each of the three methods were as follows:
Method 1: 94, 87, 90, 74, 86, 97
Method 2: 82, 85, 79, 84, 61, 72, 80
Method 3: 89, 68, 72, 76, 69, 65
The code below performs the Kruskal-Wallis test on these samples:
var method1 = Vector.Create<double>(94, 87, 90, 74, 86, 97);
var method2 = Vector.Create<double>(82, 85, 79, 84, 61, 72, 80);
var method3 = Vector.Create<double>(89, 68, 72, 76, 69, 65);
var kwTest = new KruskalWallisTest(method1, method2, method3);
Console.WriteLine("Test statistic: {0:F4}", kwTest.Statistic);
Console.WriteLine("P-value: {0:F4}", kwTest.PValue);
Console.WriteLine("Reject null hypothesis? {0}",
kwTest.Reject() ? "yes" : "no");The value of the Kruskal-Wallis statistic is 7.5023, giving a two-tailed p-value of 0.0235. As a result, the hypothesis that on average, the teaching method has no effect on exam scores is rejected at the 0.05 significance level.
The runs Test
The runs test (also called Wald–Wolfowitz test) is a test of randomness. It compares the lengths of runs of the same value in a sample to what would be expected in a random sample. In numerical data, it uses the runs of values that are above or below a cut point. The test relies only on the sequence of runs of the same value. It does not rely on any properties of the distributions.
The null hypothesis is that the samples were drawn randomly. The alternative hypothesis is that the samples were not drawn randomly. For a one-tailed test, the alternative hypothesis is that the samples tend to occur in groups (lower tailed) or that the samples tend to alternate (upper tailed).
The runs test is implemented by the RunsTest<T> class. It has three constructors. The first constructor takes no arguments. All test parameters must be provided by setting the properties of the MannWhitneyTest<T> object. The samples the test is to be applied to must be specified by setting the Sample1 and Sample2 properties. The second constructor takes two vector arguments objects that represent the samples the test is to be applied to.
The third constructor also takes two arguments. The first is once again a Vector<T> that contains the observations from the two samples combined. The second argument must be a CategoricalVector<T> with two levels that specifies which sample the observations in the first variable belong to.
Example
In this example, we investigate a series of 23 estimates for the density of the Earth by Henry Cavendish. We want to know if there is a correlation between successive observations, which would mean not all measurements were made independently.
var males = Vector.Create(19, 22, 16, 29, 24);
var females = Vector.Create(20, 11, 17, 12);
var mwTest = new MannWhitneyTest<int>(males, females);
Console.WriteLine("Test statistic: {0:F4}", mwTest.Statistic);
Console.WriteLine("P-value: {0:F4}", mwTest.PValue);
Console.WriteLine("Reject null hypothesis? {0}",
mwTest.Reject() ? "yes" : "no");The value of the test statistic is -1.7477, corresponding to a p-value of 0.08051. As a result, the hypothesis that the measurements are uncorrelated is not rejected at the 0.05 significance level.