[banner]

Summary and Analysis of Extension Program Evaluation in R

Salvatore S. Mangiafico

Brunner-Munzel and van Elteren Tests

Brunner–Munzel Test

 

The Brunner–Munzel test is used to test the stochastic equality of two independent samples, similar to the Wilcoxon–Mann–Whitney test.  The Brunner–Munzel test is sometimes called the generalized Wilcoxon test.

 

The brunnermunzel.test function in the brunnermunzel package can be used for this test.  There is also an option to conduct the test by permutation.

 

van Elteren test

 

The van Elteren test is a stratified version of the Wilcoxon–Mann–Whitney test.  That is, it compares two groups across another variable representing strata.

 

The sanon package conducts a stratified Wilcoxon–Mann–Whitney test.  It can also handle models with multiple stratifying variables.

 

A stratified test can also be conducted with the coin package.

 

Another approach is to use aligned ranks transformation anova.

 

Note that all these approaches are slightly different and return different results.

 

Packages used in this chapter

 

The packages used in this chapter include:

•  brunnermunzel

•  rcompanion

•  sanon

•  coin

•  ARTool

 

The following commands will install these packages if they are not already installed:


if(!require(brunnermunzel)){install.packages("brunnermunzel")}
if(!require(rcompanion)){install.packages("rcompanion")}
if(!require(sanon)){install.packages("sanon")}
if(!require(coin)){install.packages("coin")}
if(!require(ARTool)){install.packages("ARTool")}

 

Brunner–Munzel test example


Data = read.table(header=TRUE, stringsAsFactors=TRUE, text="

 Speaker  Likert
 Pooh      3
 Pooh      5
 Pooh      4
 Pooh      4
 Pooh      4
 Pooh      4
 Pooh      4
 Pooh      4
 Pooh      5
 Pooh      5
 Piglet    2
 Piglet    4
 Piglet    2
 Piglet    2
 Piglet    1
 Piglet    2
 Piglet    3
 Piglet    2
 Piglet    2
 Piglet    3
")


Brunner Munzel test

 

Note that the results also give the Vargha and Delaney A statistic and confidence interval for this statistic.


library(brunnermunzel)

brunnermunzel.test(Likert ~ Speaker, data=Data)


Brunner-Munzel Test

Brunner-Munzel Test Statistic = 10.354, df = 12.688, p-value = 1.495e-07

95 percent confidence interval:
 0.8558721 1.0441279

sample estimates:
P(X<Y)+.5*P(X=Y)
            0.95


Brunner Munzel test by permutation


library(brunnermunzel)

brunnermunzel.test(Likert ~ Speaker, data=Data, perm=TRUE)


permuted Brunner-Munzel Test

p-value = 0.0002382

sample estimates:
P(X<Y)+.5*P(X=Y)
            0.95


Comparison to the Wilcoxon–Mann–Whitney test


wilcox.test(Likert ~ Speaker, data=Data, correct=FALSE)


Wilcoxon rank sum test

W = 5, p-value = 0.0004065


Vargha and Delaney’s A


library(rcompanion)

1 - vda(Likert ~ Speaker, data=Data, ci=TRUE)


   VDA lower.ci upper.ci
1 0.95        1    0.843

### Note these values essentially match the values given by

###  the brunnermunzel output, with some variability in the

###  confidence interval for bootstrapping, and the fact that

###  the brunnermunzel output allows for values to exceed 1.


van Elteren test example

 

Data1 = read.table(header=TRUE, stringsAsFactors=TRUE, text="

Instructor        Supplement  Sodium
'Brendon Small'   A           1200
'Brendon Small'   A           1400
'Brendon Small'   A           1350
'Brendon Small'   A            950
'Brendon Small'   A           1400
'Brendon Small'   B           1150
'Brendon Small'   B           1300
'Brendon Small'   B           1325
'Brendon Small'   B           1425
'Brendon Small'   B           1500
'Brendon Small'   C           1250
'Brendon Small'   C           1150
'Brendon Small'   C            950
'Brendon Small'   C           1150
'Brendon Small'   C           1600
'Brendon Small'   D           1300
'Brendon Small'   D           1050
'Brendon Small'   D           1300
'Brendon Small'   D           1700
'Brendon Small'   D           1300
'Melissa Robins'  A            900
'Melissa Robins'  A           1100
'Melissa Robins'  A           1150
'Melissa Robins'  A            950
'Melissa Robins'  A           1100
'Melissa Robins'  B           1150
'Melissa Robins'  B           1250
'Melissa Robins'  B           1250
'Melissa Robins'  B           1225
'Melissa Robins'  B           1325
'Melissa Robins'  C           1125
'Melissa Robins'  C           1025
'Melissa Robins'  C            950
'Melissa Robins'  C            925
'Melissa Robins'  C           1200
'Melissa Robins'  D           1100
'Melissa Robins'  D            950
'Melissa Robins'  D           1300
'Melissa Robins'  D           1400
'Melissa Robins'  D           1100
")

xtabs(~ Instructor + Supplement, data=Data1)


                Supplement
Instructor       A B C D
  Brendon Small  5 5 5 5
  Melissa Robins 5 5 5 5

### Counts of observations per Instructor and Supplement


sanon package


library(sanon)

Out = sanon(Sodium ~ grp(Instructor) + strt(Supplement), data=Data1)

summary(Out)


       Estimate Std.Err Chisq Pr(>Chisq)  
Sodium  -0.2650  0.0809  10.7     0.0011 **


Out


Sample size: 40

Strata ( Supplement ): A, B, C, D

Response levels:
[Sodium; 19 levels] (lower) 900, 925, 950, ..., 1500, 1600, 1700 (higher)

Design Matrix:
       [,1]
Sodium    1


coin package

 

library(coin)

wilcox_test(Sodium ~ Instructor | Supplement, data=Data1)


Asymptotic Wilcoxon-Mann-Whitney Test

Instructor (Brendon Small, Melissa Robins)
stratified by Supplement

Z = 2.9858, p-value = 0.002829


Comparison to unstratified Wilcoxon-Mann-Whitney


wilcox_test(Sodium ~ Instructor, data=Data1)


Asymptotic Wilcoxon-Mann-Whitney Test

Z = 2.851, p-value = 0.004359


Aligned ranks transformation anova


library(ARTool)

model = art(Sodium ~ Instructor + Supplement + Instructor:Supplement, data=Data1)

anova(model)


Analysis of Variance of Aligned Rank Transformed Data

                        Df Df.res F value    Pr(>F)  
1 Instructor             1     32 9.74449 0.0037985 **
2 Supplement             3     32 2.55669 0.0725482  .
3 Instructor:Supplement  3     32 0.44091 0.7253272  


References

 

Kawaguchi, A, and Koch, G.G. 2015. sanon: An R Package for Stratified Analysis with Nonparametric Covariable Adjustment. JSS Journal of Statistical Software: 67(9). doi: 10.18637/jss.v067.