Summary and Analysis of Extension Program Evaluation in R

Salvatore S. Mangiafico

The Quade test is used for similar data and hypotheses as the Friedman test, namely for unreplicated complete block designs.  Because the Quade test uses subtraction to determine the ranges of values within blocks, it may not be appropriate for strictly ordinal data.

Post-hoc tests

The outcome of the Quade test tells you if there are differences among the groups, but doesn’t tell you which groups are different from other groups.  In order to determine which groups are different from others, post-hoc testing can be conducted.

Appropriate data

•  Two-way data arranged in an unreplicated complete block design

•  Dependent variable is interval, or ratio.

•  Treatment or group independent variable is a factor with two or more levels.  That is, two or more groups

•  Blocking variable is a factor with two or more levels

•  Blocks are independent of each other and have no interaction with treatments

Hypotheses

•  Null hypothesis:  The distributions of values for each group are equal.

•  Alternative hypothesis (two-sided): There is systematic difference in the distribution of values for the groups.

Interpretation

Significant results can be reported as “There was a significant difference in values among groups.”

Other notes and alternative tests

The Friedman test is used for the same kinds of data and hypotheses.  The Friedman test is described in the previous chapter.  Cumulative link models or aligned ranks transformation anova allow for more flexibility in design.

Packages used in this chapter

The packages used in this chapter include:

•  psych

•  FSA

•  lattice

•  PMCMRplus

•  rcompanion

The following commands will install these packages if they are not already installed:

if(!require(psych)){install.packages("psych")}
if(!require(FSA)){install.packages("FSA")}
if(!require(lattice)){install.packages("lattice")}
if(!require(PMCMRplus)){install.packages("PMCMRplus")}
if(!require(rcompanion)){install.packages("rcompanion")}

Instructor        Rater  Likert
'Bob Belcher'        a      4
'Bob Belcher'        b      5
'Bob Belcher'        c      4
'Bob Belcher'        d      6
'Bob Belcher'        e      6
'Bob Belcher'        f      6
'Bob Belcher'        g     10
'Bob Belcher'        h      6
'Linda Belcher'      a      8
'Linda Belcher'      b      6
'Linda Belcher'      c      8
'Linda Belcher'      d      8
'Linda Belcher'      e      8
'Linda Belcher'      f      7
'Linda Belcher'      g     10
'Linda Belcher'      h      9
'Tina Belcher'       a      7
'Tina Belcher'       b      5
'Tina Belcher'       c      7
'Tina Belcher'       d      8
'Tina Belcher'       e      8
'Tina Belcher'       f      9
'Tina Belcher'       g     10
'Tina Belcher'       h      9
'Gene Belcher'       a      6
'Gene Belcher'       b      4
'Gene Belcher'       c      5
'Gene Belcher'       d      5
'Gene Belcher'       e      6
'Gene Belcher'       f      6
'Gene Belcher'       g      5
'Gene Belcher'       h      5
'Louise Belcher'     a      8
'Louise Belcher'     b      7
'Louise Belcher'     c      8
'Louise Belcher'     d      8
'Louise Belcher'     e      9
'Louise Belcher'     f      9
'Louise Belcher'     g      8
'Louise Belcher'     h     10
")

### Order levels of the factor; otherwise R will alphabetize them

Data\$Instructor = factor(Data\$Instructor,
levels=unique(Data\$Instructor))

### Create a new variable which is the likert scores as an ordered factor

Data\$Likert.f = factor(Data\$Likert,
ordered=TRUE)

###  Check the data frame

library(psych)

str(Data)

summary(Data)

Summarize data treating Likert scores as factors

xtabs( ~ Instructor + Likert.f,
data = Data)

Likert.f
Instructor       4 5 6 7 8 9 10
Bob Belcher    2 1 4 0 0 0  1
Linda Belcher  0 0 1 1 4 1  1
Tina Belcher   0 1 0 2 2 2  1
Gene Belcher   1 4 3 0 0 0  0
Louise Belcher 0 0 0 1 4 2  1

XT = xtabs( ~ Instructor + Likert.f,
data = Data)

prop.table(XT,
margin = 1)

Likert.f
Instructor           4     5     6     7     8     9    10
Bob Belcher    0.250 0.125 0.500 0.000 0.000 0.000 0.125
Linda Belcher  0.000 0.000 0.125 0.125 0.500 0.125 0.125
Tina Belcher   0.000 0.125 0.000 0.250 0.250 0.250 0.125
Gene Belcher   0.125 0.500 0.375 0.000 0.000 0.000 0.000
Louise Belcher 0.000 0.000 0.000 0.125 0.500 0.250 0.125

Bar plots by group

Note that the bar plots don’t show the effect of the blocking variable.

library(lattice)

histogram(~ Likert.f | Instructor,
data=Data,
layout=c(1,5)      #  columns and rows of individual plots
)

Summarize data treating Likert scores as numeric

library(FSA)

Summarize(Likert ~ Instructor,
data=Data,
digits=3)

Instructor n  mean    sd min   Q1 median   Q3 max percZero
1    Bob Belcher 8 5.875 1.885   4 4.75      6 6.00  10        0
2  Linda Belcher 8 8.000 1.195   6 7.75      8 8.25  10        0
3   Tina Belcher 8 7.875 1.553   5 7.00      8 9.00  10        0
4   Gene Belcher 8 5.250 0.707   4 5.00      5 6.00   6        0
5 Louise Belcher 8 8.375 0.916   7 8.00      8 9.00  10        0

This example uses the formula notation indicating that Likert is the dependent variable, Instructor is the independent variable, and Rater is the blocking variable.  The data= option indicates the data frame that contains the variables.  For the meaning of other options, see ?quade.test.

data = Data)

Quade F = 8.0253, num df = 4, denom df = 28, p-value = 0.0001924

Post-hoc tests

### Order groups by median

Data\$Instructor = factor(Data\$Instructor,
levels = c("Linda Belcher", "Louise Belcher",
"Tina Belcher", "Bob Belcher",
"Gene Belcher"))

library(PMCMRplus)

groups = Data\$Instructor,
blocks = Data\$Rater,

QT

Pairwise comparisons using Quade's test with TDist approximation

Linda Belcher Louise Belcher Tina Belcher Bob Belcher
Louise Belcher 0.77620       -              -            -
Tina Belcher   0.50547       0.38132        -            -
Bob Belcher    0.00902       0.00544        0.04533      -
Gene Belcher   0.00108       0.00099        0.00539      0.36572

library(rcompanion)

QTT =PMCMRTable(QT)

QTT

Comparison  p.value
1  Louise Belcher - Linda Belcher = 0    0.776
2    Tina Belcher - Linda Belcher = 0    0.505
3     Bob Belcher - Linda Belcher = 0  0.00902
4    Gene Belcher - Linda Belcher = 0  0.00108
5   Tina Belcher - Louise Belcher = 0    0.381
6    Bob Belcher - Louise Belcher = 0  0.00544
7   Gene Belcher - Louise Belcher = 0 0.000987
8      Bob Belcher - Tina Belcher = 0   0.0453
9     Gene Belcher - Tina Belcher = 0  0.00539
10     Gene Belcher - Bob Belcher = 0    0.366

library(rcompanion)

cldList(p.value ~ Comparison, data = QTT)

Group Letter MonoLetter
1 LouiseBelcher      a         a
2   TinaBelcher      a         a
3    BobBelcher      b          b
4   GeneBelcher      b          b
5  LindaBelcher      a         a

Example from Conover

This example is taken from the Quade test section of Conover (1999).

Store   A     B     C     D     E
1      5     4     7    10    12
2      1     3     1     0     2
3     16    12    22    22    35
4      5     4     3     5     4

5     10     9     7    13    10
6     19    18    28    37    58
7     10     7     6     8     7
")

if(!require(tidyr)){install.packages("tidyr")}

library(tidyr)

Conover = gather(Conover1, Brand, Bottles, A:E, factor_key=TRUE)

###  Check the data frame

library(psych)

str(Conover)

summary(Conover)

data = Conover)

Quade F = 3.8293, num df = 4, denom df = 24, p-value = 0.01519

library(PMCMRplus)

groups = Conover\$Brand,
blocks = Conover\$Store,

Pairwise comparisons using Quade's test with TDist approximation

A     B     C     D
B 0.298 -     -     -
C 0.840 0.359 -     -
D 0.246 0.051 0.204 -
E 0.104 0.021 0.090 0.575