A Python Companion to Extension Program Evaluation

Salvatore S. Mangiafico

Quade Test

Quade test in SAEPER

 

For a discussion of this test, see the corresponding chapter in Summary and Analysis of Extension Program Evaluation in R (rcompanion.org/handbook/F_11.html).

 

Importing packages in this chapter

 

The following commands will import required packages used in this chapter from libraries and assign them common aliases.  You may need install these libraries first.

 

import io

 

import os

 

import numpy as np

 

import scipy.stats as stats

 

import pandas as pd

 

import pingouin as pg

 

import scikit_posthocs as sp

 

import matplotlib.pyplot as plt

 

import seaborn as sns

 

from statds.no_parametrics import quade

 

 

Setting your working directory

 

You may wish to set your working directory for exported plots.

 

os.chdir("C:/Users/Sal Mangiafico/Desktop")

 

print(os.getcwd())

 

 

Example of Quade test

 

Data = pd.read_table(sep="\\s+", filepath_or_buffer=io.StringIO("""

 

Instructor        Rater  Likert
 "Bob Belcher"        a      4

 "Bob Belcher"        b      5

 "Bob Belcher"        c      4

 "Bob Belcher"        d      6

 "Bob Belcher"        e      6

 "Bob Belcher"        f      6

 "Bob Belcher"        g     10

 "Bob Belcher"        h      6

 "Linda Belcher"      a      8

 "Linda Belcher"      b      6

 "Linda Belcher"      c      8

 "Linda Belcher"      d      8

 "Linda Belcher"      e      8

 "Linda Belcher"      f      7

 "Linda Belcher"      g     10

 "Linda Belcher"      h      9

 "Tina Belcher"       a      7

 "Tina Belcher"       b      5

 "Tina Belcher"       c      7

 "Tina Belcher"       d      8

 "Tina Belcher"       e      8

 "Tina Belcher"       f      9

 "Tina Belcher"       g     10

 "Tina Belcher"       h      9

 "Gene Belcher"       a      6

 "Gene Belcher"       b      4

 "Gene Belcher"       c      5

 "Gene Belcher"       d      5

 "Gene Belcher"       e      6

 "Gene Belcher"       f      6

 "Gene Belcher"       g      5

 "Gene Belcher"       h      5

 "Louise Belcher"     a      8

 "Louise Belcher"     b      7

 "Louise Belcher"     c      8

 "Louise Belcher"     d      8

 "Louise Belcher"     e      9

 "Louise Belcher"     f      9

 "Louise Belcher"     g      8

 "Louise Belcher"     h     10  
"""))

 

### Convert Instructor and Rater to category type

 

Data['Instructor']  = Data['Instructor'].astype('category')

 

Data['Rater']  = Data['Rater'].astype('category')

 

 

### Create new variable, Likert as a category variable

 

Data['Likert.f']  = Data['Likert'].astype('category')

 

 

### Order Speaker by desired values

 

InstructorLevels = ['Bob Belcher', 'Linda Belcher', 'Tina Belcher',

                 'Gene Belcher', 'Louise Belcher']

 

Data['Instructor'] = Data['Instructor'].cat.reorder_categories(InstructorLevels)

 

 

print(Data['Instructor'].cat.categories)

 

Index(['Bob Belcher', 'Linda Belcher', 'Tina Belcher', 'Gene Belcher',

       'Louise Belcher'],

      dtype='object')

 

 

print(Data.info())

 

 #   Column      Non-Null Count  Dtype  

---  ------      --------------  -----  

 0   Instructor  40 non-null     category

 1   Rater       40 non-null     category

 2   Likert      40 non-null     int64  

 3   Likert.f    40 non-null     category

 

 

Summarize data treating Likert scores as categories

 

pd.crosstab(Data['Instructor'], Data['Likert.f'])

 

Likert.f        4  5  6  7  8  9  10

Instructor                         

Bob Belcher     2  1  4  0  0  0   1

Linda Belcher   0  0  1  1  4  1   1

Tina Belcher    0  1  0  2  2  2   1

Gene Belcher    1  4  3  0  0  0   0

Louise Belcher  0  0  0  1  4  2   1

 

 

pd.crosstab(Data['Instructor'], Data['Likert.f'], normalize='index')

 

Likert.f            4      5      6      7     8      9     10

Instructor                                                   

Bob Belcher     0.250  0.125  0.500  0.000  0.00  0.000  0.125

Linda Belcher   0.000  0.000  0.125  0.125  0.50  0.125  0.125

Tina Belcher    0.000  0.125  0.000  0.250  0.25  0.250  0.125

Gene Belcher    0.125  0.500  0.375  0.000  0.00  0.000  0.000

Louise Belcher  0.000  0.000  0.000  0.125  0.50  0.250  0.125

 

 

Bar plots of data by group

 

sns.set_theme(style='white')

 

Plot = sns.FacetGrid(data=Data, row='Instructor',

                  margin_titles=True, height=2, aspect= 2)

 

Plot.map(sns.countplot, 'Likert.f')

 

Plot.tight_layout()

 

Plot.savefig('LikertBarBelcher.png', format='png', dpi=300)

 

 

 

Summarize data treating Likert scores as numeric

 

Summary = Data.groupby('Instructor')['Likert'].describe()

 

print(Summary)

 

                count   mean       std  min   25%  50%   75%   max

Instructor                                                       

Bob Belcher       8.0  5.875  1.885092  4.0  4.75  6.0  6.00  10.0

Linda Belcher     8.0  8.000  1.195229  6.0  7.75  8.0  8.25  10.0

Tina Belcher      8.0  7.875  1.552648  5.0  7.00  8.0  9.00  10.0

Gene Belcher      8.0  5.250  0.707107  4.0  5.00  5.0  6.00   6.0

Louise Belcher    8.0  8.375  0.916125  7.0  8.00  8.0  9.00  10.0

 

 

Quade test example

 

Using StaTDS

 

The quade() function in StaTDS expects data in wide format.  We’ll create a new data frame, Data1, with the first column as the block and the remaining columns having observations for each of the groups or treatments.

 

Rater  = np.array(['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h'])

 

Bob    = np.array(Data['Likert'][Data['Instructor'] == 'Bob Belcher'])

Linda  = np.array(Data['Likert'][Data['Instructor'] == 'Linda Belcher'])

Tina   = np.array(Data['Likert'][Data['Instructor'] == 'Tina Belcher'])

Gene   = np.array(Data['Likert'][Data['Instructor'] == 'Gene Belcher'])

Louise = np.array(Data['Likert'][Data['Instructor'] == 'Louise Belcher'])

 

Data1 = pd.DataFrame({'Rater': Rater, 'Bob': Bob, 'Linda': Linda, 'Tina': Tina, 'Gene': Gene, 'Louise': Louise})

 

Data1

 

  Rater  Bob  Linda  Tina  Gene  Louise

0     a    4      8     7     6       8

1     b    5      6     5     4       7

2     c    4      8     7     5       8

3     d    6      8     8     5       8

4     e    6      8     8     6       9

5     f    6      7     9     6       9

6     g   10     10    10     5       8

7     h    6      9     9     5      10

 

 

quade(Data1)

 

 ({'Bob': 3.923611111111111,

   'Linda': 2.0555555555555554,

   'Tina': 2.513888888888889,

   'Gene': 4.625,

   'Louise': 1.8819444444444444},

  8.025290473231268,

  0.00019236606572845994

 

### The output lists the average ranks for each group,

###  and the test statistic and the p-value.

 

 

rankings, statistic, p_value, critical_value, hypothesis = quade(Data1)

 

round(statistic, 4)

 

8.0253

 

 

round(p_value, 6)

 

0.000192

 

 

Post-hoc test for multiple comparisons of groups

 

Results below differ from those reported by R.  This has to do with differences p-value adjustment methods.

 

The following call will prevent pandas from truncating the output.

 

pd.set_option('display.max_columns', 500)

 

 

The following will order the Instructor categories by their median responses.  It appears, though, that this ordering isn’t used in the following post-hoc functions.

 

InstructorLevels = ['Linda Belcher', 'Louise Belcher','Tina Belcher',

                 'Bob Belcher','Gene Belcher']

 

Data['Instructor'] = Data['Instructor'].cat.reorder_categories(InstructorLevels)

 

 

print(Data['Instructor'].cat.categories)

 

Index(['Linda Belcher', 'Louise Belcher', 'Tina Belcher', 'Bob Belcher',

       'Gene Belcher'],

      dtype='object')

 

 

Quade post-hoc test

 

Several different p-value adjustment methods are available. See the function documentation for the options.

 

sp.posthoc_quade(Data, melted=True,

                            y_col='Likert', group_col='Instructor',

                            block_col='Rater', block_id_col='Rater',

                            p_adjust=None)

 

                Bob Belcher  Linda Belcher  Tina Belcher  Gene Belcher  Louise Belcher

Bob Belcher        1.000000       0.004508      0.027198      0.256002        0.002176 

Linda Belcher      0.004508       1.000000      0.454923      0.000215        0.776202

Tina Belcher       0.027198       0.454923      1.000000      0.001617        0.305055

Gene Belcher       0.256002       0.000215      0.001617      1.000000        0.000099

Louise Belcher     0.002176       0.776202      0.305055      0.000099        1.000000

 

 

Example from Conover

 

This example is taken from the Quade test section of Conover (1999).

 

Conover = pd.read_table(sep="\\s+", filepath_or_buffer=io.StringIO("""

 

Store   A     B     C     D     E

 1      5     4     7    10    12

 2      1     3     1     0     2

 3     16    12    22    22    35

 4      5     4     3     5     4

 5     10     9     7    13    10

 6     19    18    28    37    58

 7     10     7     6     8     7

"""))

 

 

quade(Conover)

 

  ({'A': 3.3392857142857144,

  'B': 4.357142857142857,

  'C': 3.5,

  'D': 2.1607142857142856,

  'E': 1.6428571428571428},

 3.8292515841753727,

 0.015189020073274623

 

 

rankings, statistic, p_value, critical_value, hypothesis = quade(Conover)

 

round(statistic, 4)

 

3.8293

 

 

round(p_value, 5)

 

0.01519

 

Post-hoc test

 

Results below differ from those reported by R.  This has to do with differences p-value adjustment methods.

 

pd.set_option('display.max_columns', 500)

 

 

Quade post-hoc test

 

Several different p-value adjustment methods are available. See the function documentation for the options.

 

sp.posthoc_quade(Conover, p_adjust=None)

 

          Store         A         B         C         D         E

Store  1.000000  0.028715  0.255619  0.059560  0.001082  0.000248

A      0.028715  1.000000  0.263730  0.736455  0.197214  0.072962

B      0.255619  0.263730  1.000000  0.430414  0.019976  0.005427

C      0.059560  0.736455  0.430414  1.000000  0.107638  0.035804

D      0.001082  0.197214  0.019976  0.107638  1.000000  0.593516

E      0.000248  0.072962  0.005427  0.035804  0.593516  1.000000

References

 

Conover, W.J. 1999. Practical Nonparametric Statistics, 3rd. John Wiley & Sons.

 

Luna, C., A.R. Moya,  J.M. Luna, S. Ventura . 2024. StaTDS library: Statistical tests for Data Science. Neurocomputing 595:127877. doi.org/10.1016/j.neucom.2024.127877.

 

StaTDS: Library for statistical testing and comparison of algorithm results. github.com/kdis-lab/StaTDS.

 

Statistical Tests for Data Science (StaTDS). statds.readthedocs.io/en/latest/.

©2025 by Salvatore S. Mangiafico.
Rutgers Cooperative Extension, New Brunswick, NJ.

Non-commercial reproduction of this content, with attribution, is permitted.
For-profit reproduction without permission is prohibited.

If you use the code or information in this site in a published work, please cite it as a source.  Also, if you are an instructor and use this book in your course, please let me know.   My contact information is on the Author of this Book page.

Citation

Mangiafico, S.S. 2025. Summary and Analysis of Extension Program Evaluation in R, version 0.2.7, revised 2025. rcompanion.org/python/. (Pdf version: rcompanion.org/documents/PythonCompanionProgramEvaluation.pdf.)