Tests of symmetric margins—or marginal homogeneity—for nominal data are used when the counts on a contingency table represent values that are paired or repeated in time.
As an example, consider a question repeated on a pretest and a posttest. We may want to know if the number of correct responses changed from the pretest to the posttest.
Did students have the correct answer to the question?
After
Before Correct Incorrect
Correct 2 0
Incorrect 21 7
Note that the row names and column names have the same levels, and that counts represent paired responses. That is, for each observation you must know the individual’s response before and after.
Also note that the number of students included in the table are 30, or the sum of the cell counts.
In essence, those students with the same response before and after don’t affect the assessment of the change in responses. We would focus on the “discordant” counts. That is, how many students had incorrect answers before and correct answers after, in contrast to those who had the reverse trend. Here, because 21 changed from incorrect to correct, and 0 changed from correct to incorrect, we might suspect that there was a significant change in responses from incorrect to correct.
To grasp the difference between nominal tests of association and nominal tests of symmetry, be sure to visit the coffee and tea example below in the section “An example without repeated measures, comparing test of symmetry with test of association”.
Appropriate data
• Two nominal variables with two or more levels each, and each with the same levels.
• Observations are paired or matched between the two variables.
• McNemar and McNemar–Bowker tests may not be appropriate if discordant cells have low counts.
Hypotheses
• Null hypothesis: The contingency table is symmetric. That is, the probability of cell [i, j] is equal to the probability of cell [j, i].
• Alternative hypothesis (twosided): The contingency table is not symmetric.
Interpretation
Depending on the context, significant results can be reported as e.g. “There was an increase in correct answers from the pretest to the posttest.” Or, “There was a significant change from answer A to answer B.”
Posthoc analysis
Posthoc analysis for tests on a contingency table larger than 2 x 2 can be conducted by conducting tests for the component 2 x 2 tables. A correction for multiple tests could be applied.
Other notes and alternative tests
• For unpaired data, see the tests in the chapter Association Tests for Nominal Data.
• For multiple times or groups, Cochran’s Q test can be used if the response is binary.
Packages used in this chapter
The packages used in this chapter include:
• rcompanion
The following commands will install these packages if they are not already installed:
if(!require(rcompanion)){install.packages("rcompanion")}
McNemar and McNemar–Bowker tests
For a 2 x 2 table, the most common test for symmetry is McNemar’s test. For larger tables, McNemar’s test is generalized as the McNemar–Bowker symmetry test. One drawback to the latter test is that it may fail if there are 0’s in certain locations in the matrix.
McNemar’s test may not be reliable if there are low counts in the “discordant” cells. Authors recommend that these cells to sum to at least 5 or 10 or 25.
Exact tests
Exact tests of symmetry reduce to exact tests for goodnessoffit. A 2 x 2 table is analyzed with a binomial exact test. Examples of this are shown in this chapter in the “Optional analyses: conducting exact tests for symmetry” section.
The nominalSymmetryTest function can conduct these exact tests easily.
Example of tests for paired data nominal data
Alucard teaches a Master Gardener training on rain gardens for stormwater management and one on rain barrels. He wishes to assess if people are more willing to install these green infrastructure practices after attending the training. His data follow. Note that there are 46 attendees answering each question, and the counts in each table sum to 46.
Are you planning to install a rain barrel?
After
Before Yes No
Yes 9 5
No 17 15
Are you planning to install a rain garden?
After
Before Yes No Maybe
Yes 6 0 1
No 5 3 7
Maybe 11 1 12
Rain barrel
Matrix.1 = as.matrix(read.table(header=TRUE, row.names=1, text="
Before After.yes After.no
Before.yes 9 5
Before.no 17 15
"))
Matrix.1
After.yes After.no
Before.yes 9 5
Before.no 17 15
sum(Matrix.1)
[1] 46
Rain garden
Matrix.2 = as.matrix(read.table(header=TRUE, row.names=1, text="
Before Yes.after No.after Maybe.after
Yes.before 6 0 1
No.before 5 3 7
Maybe.before 11 1 12
"))
Matrix.2
Yes.after No.after Maybe.after
Yes.before 6 0 1
No.before 5 3 7
Maybe.before 11 1 12
sum(Matrix.2)
[1] 46
McNemar and McNemar –Bowker tests
Rain barrel
mcnemar.test(Matrix.1)
McNemar's Chisquared test with continuity correction
McNemar's chisquared = 5.5, df = 1, pvalue = 0.01902
Rain garden
mcnemar.test(Matrix.2)
McNemar's Chisquared test
McNemar's chisquared = 17.833, df = 3, pvalue = 0.0004761
Rain garden example with posthoc tests
Here, the global test result is the same as the mcnemar.test result.
Looking at the posthoc tests, we see that there was a significant change from
“maybe” to “yes”.
library(rcompanion)
nominalSymmetryTest (Matrix.2)
$Global.test.for.symmetry
Dimensions p.value
1 3 x 3 0.000476
$Pairwise.symmetry.tests
Comparison p.value p.adjust
1 Yes.before/Yes.after : No.before/No.after 0.0736 0.0771
2 Yes.before/Yes.after : Maybe.before/Maybe.after 0.00937 0.0281
3 No.before/No.after : Maybe.before/Maybe.after 0.0771 0.0771
$p.adjustment
Method
1 fdr
$statistical.method
Method
1 McNemar test
Maybe to Yes, adjusted p value = 0.0281, with 11 changing from Maybe to Yes, and 1 changing from Yes to Maybe.
Before Yes.after No.after
Maybe.after
Yes.before 6 0
1
No.before 5 3 7
Maybe.before 11 1
12
Exact tests for symmetry
Rain barrel
library(rcompanion)
nominalSymmetryTest(Matrix.1,
exact=TRUE)
$Global.test.for.symmetry
Dimensions p.value
1 2 x 2 0.0169
$Statistical.method
Method
1 binomial test
Rain garden
Note that the nominalSymmetryTest function won’t return a global exact test result for tables larger than 2 x 2, but will return exact tests for the posthoc tests.
library(rcompanion)
nominalSymmetryTest (Matrix.2,
exact=TRUE)
$Global.test.for.symmetry
Dimensions p.value
1 3 x 3 NA
$Pairwise.symmetry.tests
Comparison p.value p.adjust
1 Yes.before/Yes.after : No.before/No.after 0.0625 0.0703
2 Yes.before/Yes.after : Maybe.before/Maybe.after 0.00635 0.0190
3 No.before/No.after : Maybe.before/Maybe.after 0.0703 0.0703
$p.adjustment
Method
1 fdr
$statistical.method
Method
1 binomial test
Maybe to Yes, adjusted p value = 0.0190, with 11 changing from Maybe to Yes, and 1 changing from Yes to Maybe.
Before Yes.after No.after
Maybe.after
Yes.before 6 0
1
No.before 5 3 7
Maybe.before 11 1
12
Effect size
Appropriate effect sizes for data subjected to McNemar, McNemar–Bowker, or equivalent exact tests include Cohen’s g and odds ratio.
Considering a 2 x 2 table, with a and d being the concordant cells and b and c being the discordant cells, the odds ratio is simply the greater of (b/c) or (c/b), and P is the greater of (b/(b+c)) or (c/ b+c)). Cohen’s g is P – 0.5. These statistics can be extended to larger tables.
Cohen (1988) lists interpretations for Cohen’s g. These can easily be translated into interpretations for the odds ratio. For example, the cutoff for the “small” interpretation is g ≥ 0.05, which would be 0.55 : 0.45 odds, or 0.55/0.45, or 1.22.
Because odds ratio is ratio of values, a large effect for odds ratio is far from 1. It may be useful to use the reciprocal of the odds ratio for odds ratios less than 1.

Small

Medium 
Large 
Cohen’s g 
0.05 – < 0.15 
0.15 – < 0.25 
≥ 0.25 
Odds ratio (OR) 
1.22 – < 1.86 
1.86 – < 3.00 0.538 – < 0.333 
≥ 3.00 ≤ 0.333 
Rain barrel
In the output, OR is the odds ratio, and g is Cohen’s g.
library(rcompanion)
cohenG(Matrix.1)
$Global.statistics
Dimensions OR P g
1 2 x 2 0.294 0.227 0.273
1 / 0.294
[1] 3.401361
### So, the odds ratio can be interpreted as either
0.294 or 3.4.
Confidence intervals for odds ratio can be requested also.
cohenG(Matrix.1,
ci=TRUE)
$Global.statistics
Dimensions Statistic Value lower.ci upper.ci
1 2 x 2 OR 0.294 0.0588 0.733
2 2 x 2 P 0.227 0.0526 0.428
3 2 x 2 g 0.273 0.4474 0.072
Rain garden
library(rcompanion)
cohenG(Matrix.2)
$Global.statistics
Dimensions OR P g
1 3 x 3 11.5 0.92 0.42
$Pairwise.statistics
Comparison OR P g
1 Yes.before/Yes.after : No.before/No.after Inf 1 0.5
2 Yes.before/Yes.after : Maybe.before/Maybe.after 11 0.917 0.417
3 No.before/No.after : Maybe.before/Maybe.after 7 0.875 0.375
An example without repeated measures, comparing test of symmetry with test of association
As another example, consider a survey of tea and coffee drinking, in which each respondent is asked both if they drink coffee, and if they drink tea.
Tea
Coffee Yes No
Yes 37 17
No 9 25
We could use a test of symmetry in this case if the question we wanted to answer was, Is coffee more popular than tea? That is, is it more common for someone to drink coffee and not tea than to drink tea and not coffee? (Those who drink both or drink neither are not relevant to this question.)
This is an example of using a test of symmetry to test the relative frequency of two dichotomous variables when the same subjects are surveyed.
Matrix.3 = as.matrix(read.table(header=TRUE, row.names=1, text="
Coffee Yes No
Yes 37 17
No 9 25
"))
mcnemar.test(Matrix.3)
McNemar's Chisquared test with continuity correction
McNemar's chisquared = 1.8846, df = 1, pvalue = 0.1698
### Neither coffee nor tea is more popular,
specifically because
### neither the 9 nor the 17 in the table are large relative to
### the other.
A test of association answers a very different question. Namely, Is coffee drinking associated with tea drinking? That is, is someone more likely to drink tea if they drink coffee?
chisq.test(Matrix.3)
Pearson's Chisquared test with Yates' continuity correction
Xsquared = 13.148, df = 1, pvalue = 0.0002878
### Coffee drinking and tea drinking are associated,
in this case people who
### drink coffee are likely to drink tea. This is a positive association.
### A negative association could also be significant.
Optional analysis: a 4 x 4 example with several 0’s
As an additional example, imagine a religious caucusing event in which advocates try to sway attendees to switch their religions.
Matrix row names are the attendees’ original religions, and the column names, with a “2” added, are attendees’ new religions after the caucus. Note that there are several 0 counts in the matrix.
The mcnemar.test function fails in this case because of the position of some 0 counts in the matrix.
Matrix.4 = as.matrix(read.table(header=TRUE, row.names=1, text="
Before Pastafarian.after Discordiant.after Dudist.after Jedi.after
Pastafarian 7 0 23 0
Discordiant 0 7 0 33
Dudist 3 0 7 1
Jedi 0 1 0 7
"))
Matrix.4
Pastafarian.after Discordiant.after Dudist.after Jedi.after
Pastafarian 7 0 23 0
Discordiant 0 7 0 33
Dudist 3 0 7 1
Jedi 0 1 0 7
McNemar –Bowker test
mcnemar.test(Matrix.4)
McNemar's Chisquared test
McNemar's chisquared = NaN, df = 6, pvalue = NA
Posthoc tests
library(rcompanion)
nominalSymmetryTest(Matrix.4)
$Global.test.for.symmetry
Dimensions p.value
1 4 x 4 NaN
$Pairwise.symmetry.tests
Comparison p.value
p.adjust
1 Pastafarian/Pastafarian.after : Discordiant/Discordiant.after
<NA> NA
2 Pastafarian/Pastafarian.after : Dudist/Dudist.after 0.000194
2.91e04
3 Pastafarian/Pastafarian.after : Jedi/Jedi.after
<NA> NA
4 Discordiant/Discordiant.after : Dudist/Dudist.after
<NA> NA
5 Discordiant/Discordiant.after : Jedi/Jedi.after 1.06e07
3.18e07
6 Dudist/Dudist.after : Jedi/Jedi.after 1
1.00e+00
$p.adjustment
Method
1 fdr
$statistical.method
Method
1 McNemar test
A look at the significant results
Pastafarian to Dudist, adjusted p value = 0.000291
Before
Pastafarian.after Discordiant.after Dudist.after Jedi2.after
Pastafarian 7 0 23 0
Discordiant 0 7 0 33
Dudist 3 0
7 1
Jedi 0 1 0 7
Discordiant to Jedi, adjusted p value <
0.0001
Before
Pastafarian.after Discordiant.after Dudist.after Jedi.after
Pastafarian 7 0 23 0
Discordiant 0 7 0 33
Dudist 3 0 7 1
Jedi 0 1 0 7
Optional analyses: conducting exact tests for symmetry
The exact symmetry tests for a 2 x 2 table can be conducted directly with the binom.test function. These results match the results of the nominalSymmetryTest function with the exact=TRUE option.
Rain barrel
After
Before Yes No
Yes 9 5
No 17 15
Rain barrel
For a 2 x 2 matrix, x = the count in one of the discordant cells, and n = the sum of the counts in discordant cells. The expected proportion is 0.50.
You could also follow the method for an n x n matrix below.
x = 5
n = 5 + 17
expected = 0.50
binom.test(x, n, expected)
Exact binomial test
number of successes = 5, number of trials = 22, pvalue = 0.0169
Optional: Comparing odds ratio and Cohen’s g
For a 2 x 2 table of matched counts, odds ratio and Cohen’s g are precisely related, and their interpretations according to Cohen will always coincide. An odds ratio value of 1 corresponds to a Cohen's g of 0. Cohen’s g approaches 0.5 as odds ratio approaches infinity. In the second figure below, the colors indicate Cohen’s interpretation of lessthansmall, small, medium, and large as the blue becomes darker.
References
Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences, 2nd Edition. Routledge.
Exercises N
1. Considering Alucard’s data,
a. How many students responded to the rain barrel
question?
b. How many students changed their answer on the rain
barrel question from no to yes?
c. What do you conclude about the results on the rain barrel question? Include:
• The statistical test you are using, pvalue from that test, conclusion of the test, and any other comments on the test.
• The effect size and interpretation.
• Description of the observed counts or proportions. Is there anything notable?
• Any other practical conclusions you wish to note.
d. What do you conclude about the results of the global test of the rain garden question?
e. What do you conclude about the results of the posthoc analysis of the rain garden question?
2. Considering the coffee and tea example, Do you understand the difference between the hypotheses for tests of association and tests of symmetry? Would you be comfortable choosing the correct approach?
3. Ryuk and Rem held a workshop on planting habitat for pollinators like bees and butterflies. They wish to know if attendees were more likely to do a planting after the workshop than before.
Will plant?
After
Before Yes.after No.after Maybe.after
Yes 17 0 0
No 5 9 13
Maybe 15 0 7
For each of the following, answer the question, and show the output from the analyses you used to answer the question.
a. How many students responded to this question?
b. How many students changed their answer from no to yes?
c. What do you conclude about the results of the global test?
d. What do you conclude about the results of the posthoc analysis?