Summary and Analysis of Extension Program Evaluation in R

Salvatore S. Mangiafico



Introduction to Cumulative Link Models (CLM) for Ordinal Data



In the section on nonparametric tests in this book, each test is used for data from a specific situation or design, such as comparing groups from two-sample unpaired data, or two-sample paired data, or with an unreplicated complete block design.

Cumulative link models are a different approach to analyzing ordinal data. Models can be chosen to handle simple or more complex designs.

This approach is very powerful and flexible, and might be considered the best approach for data with ordinal dependent variables in many cases.

However, a few disadvantages to using these models are that 1) your audience may not familiar with them, 2) their results can be somewhat tricky to interpret or explain, and 3) some models won’t converge or model assumptions won’t be met for some data sets.


These models are also called ordinal regression models, or proportional odds models.


The ordinal package

These models and tests will use the ordinal package, and either of two functions, clm and clmm.


A few notes on using cumulative link models:

•  The dependent variable must be an ordered factor variable.  It does not need to have numerals for levels.  For example it could have levels doctorate  >  masters  >  bachelors   >   associates  >  high.school.  But also it could have the levels  5 > 4 > 3 > 2 > 1


•  Independent variables can be factors, ordered factors, or interval/ratio variables.


•  The general interpretation for significant results of these models is that there is a significant effect of the independent variable on the dependent variable, or that there is a significant difference among groups.


•  Post-hoc tests for factors or groups can be conducted with the lsmeans package.  An optional approach for post-hoc tests is to use pairwise ordinal tests of groups.  These can be conducted with the functions pairwiseOrdinalTest and pairwiseOrdinalPairedTest.


•  The threshold = “equidistant” and threshold = “symmetric” options can be used to indicate to the software that levels of the response variable are equally spaced or symmetrically spaced, respectively.  This is useful to indicate when these conditions are assumed to be true, but are also useful to try if the model procedure produces errors.  Likert items using symmetrical language in the range of responses could be considered symmetric.  Likert items with several numbered options with anchor terms only at the ends of scale might be considered equidistant.


Analysis of deviance

The significance of the effects of independent variables will be tested with an analysis of deviance (ANODE) approach. This is analogous to the analysis of variance (ANOVA) used in linear models.


Model assumptions for CLM

Most statistical models have some assumptions about the underlying data.  In order for the model to be valid, these assumptions have to be met.


For CLM, the assumption of concern is called the proportional odds assumption.  An explanation of this assumption can be found in the Wikipedia or IDRE articles cited below.


The ordinal package can test for the proportional odds assumption with the nominal_test and

scale_test functions (Christensen 2015b).  If any independent variable fails these tests (that is, a significant p-value is returned), that variable can be handled differently in the model using the nominal and scale options in the clm function.




For more information on these models and the ordinal package, see:


•  Christensen, H.R.B.  2015a. Analysis of ordinal data with cumulative link models—estimation with the R-package ordinal. cran.r-project.org/web/packages/ordinal/vignettes/clm_intro.pdf.


library(ordinal); help(package="ordinal")


•  Wikipedia. 2015. “The model and the proportional odds assumption” in Ordered logit. en.wikipedia.org/wiki/Ordered_logit#The_model_and_the_proportional_odds_assumption.


•  IDRE . 2015. R Data Analysis Examples: Ordinal Logistic Regression. UCLA. www.ats.ucla.edu/stat/r/dae/ologit.htm.


•  Christensen, R.H.B. 2015b. Package ‘ordinal’. cran.r-project.org/web/packages/ordinal/ordinal.pdf.


•  Hervé, M. 2014. “72. Analyser des notes” in Aide-mémoire de statistique appliquée à la biologie. cran.r-project.org/doc/contrib/Herve-Aide-memoire-statistique.pdf.