The primary goal of this book is to show practitioners simple step-by-step approaches for organizing rating data, creating SAS datasets, and using appropriate SAS procedures,or special SAS macro programs to compute various inter-rater reliability coefficients. I always start with a brief and non-mathematical description of the agreement coefficients used in this book, before showing how they are calculated with SAS.

The FREQ procedure of SAS offers the calculation of Cohen’s Kappa as an option when the number of raters is limited to 2. But in addition to offering Kappa as the only agreement coefficient, the use of FREQ to compute Kappa is full of pitfalls that could easily lead a careless practitioner to wrong results. For example, if one rater does not use one category that another rater has used, SAS does not compute any Kappa at all. I refer to this problem in chapter 1 as the unbalanced-table issue. Even more seriously, if both raters use the same number of different categories, SAS will produce “very wrong” results, because the FREQ procedure will be matching wrong categories to determine agreement. I refer to this issue in chapter 1 as the “Diagonal Issue.” There are a few other potentially serious problems with weighted Kappa that I have noticed. I have clearly documented all of the problems, and propose a simple solutions to resolve them.

Dear customer, you are purchasing this copyrighted eBook with the understanding that it cannot be transmitted electronically, or by an information storage and retrieval system. You may print a single copy of the book, which will the sole hard copy in circulation at a given point in time.