NOTE: Please visit the Errata web page of this book here for errors found and corrections made.
The third edition of this book was very well received by researchers across a wide range of disciplines. Its use also gave researchers opportunities to raise important questions and identify the need for additional material on techniques that remain inadequately addressed in the literature. For example, researchers designing inter-rater reliability studies frequently wanted to know how to determine the optimal numbers of raters and subjects to include in their experiments. In addition, relatively little attention has been paid to intra-rater reliability, particularly when ratings involve quantitative measurements. The fourth edition addresses these needs while further refining and expanding the material presented in the third edition.
The methods and techniques described in this edition of the Handbook of Inter-Rater Reliability can handle missing ratings, which are common in reliability studies. This represents an important improvement over the second edition, in which the methods were limited to complete datasets. Parts II and III include new chapters designed to provide researchers with broader coverage of inter-rater reliability methods. Even chance-corrected agreement coefficients, which were addressed in the second edition, are presented here in greater depth and with improved clarity.
Features of the Fourth Edition include:
1) New material on sample-size calculations for chance-corrected agreement coefficients and intraclass correlation coefficients. Researchers will be able to determine the optimal numbers of raters, subjects, and trials per subject.
2) A completely rewritten chapter entitled “Benchmarking Inter-Rater Reliability Coefficients.”
3) A substantially expanded introductory chapter that explores alternative definitions and interpretations of inter-rater reliability.
4) Extensive revisions to all chapters to improve clarity, organization, and readability.
I expect the Handbook of Inter-Rater Reliability to serve as an essential reference for researchers, students, and practitioners seeking to assess inter-rater reliability across a broad range of disciplines.