MH Program: Marginal Homogeneity Tests for N x N Tables Version 1.2 - John Uebersax 2017-02-08 7:11 PM ***INPUT*** Example of testing zero-division possibility 3 categories Rater 1 is row variable Rater 2 is column variable ordered categories 30 0 0 0 10 2 0 4 8 Total number of cases: 54 ***BASIC TESTS*** Four-fold tables tested 30 0 0 24 10 2 4 38 8 4 2 40 McNemar Tests for Each Category --------------------------------------------------------------------- Proportion Frequency (Base Rate) Level ---------------- ---------------- Chi- (k) Rater 1 Rater 2 Rater 1 Rater 2 squared(a) p --------------------------------------------------------------------- 1 30 30 0.556 0.556 exact test 1.0000 2 12 14 0.222 0.259 exact test 0.6875 3 12 10 0.222 0.185 exact test 0.6875 --------------------------------------------------------------------- (a) or exact test Tests of Overall Marginal Homogeneity ------------------------------------------------------------ 1 category with perfect agreement was not considered. Bhapkar Test Chi-squared = 0.686 df (conservative) = 2 p = 0.710 df (nonconservative) = 1 p = 0.408 Stuart-Maxwell Test Chi-squared = 0.667 df (conservative) = 2 p = 0.717 df (nonconservative) = 1 p = 0.414 Bowker Symmetry Test ---------------------------------------------- Chi-squared = 0.667 df = 3 p = 0.8810 ***TESTS FOR ORDERED-CATEGORY DATA*** McNemar Test of Overall Bias or Direction of Change -------------------------------------------- Cases where Rater 1 level is higher: 4 Cases where Rater 2 level is higher: 2 p = 0.6875 (exact test) Four-fold tables tested (for thresholds tests) 30 0 0 24 40 2 4 8 Tests of Equal Category Thresholds --------------------------------------------------------------------- Proportion of cases below level k Threshold(a) Level ---------------- ---------------- Chi- (k) Rater 1 Rater 2 Rater 1 Rater 2 squared(b) p --------------------------------------------------------------------- 2 0.556 0.556 0.140 0.140 exact test 1.0000 3 0.778 0.815 0.765 0.896 exact test 0.6875 --------------------------------------------------------------------- (a) for probit model (b) or exact test ***GRAPHIC OUTPUT*** Marginal Distributions of Categories for Rater 1 (**) and Rater 2 (==) 0.556 + ** == | ** == | ** == | ** == | ** == | ** == == | ** == ** == ** | ** == ** == ** == | ** == ** == ** == | ** == ** == ** == 0 +----+-------+-------+---- 1 2 3 Notes: x-axis is category number or level. y-axis is proportion of cases. Proportion of cases below each level 1 2 3 |--------------------------------|-------------|------------- Rater 1 |--------------------------------|---------------|----------- Rater 2 1 2 3 +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ Scale 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1. Category thresholds (probit model) 2 3 -------------------------------|------|---------------------- Rater 1 -------------------------------|-------|--------------------- Rater 2 2 3 +---------+---------+---------+---------+---------+---------+ Scale -3 -2 -1 0 1 2 3 (c) 2000-2006 John Uebersax