Python, Statistics, Educational Measurement: Cohens kappa interrater agreement coefficient

Cohen's kappa coefficient is a measure of interrater agreement for qualitative or categorical variables. It is computed normally only for two raters or judges . Input data is usually a 2 by 2 table. Consider the following:

Yes No
Yes a b
No c d

and a specific example(from en.wikipedial):

20 5
10 15

The following Python program computes Cohen's kappa

?Download kappa.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
def kappa(a,b,c,d):
    tot = a + b + c + d
    Pa  = float(a + d)/tot
    PA1 = float(a + b)/tot
    PA2 = 1.0- PA1
    PB1 = float(a + c) /tot
    PB2 = 1.0 -PB1
    Pe  = PA1 *PB1 + PA2*PB2
    print Pa, PA1, PB1, PA2, PB2
    return (Pa -Pe)/ (1.0 -Pe)
 
def Test():
    a = 20
    b =  5
    c = 10
    d = 15
   print "kappa(20,5,10,15)=", kappa(a,b,c,d)
 
if __name__ == "__main__":
   Test()

When the above program runs it outputs 0.4 as it should.

A multi rater version of the Cohen's kappa is currently being worked on.

See Wikipedia for more details.

Comments are closed.