Python, Statistics, Educational Measurement: Cohens kappa interrater agreement coefficient

1 person like this

• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 

Cohen's kappa coefficient is a measure of interrater agreement for qualitative or categorical variables. It is computed normally only for two raters or judges . Input data is usually a 2 by 2 table. Consider the following:

and a specific example(from en.wikipedial):

The following Python program computes Cohen's kappa

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

def kappa(a,b,c,d):
    tot = a + b + c + d
    Pa  = float(a + d)/tot
    PA1 = float(a + b)/tot
    PA2 = 1.0- PA1
    PB1 = float(a + c) /tot
    PB2 = 1.0 -PB1
    Pe  = PA1 *PB1 + PA2*PB2
    print Pa, PA1, PB1, PA2, PB2
    return (Pa -Pe)/ (1.0 -Pe)
 
def Test():
    a = 20
    b =  5
    c = 10
    d = 15
   print "kappa(20,5,10,15)=", kappa(a,b,c,d)
 
if __name__ == "__main__":
   Test()

When the above program runs it outputs 0.4 as it should.

A multi rater version of the Cohen's kappa is currently being worked on.

See Wikipedia for more details.

This entry was posted on Monday, October 19th, 2009 at 1:37 am and is filed under Nonparametrics. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.