Cohen's Kappa (κ)

Measures inter-rater agreement for categorical items:

$$\kappa = \frac{P_o - P_e}{1 - P_e}$$

Where:

• $P_o$ = observed agreement (proportion of matching ratings)
• $P_e$ = expected agreement by chance

$$P_o = \frac{\sum_{i} n_{ii}}{n}$$ $$P_e = \sum_{i} p_{i+} \times p_{+i}$$

Interpretation:

• κ < 0: Less than chance
• 0.01–0.20: Slight
• 0.21–0.40: Fair
• 0.41–0.60: Moderate
• 0.61–0.80: Substantial
• 0.81–1.00: Almost perfect