Prevalence of "correct" labels in ground truth
50%
Percent agreement
95.0%
Cohen's κ
0.90
0
0.5
1.0
50%
75%
99%
prevalence of "correct" label
% agreement
Cohen's κ
Balanced classes.
Kappa tracks percent agreement closely when both labels are common.