You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Michael Christopher <mi...@bluewin.ch> on 2014/05/25 17:05:38 UTC
Calculation of Kappa value
Hi
With the Confusion Matrix below I get a Kappa value of 0,8023. Actually
it should be 1, because there are no wrong classifications.
I debugged and found out that it happens, because the samples variable
is two times incremented in the "putCount" method
(org.apache.mahout.classifier.ConfusionMatrix). Commenting out this
incrementation lead to the Kappa value of 1.
Is this a bug or do I something something?
=======================================================
Confusion Matrix
-------------------------------------------------------
a b <--Classified as
6 0 | 6 a = 1_YES
0 9 | 9 b = 2_NO
=======================================================
Statistics
-------------------------------------------------------
Kappa 0,8023
Accuracy 100%
Reliability 66,6667%
Reliability (standard deviation) 0,5774
Re: Calculation of Kappa value
Posted by Michael Christopher <mi...@bluewin.ch>.
Thanks Sebastian
No, I downloaded the source code (version 0.9) from mirror. I am bit
busy at the moment. will try to work with the trunk and provide a patch
in about 2 weeks.
Regards
Michael
On 25.05.2014 17:08, Sebastian Schelter wrote:
> Could be a bug introduced by a recent modification of the Confusion
> matrix. Are you using trunk? Can you provide a patch that fixes the
> issue?
>
> Best,
> Sebastian
>
>
> On 05/25/2014 05:05 PM, Michael Christopher wrote:
>> Hi
>>
>> With the Confusion Matrix below I get a Kappa value of 0,8023. Actually
>> it should be 1, because there are no wrong classifications.
>>
>> I debugged and found out that it happens, because the samples variable
>> is two times incremented in the "putCount" method
>> (org.apache.mahout.classifier.ConfusionMatrix). Commenting out this
>> incrementation lead to the Kappa value of 1.
>>
>> Is this a bug or do I something something?
>>
>>
>> =======================================================
>> Confusion Matrix
>> -------------------------------------------------------
>> a b <--Classified as
>> 6 0 | 6 a = 1_YES
>> 0 9 | 9 b = 2_NO
>>
>> =======================================================
>> Statistics
>> -------------------------------------------------------
>> Kappa 0,8023
>> Accuracy 100%
>> Reliability 66,6667%
>> Reliability (standard deviation) 0,5774
>>
>>
>>
>>
>>
>
Re: Calculation of Kappa value
Posted by Sebastian Schelter <ss...@apache.org>.
Could be a bug introduced by a recent modification of the Confusion
matrix. Are you using trunk? Can you provide a patch that fixes the issue?
Best,
Sebastian
On 05/25/2014 05:05 PM, Michael Christopher wrote:
> Hi
>
> With the Confusion Matrix below I get a Kappa value of 0,8023. Actually
> it should be 1, because there are no wrong classifications.
>
> I debugged and found out that it happens, because the samples variable
> is two times incremented in the "putCount" method
> (org.apache.mahout.classifier.ConfusionMatrix). Commenting out this
> incrementation lead to the Kappa value of 1.
>
> Is this a bug or do I something something?
>
>
> =======================================================
> Confusion Matrix
> -------------------------------------------------------
> a b <--Classified as
> 6 0 | 6 a = 1_YES
> 0 9 | 9 b = 2_NO
>
> =======================================================
> Statistics
> -------------------------------------------------------
> Kappa 0,8023
> Accuracy 100%
> Reliability 66,6667%
> Reliability (standard deviation) 0,5774
>
>
>
>
>