You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by rado tzvetkov <rt...@yahoo.com> on 2012/10/09 13:53:49 UTC
[math] G-Tests in math.stat.inference
Need for additional statistics hypothesis Tests in math.stat.inference
1. Implementation of G-Test (log-likelihood ratio LLR test for independence and goodnes-of-fit)
2. Reference: http://en.wikipedia.org/wiki/G-test
3.Reasons-Usefulness: G-testsare tests that are increasingly being used in situations where chi-squared tests were previously recommended. The approximation to the theoretical chi-squared distribution for the G-test is better than for the Pearson chi-squared tests. In cases whereObserved >2*Expected for some cell case, the G-test is always better than the chi-squared test.
For testing goodness-of-fit the G-test is infinitely more efficient than the chi squared test in the sense of Bahadur, but the two tests are equally efficient in the sense of Pitman or in the sense of Hodge and Lehman.
I estimate the implementation effort as relatively small as the implementation differs only on few lines with the existing Chi-square tests.
Also I already have code to contribute and tests for G-Test for independence. (if needed) apache mahout also has some code implemented for LLR
best regards
Rado
P.S. Enhancing math.stat.inference with additional test could be generally good.
Re: [math] G-Tests in math.stat.inference
Posted by Phil Steitz <ph...@gmail.com>.
Thanks, Rado. You are most welcome to open a JIRA ticket [1] and
attach code with references there. I agree this would be a useful
addition to [math]. Thanks in advance.
Phil
[1] http://commons.apache.org/math/developers.html
On 10/9/12 4:53 AM, rado tzvetkov wrote:
> Need for additional statistics hypothesis Tests in math.stat.inference
>
> 1. Implementation of G-Test (log-likelihood ratio LLR test for independence and goodnes-of-fit)
>
> 2. Reference: http://en.wikipedia.org/wiki/G-test
>
> 3.Reasons-Usefulness: G-testsare tests that are increasingly being used in situations where chi-squared tests were previously recommended. The approximation to the theoretical chi-squared distribution for the G-test is better than for the Pearson chi-squared tests. In cases whereObserved >2*Expected for some cell case, the G-test is always better than the chi-squared test.
> For testing goodness-of-fit the G-test is infinitely more efficient than the chi squared test in the sense of Bahadur, but the two tests are equally efficient in the sense of Pitman or in the sense of Hodge and Lehman.
>
>
> I estimate the implementation effort as relatively small as the implementation differs only on few lines with the existing Chi-square tests.
>
> Also I already have code to contribute and tests for G-Test for independence. (if needed) apache mahout also has some code implemented for LLR
>
>
> best regards
>
> Rado
>
> P.S. Enhancing math.stat.inference with additional test could be generally good.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] G-Tests in math.stat.inference
Posted by Phil Steitz <ph...@gmail.com>.
On 10/11/12 4:04 AM, rado tzvetkov wrote:
> Please find,
>
> the implementation Patch with the Source, Tests and Comments attached to the
>
> https://issues.apache.org/jira/browse/MATH-878
Thanks!
Phil
>
> br
>
> Rado
>
>
>
> ________________________________
> From: Ted Dunning <te...@gmail.com>
> To: rado tzvetkov <rt...@yahoo.com>
> Sent: Wednesday, October 10, 2012 4:34 PM
> Subject: Re: [math] G-Tests in math.stat.inference
>
>
> And the signed root version is indeed not suitable for more than the 2x2 case.
>
> All you need for the general case is element by element function application, row and column sums and overall sum.
>
> Sent from my iPhone
>
> On Oct 10, 2012, at 4:30 AM, rado tzvetkov <rt...@yahoo.com> wrote:
>
>
>
>> Mahout don't have functionality for real cases (it's only [2x2] ) which is suitable for language study or similar but not in biology
>>
>>
>>
>>
>> ________________________________
>> From: Ted Dunning <te...@gmail.com>
>> To: Commons Developers List <de...@commons.apache.org>; rado tzvetkov <rt...@yahoo.com>
>> Sent: Tuesday, October 9, 2012 5:37 PM
>> Subject: Re: [math] G-Tests in math.stat.inference
>>
>>
>> Feel free to grab and adapt the Mahout code. It has some added wrinkles for convenience like the signed square root variant of the G-test.
>>
>>
>> On Tue, Oct 9, 2012 at 12:53 PM, rado tzvetkov <rt...@yahoo.com> wrote:
>>
>> Also I already have code to contribute and tests for G-Test for independence. (if needed) apache mahout also has some code implemented for LLR
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] G-Tests in math.stat.inference
Posted by rado tzvetkov <rt...@yahoo.com>.
Please find,
the implementation Patch with the Source, Tests and Comments attached to the
https://issues.apache.org/jira/browse/MATH-878
br
Rado
________________________________
From: Ted Dunning <te...@gmail.com>
To: rado tzvetkov <rt...@yahoo.com>
Sent: Wednesday, October 10, 2012 4:34 PM
Subject: Re: [math] G-Tests in math.stat.inference
And the signed root version is indeed not suitable for more than the 2x2 case.
All you need for the general case is element by element function application, row and column sums and overall sum.
Sent from my iPhone
On Oct 10, 2012, at 4:30 AM, rado tzvetkov <rt...@yahoo.com> wrote:
>
>Mahout don't have functionality for real cases (it's only [2x2] ) which is suitable for language study or similar but not in biology
>
>
>
>
>________________________________
> From: Ted Dunning <te...@gmail.com>
>To: Commons Developers List <de...@commons.apache.org>; rado tzvetkov <rt...@yahoo.com>
>Sent: Tuesday, October 9, 2012 5:37 PM
>Subject: Re: [math] G-Tests in math.stat.inference
>
>
>Feel free to grab and adapt the Mahout code. It has some added wrinkles for convenience like the signed square root variant of the G-test.
>
>
>On Tue, Oct 9, 2012 at 12:53 PM, rado tzvetkov <rt...@yahoo.com> wrote:
>
>Also I already have code to contribute and tests for G-Test for independence. (if needed) apache mahout also has some code implemented for LLR
>>
>
>
>
Re: [math] G-Tests in math.stat.inference
Posted by Ted Dunning <te...@gmail.com>.
Feel free to grab and adapt the Mahout code. It has some added wrinkles
for convenience like the signed square root variant of the G-test.
On Tue, Oct 9, 2012 at 12:53 PM, rado tzvetkov <rt...@yahoo.com> wrote:
> Also I already have code to contribute and tests for G-Test for
> independence. (if needed) apache mahout also has some code implemented for
> LLR
>