You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@commons.apache.org by mi...@apache.org on 2012/06/08 13:04:11 UTC
svn commit: r1348024 - in /commons/proper/math/trunk/src:
main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
test/java/org/apache/commons/math3/stat/inference/MannWhitneyUTestTest.java
Author: mikl
Date: Fri Jun 8 11:04:11 2012
New Revision: 1348024
URL: http://svn.apache.org/viewvc?rev=1348024&view=rev
Log:
MATH-790: Patch applied to fix the overflow issue.
Modified:
commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
commons/proper/math/trunk/src/test/java/org/apache/commons/math3/stat/inference/MannWhitneyUTestTest.java
Modified: commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java?rev=1348024&r1=1348023&r2=1348024&view=diff
==============================================================================
--- commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java (original)
+++ commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java Fri Jun 8 11:04:11 2012
@@ -170,11 +170,11 @@ public class MannWhitneyUTest {
final int n2)
throws ConvergenceException, MaxCountExceededException {
- final int n1n2prod = n1 * n2;
+ final double n1n2prod = n1 * n2;
// http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
- final double EU = (double) n1n2prod / 2.0;
- final double VarU = (double) (n1n2prod * (n1 + n2 + 1)) / 12.0;
+ final double EU = n1n2prod / 2.0;
+ final double VarU = n1n2prod * (n1 + n2 + 1) / 12.0;
final double z = (Umin - EU) / FastMath.sqrt(VarU);
Modified: commons/proper/math/trunk/src/test/java/org/apache/commons/math3/stat/inference/MannWhitneyUTestTest.java
URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math3/stat/inference/MannWhitneyUTestTest.java?rev=1348024&r1=1348023&r2=1348024&view=diff
==============================================================================
--- commons/proper/math/trunk/src/test/java/org/apache/commons/math3/stat/inference/MannWhitneyUTestTest.java (original)
+++ commons/proper/math/trunk/src/test/java/org/apache/commons/math3/stat/inference/MannWhitneyUTestTest.java Fri Jun 8 11:04:11 2012
@@ -100,4 +100,16 @@ public class MannWhitneyUTestTest {
// expected
}
}
+
+ @Test
+ public void testBigDataSet() throws Exception {
+ double[] d1 = new double[1500];
+ double[] d2 = new double[1500];
+ for (int i = 0; i < 1500; i++) {
+ d1[i] = 2 * i;
+ d2[i] = 2 * i + 1;
+ }
+ double result = testStatistic.mannWhitneyUTest(d1, d2);
+ Assert.assertTrue(result > 0.1);
+ }
}
Re: svn commit: r1348024 - in /commons/proper/math/trunk/src:
main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java test/java/org/apache/commons/math3/stat/inference/MannWhitneyUTestTest.java
Posted by Mikkel Meyer Andersen <mi...@mikl.dk>.
2012/6/12 Thomas Neidhart <th...@gmail.com>:
> On Tue, Jun 12, 2012 at 8:13 AM, Mikkel Meyer Andersen <mi...@mikl.dk> wrote:
>
>> 2012/6/12 Thomas Neidhart <th...@gmail.com>:
>> > On 06/08/2012 01:04 PM, mikl@apache.org wrote:
>> >> Author: mikl
>> >> Date: Fri Jun 8 11:04:11 2012
>> >> New Revision: 1348024
>> >>
>> >> URL: http://svn.apache.org/viewvc?rev=1348024&view=rev
>> >> Log:
>> >> MATH-790: Patch applied to fix the overflow issue.
>> >>
>> >> Modified:
>> >>
>> commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
>> >>
>> commons/proper/math/trunk/src/test/java/org/apache/commons/math3/stat/inference/MannWhitneyUTestTest.java
>> >>
>> >> Modified:
>> commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
>> >> URL:
>> http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java?rev=1348024&r1=1348023&r2=1348024&view=diff
>> >>
>> ==============================================================================
>> >> ---
>> commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
>> (original)
>> >> +++
>> commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
>> Fri Jun 8 11:04:11 2012
>> >> @@ -170,11 +170,11 @@ public class MannWhitneyUTest {
>> >> final int n2)
>> >> throws ConvergenceException, MaxCountExceededException {
>> >>
>> >> - final int n1n2prod = n1 * n2;
>> >> + final double n1n2prod = n1 * n2;
>> >>
>> >> //
>> http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
>> >> - final double EU = (double) n1n2prod / 2.0;
>> >> - final double VarU = (double) (n1n2prod * (n1 + n2 + 1)) / 12.0;
>> >> + final double EU = n1n2prod / 2.0;
>> >> + final double VarU = n1n2prod * (n1 + n2 + 1) / 12.0;
>> >>
>> >> final double z = (Umin - EU) / FastMath.sqrt(VarU);
>> >
>> > just a small thing, but wouldn't it be better to do a long
>> > multiplication and convert the result to double?
>> >
>> > Thomas
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> > For additional commands, e-mail: dev-help@commons.apache.org
>> >
>> Do you mean for n1n2prod? Sorry for my ignorance, but what would that
>> help? Wouldn't that require more implicit conversions?
>>
>
> I just quickly checked to make sure I had the right understanding, but the
> line:
>
>>> + final double n1n2prod = n1 * n2;
>
> still does a int multiplication and may overflow (if n1 and n2 are too big).
>
> Maybe the overflow problem was not in this line but in this one:
>
>>> + final double VarU = n1n2prod * (n1 + n2 + 1) / 12.0;
>
> so the problem is hidden right now.
>
> Thomas
Thomas, thanks for this. I have reopened the issue,
https://issues.apache.org/jira/browse/MATH-790 . Would you be so kind
to repeat your argument in there, and we can continue our discussion
there?
Cheers, Mikkel.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: svn commit: r1348024 - in /commons/proper/math/trunk/src:
main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java test/java/org/apache/commons/math3/stat/inference/MannWhitneyUTestTest.java
Posted by Thomas Neidhart <th...@gmail.com>.
On Tue, Jun 12, 2012 at 8:13 AM, Mikkel Meyer Andersen <mi...@mikl.dk> wrote:
> 2012/6/12 Thomas Neidhart <th...@gmail.com>:
> > On 06/08/2012 01:04 PM, mikl@apache.org wrote:
> >> Author: mikl
> >> Date: Fri Jun 8 11:04:11 2012
> >> New Revision: 1348024
> >>
> >> URL: http://svn.apache.org/viewvc?rev=1348024&view=rev
> >> Log:
> >> MATH-790: Patch applied to fix the overflow issue.
> >>
> >> Modified:
> >>
> commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
> >>
> commons/proper/math/trunk/src/test/java/org/apache/commons/math3/stat/inference/MannWhitneyUTestTest.java
> >>
> >> Modified:
> commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
> >> URL:
> http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java?rev=1348024&r1=1348023&r2=1348024&view=diff
> >>
> ==============================================================================
> >> ---
> commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
> (original)
> >> +++
> commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
> Fri Jun 8 11:04:11 2012
> >> @@ -170,11 +170,11 @@ public class MannWhitneyUTest {
> >> final int n2)
> >> throws ConvergenceException, MaxCountExceededException {
> >>
> >> - final int n1n2prod = n1 * n2;
> >> + final double n1n2prod = n1 * n2;
> >>
> >> //
> http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
> >> - final double EU = (double) n1n2prod / 2.0;
> >> - final double VarU = (double) (n1n2prod * (n1 + n2 + 1)) / 12.0;
> >> + final double EU = n1n2prod / 2.0;
> >> + final double VarU = n1n2prod * (n1 + n2 + 1) / 12.0;
> >>
> >> final double z = (Umin - EU) / FastMath.sqrt(VarU);
> >
> > just a small thing, but wouldn't it be better to do a long
> > multiplication and convert the result to double?
> >
> > Thomas
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> Do you mean for n1n2prod? Sorry for my ignorance, but what would that
> help? Wouldn't that require more implicit conversions?
>
I just quickly checked to make sure I had the right understanding, but the
line:
>> + final double n1n2prod = n1 * n2;
still does a int multiplication and may overflow (if n1 and n2 are too big).
Maybe the overflow problem was not in this line but in this one:
>> + final double VarU = n1n2prod * (n1 + n2 + 1) / 12.0;
so the problem is hidden right now.
Thomas
Re: svn commit: r1348024 - in /commons/proper/math/trunk/src:
main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java test/java/org/apache/commons/math3/stat/inference/MannWhitneyUTestTest.java
Posted by Mikkel Meyer Andersen <mi...@mikl.dk>.
2012/6/12 Thomas Neidhart <th...@gmail.com>:
> On 06/08/2012 01:04 PM, mikl@apache.org wrote:
>> Author: mikl
>> Date: Fri Jun 8 11:04:11 2012
>> New Revision: 1348024
>>
>> URL: http://svn.apache.org/viewvc?rev=1348024&view=rev
>> Log:
>> MATH-790: Patch applied to fix the overflow issue.
>>
>> Modified:
>> commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
>> commons/proper/math/trunk/src/test/java/org/apache/commons/math3/stat/inference/MannWhitneyUTestTest.java
>>
>> Modified: commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java?rev=1348024&r1=1348023&r2=1348024&view=diff
>> ==============================================================================
>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java (original)
>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java Fri Jun 8 11:04:11 2012
>> @@ -170,11 +170,11 @@ public class MannWhitneyUTest {
>> final int n2)
>> throws ConvergenceException, MaxCountExceededException {
>>
>> - final int n1n2prod = n1 * n2;
>> + final double n1n2prod = n1 * n2;
>>
>> // http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
>> - final double EU = (double) n1n2prod / 2.0;
>> - final double VarU = (double) (n1n2prod * (n1 + n2 + 1)) / 12.0;
>> + final double EU = n1n2prod / 2.0;
>> + final double VarU = n1n2prod * (n1 + n2 + 1) / 12.0;
>>
>> final double z = (Umin - EU) / FastMath.sqrt(VarU);
>
> just a small thing, but wouldn't it be better to do a long
> multiplication and convert the result to double?
>
> Thomas
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
Do you mean for n1n2prod? Sorry for my ignorance, but what would that
help? Wouldn't that require more implicit conversions?
Cheers, Mikkel.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: svn commit: r1348024 - in /commons/proper/math/trunk/src: main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
test/java/org/apache/commons/math3/stat/inference/MannWhitneyUTestTest.java
Posted by Thomas Neidhart <th...@gmail.com>.
On 06/08/2012 01:04 PM, mikl@apache.org wrote:
> Author: mikl
> Date: Fri Jun 8 11:04:11 2012
> New Revision: 1348024
>
> URL: http://svn.apache.org/viewvc?rev=1348024&view=rev
> Log:
> MATH-790: Patch applied to fix the overflow issue.
>
> Modified:
> commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
> commons/proper/math/trunk/src/test/java/org/apache/commons/math3/stat/inference/MannWhitneyUTestTest.java
>
> Modified: commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java?rev=1348024&r1=1348023&r2=1348024&view=diff
> ==============================================================================
> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java (original)
> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java Fri Jun 8 11:04:11 2012
> @@ -170,11 +170,11 @@ public class MannWhitneyUTest {
> final int n2)
> throws ConvergenceException, MaxCountExceededException {
>
> - final int n1n2prod = n1 * n2;
> + final double n1n2prod = n1 * n2;
>
> // http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
> - final double EU = (double) n1n2prod / 2.0;
> - final double VarU = (double) (n1n2prod * (n1 + n2 + 1)) / 12.0;
> + final double EU = n1n2prod / 2.0;
> + final double VarU = n1n2prod * (n1 + n2 + 1) / 12.0;
>
> final double z = (Umin - EU) / FastMath.sqrt(VarU);
just a small thing, but wouldn't it be better to do a long
multiplication and convert the result to double?
Thomas
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org