You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Phil Steitz (JIRA)" <ji...@apache.org> on 2011/06/12 20:19:51 UTC
[jira] [Created] (MATH-588) Weighted Mean evaluation may not have
optimal numerics
Weighted Mean evaluation may not have optimal numerics
------------------------------------------------------
Key: MATH-588
URL: https://issues.apache.org/jira/browse/MATH-588
Project: Commons Math
Issue Type: Bug
Affects Versions: 2.2, 2.1
Reporter: Phil Steitz
Fix For: 3.0
I recently got this in a test run
{code}
testWeightedConsistency(org.apache.commons.math.stat.descriptive.moment.MeanTest) Time elapsed: 0 sec <<< FAILURE!
java.lang.AssertionError: expected:<0.002282165958997601> but was:<0.002282165958997157>
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.failNotEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:441)
at org.apache.commons.math.TestUtils.assertRelativelyEquals(TestUtils.java:178)
at org.apache.commons.math.TestUtils.assertRelativelyEquals(TestUtils.java:153)
at org.apache.commons.math.stat.descriptive.UnivariateStatisticAbstractTest.testWeightedConsistency(UnivariateStatisticAbstractTest.java:170)
{code}
The correction formula used to compute the unweighted mean may not be appropriate or optimal in the presence of weights:
{code}
// Compute initial estimate using definitional formula
double sumw = sum.evaluate(weights,begin,length);
double xbarw = sum.evaluate(values, weights, begin, length) / sumw;
// Compute correction factor in second pass
double correction = 0;
for (int i = begin; i < begin + length; i++) {
correction += weights[i] * (values[i] - xbarw);
}
return xbarw + (correction/sumw);
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-588) Weighted Mean evaluation may not have
optimal numerics
Posted by "Thomas Neidhart (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200902#comment-13200902 ]
Thomas Neidhart commented on MATH-588:
--------------------------------------
Corrected the equality test in r1240795 as it was leading to failure. In fact the test can range from very small to very large values which really requires a relative error estimate.
The test is problematic in general, as it may contain values from very different scales (due to its random nature), leading to unavoidable precision errors in the above formula.
> Weighted Mean evaluation may not have optimal numerics
> ------------------------------------------------------
>
> Key: MATH-588
> URL: https://issues.apache.org/jira/browse/MATH-588
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 2.1, 2.2
> Reporter: Phil Steitz
> Fix For: 3.0
>
>
> I recently got this in a test run
> {code}
> testWeightedConsistency(org.apache.commons.math.stat.descriptive.moment.MeanTest) Time elapsed: 0 sec <<< FAILURE!
> java.lang.AssertionError: expected:<0.002282165958997601> but was:<0.002282165958997157>
> at org.junit.Assert.fail(Assert.java:91)
> at org.junit.Assert.failNotEquals(Assert.java:645)
> at org.junit.Assert.assertEquals(Assert.java:441)
> at org.apache.commons.math.TestUtils.assertRelativelyEquals(TestUtils.java:178)
> at org.apache.commons.math.TestUtils.assertRelativelyEquals(TestUtils.java:153)
> at org.apache.commons.math.stat.descriptive.UnivariateStatisticAbstractTest.testWeightedConsistency(UnivariateStatisticAbstractTest.java:170)
> {code}
> The correction formula used to compute the unweighted mean may not be appropriate or optimal in the presence of weights:
> {code}
> // Compute initial estimate using definitional formula
> double sumw = sum.evaluate(weights,begin,length);
> double xbarw = sum.evaluate(values, weights, begin, length) / sumw;
> // Compute correction factor in second pass
> double correction = 0;
> for (int i = begin; i < begin + length; i++) {
> correction += weights[i] * (values[i] - xbarw);
> }
> return xbarw + (correction/sumw);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MATH-588) Weighted Mean evaluation may not have
optimal numerics
Posted by "Thomas Neidhart (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Neidhart resolved MATH-588.
----------------------------------
Resolution: Fixed
Fixed it in r1240790.
There was a too strict equality test using an relative error of 10-14 which resulted in certain unforunate cases of an absolute error of 10-18.
> Weighted Mean evaluation may not have optimal numerics
> ------------------------------------------------------
>
> Key: MATH-588
> URL: https://issues.apache.org/jira/browse/MATH-588
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 2.1, 2.2
> Reporter: Phil Steitz
> Fix For: 3.0
>
>
> I recently got this in a test run
> {code}
> testWeightedConsistency(org.apache.commons.math.stat.descriptive.moment.MeanTest) Time elapsed: 0 sec <<< FAILURE!
> java.lang.AssertionError: expected:<0.002282165958997601> but was:<0.002282165958997157>
> at org.junit.Assert.fail(Assert.java:91)
> at org.junit.Assert.failNotEquals(Assert.java:645)
> at org.junit.Assert.assertEquals(Assert.java:441)
> at org.apache.commons.math.TestUtils.assertRelativelyEquals(TestUtils.java:178)
> at org.apache.commons.math.TestUtils.assertRelativelyEquals(TestUtils.java:153)
> at org.apache.commons.math.stat.descriptive.UnivariateStatisticAbstractTest.testWeightedConsistency(UnivariateStatisticAbstractTest.java:170)
> {code}
> The correction formula used to compute the unweighted mean may not be appropriate or optimal in the presence of weights:
> {code}
> // Compute initial estimate using definitional formula
> double sumw = sum.evaluate(weights,begin,length);
> double xbarw = sum.evaluate(values, weights, begin, length) / sumw;
> // Compute correction factor in second pass
> double correction = 0;
> for (int i = begin; i < begin + length; i++) {
> correction += weights[i] * (values[i] - xbarw);
> }
> return xbarw + (correction/sumw);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MATH-588) Weighted Mean evaluation may not have
optimal numerics
Posted by "Phil Steitz (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phil Steitz updated MATH-588:
-----------------------------
Assignee: (was: Phil Steitz)
> Weighted Mean evaluation may not have optimal numerics
> ------------------------------------------------------
>
> Key: MATH-588
> URL: https://issues.apache.org/jira/browse/MATH-588
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 2.1, 2.2
> Reporter: Phil Steitz
> Fix For: 3.0
>
>
> I recently got this in a test run
> {code}
> testWeightedConsistency(org.apache.commons.math.stat.descriptive.moment.MeanTest) Time elapsed: 0 sec <<< FAILURE!
> java.lang.AssertionError: expected:<0.002282165958997601> but was:<0.002282165958997157>
> at org.junit.Assert.fail(Assert.java:91)
> at org.junit.Assert.failNotEquals(Assert.java:645)
> at org.junit.Assert.assertEquals(Assert.java:441)
> at org.apache.commons.math.TestUtils.assertRelativelyEquals(TestUtils.java:178)
> at org.apache.commons.math.TestUtils.assertRelativelyEquals(TestUtils.java:153)
> at org.apache.commons.math.stat.descriptive.UnivariateStatisticAbstractTest.testWeightedConsistency(UnivariateStatisticAbstractTest.java:170)
> {code}
> The correction formula used to compute the unweighted mean may not be appropriate or optimal in the presence of weights:
> {code}
> // Compute initial estimate using definitional formula
> double sumw = sum.evaluate(weights,begin,length);
> double xbarw = sum.evaluate(values, weights, begin, length) / sumw;
> // Compute correction factor in second pass
> double correction = 0;
> for (int i = begin; i < begin + length; i++) {
> correction += weights[i] * (values[i] - xbarw);
> }
> return xbarw + (correction/sumw);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira