You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Gilles Sadowski <gi...@harfang.homelinux.org> on 2012/07/23 00:44:21 UTC

[Math] Random tests sometimes fail (r1363105)

Hi Thomas.

I've encountered a failure of one the tests which you introduced in
revision 1363105.  Here is an excerpt of the error:
---
testNormalDistributionUnsymmetricMatrix(org.apache.commons.math3.linear.EigenDecompositionTest)
Time elapsed: 0.044 sec  <<< FAILURE!
java.lang.AssertionError: The norm of (X-Y) is too large
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.commons.math3.linear.EigenDecompositionTest.checkUnsymmetricMatrix(EigenDecompositionTest.java:422)
        at org.apache.commons.math3.linear.EigenDecompositionTest.testNormalDistributionUnsymmetricMatrix(EigenDecompositionTest.java:404)
[... etc ...]
---

For tests that uses a RNG, the conclusion was reached that it is better (for
unit tests) to select a seed for which the test succeeds.


Regards,
Gilles

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [Math] Random tests sometimes fail (r1363105)

Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
Hello.

> >
> > I've encountered a failure of one the tests which you introduced in
> > revision 1363105.  Here is an excerpt of the error:
> > ---
> >
> > testNormalDistributionUnsymmetricMatrix(org.apache.commons.math3.linear.EigenDecompositionTest)
> > Time elapsed: 0.044 sec  <<< FAILURE!
> > java.lang.AssertionError: The norm of (X-Y) is too large
> >         at org.junit.Assert.fail(Assert.java:93)
> >         at org.junit.Assert.assertTrue(Assert.java:43)
> >         at
> > org.apache.commons.math3.linear.EigenDecompositionTest.checkUnsymmetricMatrix(EigenDecompositionTest.java:422)
> >         at
> > org.apache.commons.math3.linear.EigenDecompositionTest.testNormalDistributionUnsymmetricMatrix(EigenDecompositionTest.java:404)
> > [... etc ...]
> > ---
> >
> > For tests that uses a RNG, the conclusion was reached that it is better
> > (for
> > unit tests) to select a seed for which the test succeeds.
> >
> 
> ok thanks for the hint. The random test principle is flawed but I wanted to
> check with lots of different input data if the whole algorithm is stable.
> Now there was not yet a way to quickly display the corresponding matrix,
> but I added a RealMatrixFormat for that purpose. So I will update the test
> and use also a fixed seed to make it predictable (and do more investigation
> on failing cases).

Maybe that we should create an informal set of validation tests in the same
way that there are performance tests. I.e. such classes would have a name
that ends with "...TestValidation".
Not ending in "...Test", they would not be run automatically. [Cf. the
"FastMathTestPerformance" class.]


Regards,
Gilles

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [Math] Random tests sometimes fail (r1363105)

Posted by Thomas Neidhart <th...@gmail.com>.
On Mon, Jul 23, 2012 at 12:44 AM, Gilles Sadowski <
gilles@harfang.homelinux.org> wrote:

> Hi Thomas.
>

Hi Gilles,


>
> I've encountered a failure of one the tests which you introduced in
> revision 1363105.  Here is an excerpt of the error:
> ---
>
> testNormalDistributionUnsymmetricMatrix(org.apache.commons.math3.linear.EigenDecompositionTest)
> Time elapsed: 0.044 sec  <<< FAILURE!
> java.lang.AssertionError: The norm of (X-Y) is too large
>         at org.junit.Assert.fail(Assert.java:93)
>         at org.junit.Assert.assertTrue(Assert.java:43)
>         at
> org.apache.commons.math3.linear.EigenDecompositionTest.checkUnsymmetricMatrix(EigenDecompositionTest.java:422)
>         at
> org.apache.commons.math3.linear.EigenDecompositionTest.testNormalDistributionUnsymmetricMatrix(EigenDecompositionTest.java:404)
> [... etc ...]
> ---
>
> For tests that uses a RNG, the conclusion was reached that it is better
> (for
> unit tests) to select a seed for which the test succeeds.
>

ok thanks for the hint. The random test principle is flawed but I wanted to
check with lots of different input data if the whole algorithm is stable.
Now there was not yet a way to quickly display the corresponding matrix,
but I added a RealMatrixFormat for that purpose. So I will update the test
and use also a fixed seed to make it predictable (and do more investigation
on failing cases).

Thomas

RE: [Math] Random tests sometimes fail (r1363105)

Posted by "Becksfort, Jared" <Ja...@STJUDE.ORG>.
That is a good idea.  I will add that to the multivariate normal distribution unit tests that I recently sent.
________________________________________
From: Gilles Sadowski [gilles@harfang.homelinux.org]
Sent: Sunday, July 22, 2012 5:44 PM
To: dev@commons.apache.org
Subject: [Math] Random tests sometimes fail (r1363105)

Hi Thomas.

I've encountered a failure of one the tests which you introduced in
revision 1363105.  Here is an excerpt of the error:
---
testNormalDistributionUnsymmetricMatrix(org.apache.commons.math3.linear.EigenDecompositionTest)
Time elapsed: 0.044 sec  <<< FAILURE!
java.lang.AssertionError: The norm of (X-Y) is too large
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.commons.math3.linear.EigenDecompositionTest.checkUnsymmetricMatrix(EigenDecompositionTest.java:422)
        at org.apache.commons.math3.linear.EigenDecompositionTest.testNormalDistributionUnsymmetricMatrix(EigenDecompositionTest.java:404)
[... etc ...]
---

For tests that uses a RNG, the conclusion was reached that it is better (for
unit tests) to select a seed for which the test succeeds.


Regards,
Gilles

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org



Email Disclaimer:  www.stjude.org/emaildisclaimer
Consultation Disclaimer:  www.stjude.org/consultationdisclaimer


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org