[jira] Issue Comment Edited: (MATH-520) Some unit tests fail in "CMAESOptimizerTest"

["Phil Steitz (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MATH-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12997139#comment-12997139 ] 

Phil Steitz edited comment on MATH-520 at 2/20/11 2:03 PM:
-----------------------------------------------------------

I agree that the tests should (eventually) behave completely deterministically.  My sloppiness in this led to annoying failures until we eventually fixed all of the non-deterministic tests that I had sprinkled through the statistics and random packages.  At the same time, as the code was stabilizing, these tests did turn up some real bugs that might not have been discovered if the tests were not allowed to run non-deterministically.  

I have no idea what is going on in this case, but I suspect there is a bug somewhere and the non-deterministic test may have helped us find it.  So I would advocate resolving the issue without just resorting to finding a "winning" seed and letting the tests run for a while non-deterministically until we get closer to the 3.0 release.

      was (Author: psteitz):
    I agree that the tests should (eventually) behave completely deterministically.  My sloppiness in this led to annoying failures until we eventually fixed all of the non=deterministic tests that I had sprinkled through the statistics and random packages.  At the same time, as the code was stabilizing, these tests did turn up some real bugs that might not have been discovered if the tests were not allowed to run non-deterministically.  

I have no idea what is going on in this case, but I suspect there is a bug somewhere and the non-deterministic test may have helped us find it.  So I would advocate resolving the issue without just resorting to finding a "winning" seed and letting the tests run for a while non-deterministically until we get closer to the 3.0 release.
  
> Some unit tests fail in "CMAESOptimizerTest"
> --------------------------------------------
>
>                 Key: MATH-520
>                 URL: https://issues.apache.org/jira/browse/MATH-520
>             Project: Commons Math
>          Issue Type: Task
>            Reporter: Gilles
>              Labels: test
>             Fix For: 3.0
>
>
> Out of 28 test runs, "CMAESOptimizerTest" failed 6 times. Those failures were sometimes in "testMaximize" and sometimes in "testDiagonalRosen".
> Is there some way to reduce the failure ratio?
> Anyways, it would probably be useful to introduce comments for those methods that are expected to fail sometimes.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira